Impact

• Evade ML Model

  • Adversaries can Craft Adversarial Data that prevent a machine learning model from correctly identifying the contents of the data. This technique can be used to evade a downstream task where machine learning is utilized. The adversary may evade machine learning based virus/malware detection, or network scanning towards the goal of a traditional cyber attack.
• Denial of ML Service

  • Adversaries may target machine learning systems with a flood of requests for the purpose of degrading or shutting down the service. Since many machine learning systems require significant amounts of specialized compute, they are often expensive bottlenecks that can become overloaded. Adversaries can intentionally craft inputs that require heavy amounts of useless compute from the machine learning system.
• Spamming ML System with Chaff Data

  • Adversaries may spam the machine learning system with chaff data that causes increase in the number of detections. This can cause analysts at the victim organization to waste time reviewing and correcting incorrect inferences.
• Erode ML Model Integrity

  • Adversaries may degrade the target model's performance with adversarial data inputs to erode confidence in the system over time. This can lead to the victim organization wasting time and money both attempting to fix the system and performing the tasks it was meant to automate by hand.
• Cost Harvesting

  • Adversaries may target different machine learning services to send useless queries or computationally expensive inputs to increase the cost of running services at the victim organization. Sponge examples are a particular type of adversarial data designed to maximize energy consumption and thus operating cost.
• External Harms

  • Adversaries may abuse their access to a victim system and use its resources or capabilities to further their goals by causing harms external to that system. These harms could affect the organization (e.g. Financial Harm, Reputational Harm), its users (e.g. User Harm), or the general public (e.g. Societal Harm).
• External Harms: Financial Harm

  • Financial harm involves the loss of wealth, property, or other monetary assets due to theft, fraud or forgery, or pressure to provide financial resources to the adversary.
• External Harms: Reputational Harm

  • Reputational harm involves a degradation of public perception and trust in organizations. Examples of reputation-harming incidents include scandals or false impersonations.
• External Harms: Societal Harm

  • Societal harms might generate harmful outcomes that reach either the general public or specific vulnerable groups such as the exposure of children to vulgar content.
• External Harms: User Harm

  • User harms may encompass a variety of harm types including financial and reputational that are directed at or felt by individual victims of the attack rather than at the organization level.
• External Harms: ML Intellectual Property Theft

  • Adversaries may exfiltrate ML artifacts to steal intellectual property and cause economic harm to the victim organization.
  • Proprietary training data is costly to collect and annotate and may be a target for Exfiltration and theft.
  • MLaaS providers charge for use of their API. An adversary who has stolen a model via Exfiltration or via Extract ML Model now has unlimited use of that service without paying the owner of the intellectual property.