Exfilteration

• Exfiltration

  • The adversary is trying to steal machine learning artifacts or other information about the machine learning system.
  • Exfiltration consists of techniques that adversaries may use to steal data from your network. Data may be stolen for it's valuable intellectual property, or for use in staging future operations.
  • Techniques for getting data out of a target network typically include transferring it over their command and control channel or an alternate channel and may also include putting size limits on the transmission.
• Exfiltration via ML Inference API

  • Adversaries may exfiltrate private information via ML Model Inference API Access. ML Models have been shown leak private information about their training data (e.g. Infer Training Data Membership, Invert ML Model). The model itself may also be extracted (Extract ML Model) for the purposes of ML Intellectual Property Theft.
  • Exfiltration of information relating to private training data raises privacy concerns. Private training data may include personally identifiable information, or other protected data.
• Exfiltration via ML Inference API: Infer Training Data Membership

  • Adversaries may infer the membership of a data sample in its training set, which raises privacy concerns. Some strategies make use of a shadow model that could be obtained via Train Proxy via Replication, others use statistics of model prediction scores.
  • This can cause the victim model to leak private information, such as PII of those in the training set or other forms of protected IP.
• Exfiltration via ML Inference API: Invert ML Model

  • Machine learning models' training data could be reconstructed by exploiting the confidence scores that are available via an inference API. By querying the inference API strategically, adversaries can back out potentially private information embedded within the training data. This could lead to privacy violations if the attacker can reconstruct the data of sensitive features used in the algorithm.
• Inference API is a type of API that allows users to make predictions using pre-trained machine-learning models. It is a crucial component in the deployment of machine learning models for real-time predictions and decision-making.
• Exfiltration via ML Inference API: Extract ML Model

  • Adversaries may extract a functional copy of a private model. By repeatedly querying the victim's ML Model Inference API Access, the adversary can collect the target model's inferences into a dataset. The inferences are used as labels for training a separate model offline that will mimic the behavior and performance of the target model.
  • Adversaries may extract the model to avoid paying per query in a machine learning as a service setting. Model extraction is used for ML Intellectual Property Theft.
• Machine learning inference is the process of running data points into a machine learning model to calculate an output such as a single numerical score. This process is also referred to as "operationalizing a machine learning model" or "putting a machine learning model into production."
• Exfiltration via Cyber Means

  • Adversaries may exfiltrate ML artifacts or other information relevant to their goals via traditional cyber means.
• Exfilteration is also possible with LLM Meta Prompt Extraction
• LLM Data Leakage

  • Adversaries may craft prompts that induce the LLM to leak sensitive information. This can include private user data or proprietary information. The leaked information may come from proprietary training data, data sources the LLM is connected to, or information from other users of the LLM.