Adversaries can Craft Adversarial Data that prevent a machine learning model from correctly identifying the contents of the data. This technique can be used to evade a downstream task where machine learning is utilized. The adversary may evade machine learning based virus/malware detection, or network scanning towards the goal of a traditional cyber attack.
LLM Prompt Injections and Jailbreaks are also techniques to evade defenses
Defense Evasion
The adversary is trying to avoid being detected by machine learning-enabled security software.
Defense Evasion consists of techniques that adversaries use to avoid detection throughout their compromise. Techniques used for defense evasion include evading ML-enabled security software such as malware detectors.