Self-Propagating Worm Created to Target Generative AI Systems
Summary:
Researchers have developed a self-propagating worm dubbed “Morris II” specifically designed to target generative AI systems. This worm aims to exploit vulnerabilities within GenAI ecosystems, potentially spreading malware and extracting personal data. The research paper outlines how the worm utilizes adversarial self-replicating prompts to manipulate GenAI models into carrying out malicious activities. Unlike traditional malware, Morris II operates “passively” moving to new targets without requiring further action from the attackers, a technique referred to as “0-click propagation.”
The worm primarily targets GenAI-powered email assistants equipped with auto response functionality. By crafting messages containing adversarial self-replicating prompts, attackers prompt GenAI models to generate responses that unknowingly engage in malicious activities. These activities may include spamming or exfiltrating sensitive user data, such as email addresses and phone numbers, extracted from the context provide in the query.
Security Officer Comments:
The study evaluates the effectiveness of Morris II against various GenAI models, and open source large language models. The researchers assess the worm’s capabilities in terms of carrying out malicious activities and spreading to new hosts. Researchers warn that this approach could potentially lead to cyber attacks targeting the entire GenAI ecosystem.
Suggested Corrections:
To mitigate this threat recommendations include:
https://www.infosecurity-magazine.com/news/worm-created-generative-ai-systems/
Researchers have developed a self-propagating worm dubbed “Morris II” specifically designed to target generative AI systems. This worm aims to exploit vulnerabilities within GenAI ecosystems, potentially spreading malware and extracting personal data. The research paper outlines how the worm utilizes adversarial self-replicating prompts to manipulate GenAI models into carrying out malicious activities. Unlike traditional malware, Morris II operates “passively” moving to new targets without requiring further action from the attackers, a technique referred to as “0-click propagation.”
The worm primarily targets GenAI-powered email assistants equipped with auto response functionality. By crafting messages containing adversarial self-replicating prompts, attackers prompt GenAI models to generate responses that unknowingly engage in malicious activities. These activities may include spamming or exfiltrating sensitive user data, such as email addresses and phone numbers, extracted from the context provide in the query.
Security Officer Comments:
The study evaluates the effectiveness of Morris II against various GenAI models, and open source large language models. The researchers assess the worm’s capabilities in terms of carrying out malicious activities and spreading to new hosts. Researchers warn that this approach could potentially lead to cyber attacks targeting the entire GenAI ecosystem.
Suggested Corrections:
To mitigate this threat recommendations include:
- Rephrase the entire output in GenAI models to ensure the output does not consist of pieces that are similar to the input and do not yield the same inference
- Implement countermeasures against jailbreaking to prevent attackers from using known techniques to replicate the input into the output
- Use techniques developed to detect malicious propagation patterns associated with computer worms. For the RAG-based worm, the easiest method is to use a non-active RAG
https://www.infosecurity-magazine.com/news/worm-created-generative-ai-systems/