We are entering the greatest window of opportunity for criminals and malicious actors since the creation of the Internet. Carnegie Mellon researchers have shown how AI systems can autonomously plan and execute cyberattacks against enterprise-grade defenses. Cybercriminals are already using AI systems to attack consumers with phishing, fraud, and other types of social engineering attacks in increasing numbers and convincing sophistication.

Threat actors can rapidly iterate and adopt emerging technologies, such as powerful AI tools for cyberattacks, with little regard for testing or failures. They can move much faster and reap the benefits in pursuit of their malicious goals than cybersecurity defenders can embrace effective countermeasures. The speed advantage creates a window of opportunity for attackers to victimize targets at a greater scale and effectiveness.

The resulting disruption, impacts and pain will create strong incentives for innovators to develop robust capabilities that subdue the specific advantages to restore balance and counteract the evolving threats.

CMU Article: https://engineering.cmu.edu/news-events/news/2025/07/24-when-llms-autonomously-attack.html

Download Research Paper: https://arxiv.org/abs/2501.16466