Artificial Exploits, Real Limitations: How AI Cyber Attacks Fall Short
New Forescout Vedere Labs research has been released that takes a look at the real-world threat of generative AI in cyberattacks. While concerns are rising over so-called “vibe hacking”, a term used to describe AI-driven autonomous hacking, the findings from over 50 simulations of AI models suggest the threat is still emerging, not imminent.
The research evaluates the effectiveness of leading large language models (LLMs) in performing tasks related to vulnerability research (VR) and exploit development (ED). The study spans commercial, open-source, and underground AI models.
Key findings include:
- High Failure Rates: 48% of models failed the first VR task, 55% failed the second; 66% failed the first ED task, and a staggering 93% failed the second.
- Instability and Inconsistency: Most models were unreliable, often producing different outputs across identical prompts, experiencing timeouts, or generating unusable results.
- No End-to-End Threat: No single model was capable of completing all required stages of an exploit pipeline, limiting real-world threat viability today.
- Commercial vs. Underground Tools: Commercial models performed the best, but only three produced a working exploit. Open-source tools struggled the most, while free underground models performed better but suffered from usability and output issues.
Michele Campobasso, Forescout’s Senior Security Researcher for EMEA, “This current wave of AI popularisation, AI-assisted or not, threat actors are likely to continue relying on familiar tactics, techniques, and procedures (TTPs). An AI-generated exploit is still just an exploit; it can be detected, blocked, or mitigated by patching.
This means that the fundamentals of cybersecurity remain unchanged. Core principles such as cyber hygiene, defense-in-depth, least privilege, network segmentation and Zero Trust are still critical. Investments in risk and exposure management, network security and threat detection and response remain not only valid, but more urgent than ever.
If AI lowers the barrier to launching attacks, we may see them become more frequent, but not necessarily more sophisticated. Rather than reinventing defensive strategies, organisations should focus on enforcing them more dynamically and effectively across all environments.”
The full blog is available here.