November 21, 2026

AI is finding more security flaws than ever, but someone still has to check if they’re real

pr_asset_ai_pentesting_survey_branded

AI tools are transforming how companies test their own systems for security weaknesses, but a new survey suggests the technology may be creating almost as many problems as it solves.

Pentest-Tools.com, a Bucharest-based cybersecurity company, surveyed 158 security professionals in June 2026 about how they use AI in penetration testing, the process of deliberately probing a company’s systems to find weaknesses before criminals do.

The results suggest AI has made one part of the job much faster, discovering potential vulnerabilities, while making another part of the job harder: working out which of those “discoveries” are actually real.

Nearly 9 in 10 practitioners (87.8%) who had used AI tools to generate findings said the results needed significant manual checking before they could be trusted. Roughly a quarter said more than 25% of what the AI produced turned out to be unusable.

The problem gets worse at scale. Modern AI tools can generate hundreds of potential vulnerabilities from a single scan. But when asked whether their team could actually process 500 or more AI-flagged issues from one engagement, only one in five practitioners (20.3%) said they had a system in place to do it. Nearly 3 in 10 (29.7%) said that volume of findings would simply be unmanageable.

One respondent’s account, quoted in the survey, captured the frustration succinctly: an AI tool produced 300 findings from a single test, but after two days spent checking them, 250 turned out to be false alarms, duplicate issues, or vulnerabilities that did not actually exist. “I bought the tool to save time,” the respondent said, “but I did more manual work than before.”

Fabricated results, sometimes described by respondents as AI “hallucinations,” were named as the single biggest frustration with AI pentesting tools, cited far more often than cost or software glitches. A security manager at a mid-market company described the effect on trust bluntly, calling it “confidence that turns out to be just a big lie.”

The research also points to a specific blind spot in what AI tools can currently catch: business logic, meaning flaws in how an application is designed to behave, rather than bugs in its code. Security professionals gave examples such as a discount code that should only work once per customer, a shopping cart that allows negative quantities to be entered (effectively giving money back to the customer), or web addresses that expose another user’s private data simply by changing a number in the URL. None of these trigger an error message or an obvious code fault, which is part of why they are difficult for AI systems to catch and why human testers say they remain essential to the process.

At the same time, security teams are increasingly being asked to test AI systems themselves. The survey found that 75.3% of practitioners already test AI-powered or LLM-integrated applications as standard practice, with a further 17.1% planning to start within the next year, a combined total of 92.4%. More than a third (37.3%) said that internal stakeholders are now demanding more frequent security testing than they did a year ago, driven by growing awareness of AI-related attack risks.

Interestingly, when companies choose which AI security tools to buy, accuracy matters more to them than price. 63% of respondents ranked false positive rate and signal quality as their top consideration when evaluating AI pentesting software, ahead of proof that a vulnerability is genuinely exploitable (53%), with cost coming in third at 47%.

A Pentest-Tools.com spokesperson said: “AI is speeding up what practitioners can find. The issue is in what follows after that. When you have 300 findings from a tool and 250 of them are invalid, the time savings in discovery are spent on triage. The value of AI in pentesting lies in actionable findings.”

The findings arrive as UK and global businesses continue to invest heavily in AI-powered security tools, often on the promise that automation will reduce the workload on stretched security teams. This research suggests that, for now at least, AI is shifting where that workload sits rather than removing it altogether, moving effort away from finding vulnerabilities and toward proving which ones are genuine.