New Study from Cloud Security Alliance Finds AI Improves Analyst Accuracy, Speed, and Consistency in Security Investigations

Security operations center (SOC) analysts assisted by AI are faster and more accurate compared to counterparts working manually

SEATTLE – Oct. 7, 2025 – Beyond the Hype: A Benchmark Study of AI in the SOC, a new report from the Cloud Security Alliance (CSA), the world’s leading not-for-profit organization committed to AI, cloud, and Zero Trust cybersecurity education, and Dropzone AI, the leading provider of AI SOC analysts, has found that AI-assisted security analysts demonstrate greater speed and accuracy compared to those working manually in a first-ever large scale study, consisting of over 140 participants. Analysts assisted by AI not only completed escalated alert investigations from 45–61% faster but were also 22-29% more accurate than their manual counterparts.

“These patterns suggest that AI-driven investigation platforms can improve human accuracy and speed while sustaining investigative quality across repeated or complex tasks. This highlights the importance of AI in high-volume SOC environments where efficiency and consistency are essential,” said Hillary Baron, Associate Vice President, Cloud Security Alliance.

The findings come from a multi-scenario study evaluating how SOC analysts perform under varying investigative conditions, with and without AI assistance. The study assessed analyst performance across two escalated security alert scenarios: an AWS S3 bucket alert and a Microsoft Entra ID failed login alert. Participants were divided into two groups—those using Dropzone AI, an AI-enabled investigation platform, and those working without automated assistance. Their responses were evaluated across four objective measures (accuracy, speed, completeness, and detail) and three subjective perceptions (difficulty, confidence, and attitudes toward AI).

This study focused on escalated investigations that require analyst involvement. Overall, AI-assisted analysts, even when using AI tooling for the first time, produced more thorough investigations in less time and with greater accuracy and resistance to fatigue. Key findings include:

• Accuracy - AI-assisted analysts achieved higher accuracy scores compared to manual analysts at 22% greater accuracy in the first scenario and 29% greater accuracy in the second scenario.
• Speed - AI-assisted investigations were completed 45% faster in the first scenario and 61% faster in the second.
• Completeness - AI-assisted analysts maintained or slightly increased their level of detail during the study, while manual analysts’ report detail dropped by 27% and conclusion detail by 20%.

“Security leaders are looking for real-world data on how AI benefits SOC effectiveness and efficiency. This study shows that AI tools materially improve quality, speed, and accuracy in escalated alert investigations,” said Edward Wu, Founder and CEO of Dropzone AI. “As cyber threats continue to grow in volume and sophistication, it’s essential to support human judgment at scale—and this benchmark study demonstrates that AI can make that happen in a way that was not previously possible.”

The study, conducted in July and August 2025, randomly assigned 148 participants with varying levels of SOC and incident response experience to either the AI-assisted group or the manual (control) group. The scoring rubric was based on expert-modeled “ideal responses” for each scenario, representing technically thorough investigations. CSA research analysts performed the data analysis and interpretation. Dropzone is a CSA Corporate Member who supports the project but has no influence on the content or editorial process of CSA research.

Download the full report.

About Dropzone AI
Dropzone AI weaponizes LLMs for cyber defenders, equipping them with armies of AI agents so that they can overmatch attackers. With Dropzone autonomously handling routine Tier 1 alert triage, organizations can spend less time on reactive security and more time on proactive security. The Dropzone AI SOC Analyst replicates the techniques of elite analysts and is trusted by more than 200 organizations, including Mysten Labs, Pipe, UiPath, and Zapier. Learn more by visiting www.dropzone.ai.

About Cloud Security Alliance
The Cloud Security Alliance (CSA) is the world’s leading not-for-profit organization committed to awareness, practical implementation, and credentialing of forward-looking cybersecurity topics, including AI, cloud, and Zero Trust. In an era where digital transformation drives business success, CSA stands as the global authority ensuring organizations can operate securely while harnessing cutting-edge technology. Through volunteer-driven research, globally-accepted standards, and award-winning vendor-neutral education programs that unite technical experts, industry practitioners, and varied associations, governments, chapters, and corporate members, CSA bridges the gap between innovation and pragmatic security execution. Visit CSA’s website to learn more.

Media Contact
Kristina Rundquist
ZAG Communications for the CSA
[email protected]