Artificial intelligence can help security teams explore code and application behavior, propose test cases, and organize findings. But a plausible vulnerability report is not evidence that a vulnerability exists, and a confirmed vulnerability is not automatically an operational risk. Teams need reproducible checks and context about the affected system.
A Dark Reading report on Google’s Pagebreak AI agent describes the agent testing web applications and reports hundreds of flaws in the tests discussed. These reported results apply to those tests; they do not establish a general success rate for AI agents or predict results across other applications and deployments. The practical question is how many candidate findings are reproducible, how severe they are in context, and whether teams can remediate them safely.
What is AI in cybersecurity?
AI in cybersecurity refers to using machine-learning systems, language models, and tool-using agents to support security work. Depending on their design and access, these systems may help classify or summarize signals, generate code or queries, work with documentation, or interact with testing tools. An agent may break a broad objective into steps, call tools, inspect responses, and revise its next action.
AI is not a single defensive control, and its outputs do not guarantee correctness. A language model can produce a false explanation; an agent can mistake an unusual response for proof of a flaw. Treat AI as an aid to investigation, not an authority that certifies safety or exploitability.
How AI is used in cybersecurity
Security teams may use AI to help triage alerts, explain unfamiliar code, identify suspicious patterns, draft test ideas, and organize findings. In application security, an agent may help map reachable functionality, generate inputs, or explore variations. Human testers can assess promising paths and the business logic behind them.
AI may also assist with remediation handoffs by turning an observation into a candidate report, identifying potentially affected components, or helping developers understand a likely cause. Reports are more useful when they retain the original request, response, code location, and conditions needed to reproduce the behavior. A polished narrative without supporting evidence makes review harder.
Why repeatable validation matters
Test a candidate finding against explicit conditions that can be repeated. For a web issue, that might mean replaying a request in an authorized, controlled environment, confirming the relevant response or state change, and varying one condition at a time. A finding is better supported when another reviewer can reproduce it without relying on an agent’s interpretation. The Pagebreak report describes reported results from its tests; candidates still need checking before they are treated as confirmed findings.
Validation helps separate flaws from noise. Applications may return different errors, redirect users, or expose content based on permissions and state. A response may have a benign explanation, while a failed attempt may not rule out a flaw if the environment lacks the required account, configuration, or preconditions. Document what was tested and what the result establishes.
Validation should be safe and scoped. Testing belongs in an authorized environment, with limits to prevent disruption, data exposure, or unintended access. Agents that can take actions should have constrained permissions, monitored tool use, and clear stop conditions. Define targets and allowed actions to reduce the risk of mistakes.
From a confirmed flaw to a risk assessment
Confirmation answers whether a weakness exists under stated conditions. Risk assessment asks what that weakness means for the organization. Examine the affected asset, exposure, required access, exploit prerequisites, potential impact, and sensitivity of reachable data or operations. Consider whether compensating controls or segmentation change the practical attack path.
Separate observation from inference. State the reproducible behavior first, then describe the security consequence and assumptions. For example, distinguish “the endpoint returned another user’s record under this account and request” from a broader claim about all accounts or records. The narrower statement is easier to validate and provides a sound basis for assessing scope. A model-generated severity score is a suggestion, not an objective measurement; scoring should reflect organizational context and receive review from accountable security personnel.
Connect technical severity to remediation decisions. A remotely reachable flaw in a high-value service may deserve immediate attention, while a similar bug behind strong access controls may call for a different response. Record confidence, affected versions, reproducibility, business impact, and remaining uncertainty; assign owners and retest after a fix.
A practical workflow for AI-assisted discovery
Start with a defined scope: approved targets, test accounts, environments, rate limits, and prohibited actions. Give the agent only the access needed for the task. Ask it for candidate hypotheses with supporting evidence rather than unqualified declarations. Preserve its tool actions and relevant inputs so investigators can review how it reached a conclusion.
Reproduce each promising candidate independently, using repeatable tests and a safe test environment where possible. Confirm preconditions and establish whether the observed behavior crosses a security boundary. Have a human reviewer consider alternative explanations, assess impact, and check that testing stays within authorization. For related context, see AI-assisted vulnerability discovery.
Map validated findings to remediation. Explain the vulnerable behavior in developer-usable terms, identify affected assets, and recommend a fix that addresses the cause rather than only the symptom. Track the finding through correction and retest. Measure outcomes such as validated issues, time to triage, remediation quality, and false-positive burden—not raw candidate volume.
What AI can and cannot establish
AI-assisted exploration can produce useful leads, but the strength of a conclusion depends on the evidence collected. A repeatable test supports a claim about the behavior observed under its stated conditions; it does not, on its own, establish behavior in untested configurations or prove that a system is secure. Risk priority additionally depends on exposure, impact, controls, and organizational context.
No test can establish every deployment condition from limited observations, and an agent cannot decide an organization’s acceptable risk. Use bounded automation, keep a clear record of uncertainty, and assign responsibility for validation and remediation decisions to people.
Sources
- Dark Reading: Google’s Pagebreak AI agent found 500 flaws in web apps — reporting on the Pagebreak tests and their reported results.
is ai in cybersecurity