Waymo's AI Evaluation: A Playbook for the Security & Compliance Analyst
Most enterprise teams ship AI models the moment they hit a target benchmark score. Waymo takes the opposite view: an AI project isn't ready for production when the model performs well—it's ready when its evaluation suite proves it can handle real-world chaos safely.
Speaking at VB Transform 2026, Manasi Joshi, Waymo's Director of Engineering for Systems Intelligence and Machine Learning, laid out the autonomous vehicle leader's governance blueprint. Waymo calls its framework "eval-forced development" or "eval-centric development." Instead of treating testing as a gate check at the end of a sprint, Waymo ties a project's maturity directly to the sophistication of its evaluation pipeline.
That shift matters far beyond autonomous vehicles. Whether an enterprise is building customer service agents, automated coding assistants, or financial workflows, standard model benchmarks tell you very little about how a system behaves under real-world pressure—a challenge that underscores why autonomous agents demand classical security discipline. For any security & compliance analyst tasked with auditing autonomous software, Waymo's operational model offers a practical blueprint for replacing guesswork with verifiable resilience.
Why Raw Model Performance Fails the Security Test
When an autonomous vehicle makes a mistake, the consequences are immediate and physical. Waymo has driven more than 220 million fully autonomous "rider-only" miles, recording 17 times fewer serious crash injuries than human drivers over the exact same distance. That safety record wasn't achieved by tuning a model until its loss curves looked clean. It was built through rigorous, continuous testing across every layer of the engineering stack.
Joshi emphasized that evaluation cannot be treated as a single milestone prior to deployment. Models drift. User behaviors change. Data streams evolve. An agentic system that passes initial acceptance testing can degrade rapidly when exposed to novel inputs or updated underlying APIs.
In enterprise security and compliance, relying on static point-in-time assessments is equally dangerous. Securing autonomous systems requires continuous monitoring across post-training runs, open-loop offline evaluations, and closed-loop interactive simulations. A static security scan run six months ago is useless when prompt injection tactics or API dependencies change weekly.
"Eval is not a one-time task to launch a model." — Manasi Joshi, Waymo
Eval-Forced Development for the Security & Compliance Analyst
Under Waymo's eval-forced development model, engineering teams cannot advance a project through lifecycle stages based on optimism or isolated test runs. Readiness is defined by eval maturity. If a team cannot reliably measure a system's failure modes across complex scenarios, that system remains in development regardless of how impressive its demo looks.
Joshi put it bluntly during her interview: "The stage at which our projects are maturing can be easily kind of transpired based on the eval maturity that they showcase."
For a security & compliance analyst, adapting this principle means establishing continuous evaluation pipelines alongside traditional security tooling. While a security & compliance analyzer for Veeam validates backup integrity and recovery readiness, AI evals must continuously assess model outputs, permission boundary enforcement, and data privacy controls.
Consider an enterprise deploying AI agents across productivity environments like Office 365. Simply checking whether the model connects to Microsoft 365 APIs without error doesn't confirm its safety. An eval-centric governance pipeline tests whether the agent respects role-based access limits, avoids leaking sensitive documents across tenant boundaries, and maintains audit logs during multi-step executions, complementing broader action-based agent governance frameworks. When evaluation datasets and metrics are baked into the development pipeline, security policies stop being passive documentation and become automated gates.
Stress-Testing Rare Edge Cases in Enterprise Workloads
Waymo structures its evaluation hierarchy around high-consequence edge cases. Rather than testing standard highway driving over and over, task owners construct specialized evaluation datasets targeting rare, dangerous environments—vulnerable road users, sudden construction zones, and complex railroad crossings. The system relies on first-party driving logs, targeted third-party inputs, and synthetic simulations spanning billions of miles to expose models to non-routine hazards.
The same principle applies outside autonomous driving. Enterprises need to test not only the routine requests their agents handle successfully, but also uncommon situations where errors could create financial, legal, security, or reputational damage.
To build a resilient cloud security incident response playbook, risk managers must intentionally stress-test agent behavior under adverse conditions:
- Adversarial manipulation: Testing how agents respond to indirect prompt injections embedded in external emails or uploaded files.
- Resource exhaustion and looping: Evaluating how systems handle infinite tool-calling loops or malformed API payloads without crashing downstream services.
- Data boundaries and exfiltration: Verifying that agents refuse requests to bypass data classification tags or export unredacted credentials.
Evaluating these edge cases requires curated, domain-specific testing datasets. As Joshi noted during her interview, performance claims are only as credible as the evaluation data behind them. Enterprise security teams must pair every deployment approval with explicit documentation detailing the test sets, edge-case coverage, and synthetic attack vectors used to validate the model.
Human-in-the-Loop Oversight for High-Stakes AI Deployments
Despite operating an evaluation pipeline capable of running billions of simulated miles, Waymo does not delegate final deployment decisions to automated scoring engines. Production-readiness reviews at Waymo involve extensive human oversight. Designated internal safety leaders must inspect evaluation results and explicitly sign off on software updates and service-area expansions.
As Joshi pointed out, when human lives and public safety are on the line, fully automated release gates are insufficient. The same logic applies to enterprise risk management. Automated evals provide critical telemetry, but high-risk actions—such as modifying firewall rules, issuing refund payouts above set thresholds, or granting elevated cloud permissions—demand accountable human sign-off.
"This is not AI-driven and completely automated and zero human oversight. Human lives are at stake." — Manasi Joshi
Establishing clear lines of human accountability prevents the ambiguity that often accompanies agentic automation. Named decision-makers must review eval metrics, sign off on deployment readiness, and take responsibility for system behavior in production. Automated evaluation supplies the evidence; human leaders make the call.
Optimizing Infrastructure and Agentic Workflows at Scale
Running continuous evaluation across complex model architectures demands massive computational and data resources. Waymo faces the same infrastructure pressures troubling enterprise IT departments: demand for compute, storage, memory, and network throughput consistently outpaces supply.
To manage these constraints, Waymo focuses heavily on data efficiency—selecting the most informative training and evaluation samples rather than accumulating uncurated data volume. The company's technology stack spans onboard real-time inference systems inside vehicles and off-board infrastructure dedicated to model training, simulation, and batch evals. Since introducing transformers into its architecture in 2017, Waymo has expanded into large language models, vision-language models, vision-language-action models, and generative multimodal architectures.
Interestingly, Waymo also uses AI agents internally to help its engineers maintain this complex ecosystem. Internal agents analyze data distributions, evaluate data efficiency, and triage failures across vehicle telemetry, model training runs, and evaluation pipelines. Crucially, Waymo subjects these internal productivity agents to the same evaluation rigor as its driving software, ensuring automated triage tools don't hallucinate root causes or mislead human engineers.
For enterprise security and compliance teams, Waymo's methodology provides a clear operational framework. Deploying agentic AI requires far more than choosing a capable base model. It requires defining clear business outcomes, building representative evaluation datasets, running continuous post-launch evals, optimizing infrastructure efficiency, and assigning human accountability to every deployment.
As Joshi summarized at VB Transform 2026: "Earning trust is supremely important."
