Kubernetes Optimized the World for Stateless HTTP Requests
Here's something most platform teams don't want to hear: Kubernetes was built for a workload that no longer dominates.
For roughly a decade, it was the right answer. Containers got organized. Services scaled horizontally. Platform teams shared a vocabulary for running software in production. Kubernetes abstracted enough of the underlying complexity that engineers could stop thinking about servers and start thinking about services. EKS made that model the default for most enterprise teams on AWS.
The workload that defined that era was the stateless HTTP request. Fast in, fast out, disposable. A user taps something, a request hits a service, the service returns a response, and the container is done. Kubernetes was optimized for that pattern down to the scheduler internals: bin-pack containers onto nodes, autoscale on CPU and memory, evict and reschedule when something breaks. The whole system runs on the assumption that individual units of work are short, stateless, and interchangeable.
That assumption doesn't hold for the workloads that matter most right now. Not even close.
What Is Agentic AI? The Definition That Changes Everything
IBM defines agentic AI as autonomous systems that plan, act, and adapt toward goals without step-by-step human direction. Google Cloud frames it similarly: AI systems that independently execute multi-step workflows, make decisions, and use tools to achieve objectives. Software Development Companies building these systems describe it as the shift from AI that answers questions to AI that does work.
The definition matters here because it's not semantic. It's architectural.
An agentic AI system reasons across time, calls external tools, spawns subprocesses, writes and executes code, and makes decisions that depend on what happened five steps earlier in the same task. A single-agent workflow might run for minutes or hours, touching a dozen external systems and generating intermediate outputs that subsequent steps depend on.
That's not a stateless request. That's a persistent, stateful process with memory, context, and consequences. And Kubernetes was never asked to handle it.
The Four Things Agent Execution Actually Requires
Concretely, agent workloads need four things that the Kubernetes model wasn't designed to provide.
First, isolated execution environments that provision in milliseconds, not minutes. Each agent task needs its own sandbox for code execution and tool calls without blocking the reasoning loop. The difference between a two-second environment and a two-minute environment isn't a performance optimization — it determines whether the architecture is viable at all.
Second, durable state management across the full task lifecycle. An agent needs to pause, hand off, or resume without re-initializing from scratch and burning tokens to reconstruct context it already built. Every restart that loses state is a tax on the system.
Third, coordination primitives for multi-agent work. The ability to spawn sub-agents, pass structured outputs between them, and track task dependencies across a graph of concurrent processes. Production agent systems are rarely single agents — they're pipelines of specialized agents with handoffs that need to be reliable and inspectable.
Fourth, credentials and secrets management that travel with the execution context. Agents authenticate to external services securely without exposing credentials in the task definition, logs, or environment variables of a shared container. This isn't nice-to-have security theater — it's the difference between a contained tool call and a compromised production environment.
Where the Mismatch Shows Up in Production
Kubernetes and EKS expose this mismatch quickly in practice, and the symptoms are unmistakable.
Pod eviction terminates an agent mid-task with no clean recovery path. Autoscaling reads CPU utilization as the load signal, but an agent holding a long inference connection looks idle to the scheduler even when it's doing the most consequential work in the pipeline. Provisioning a new environment takes 45 seconds to two minutes on a well-tuned cluster; agent workloads need that in under two seconds or the reasoning loop stalls and the user experience degrades visibly.
These aren't edge cases or misconfigurations. They're the normal operating conditions for production agent workloads running on infrastructure that wasn't designed for them.
The utilization data makes the broader cost picture even starker. The 2026 State of Kubernetes Optimization Report from CAST AI, drawn from analysis of over 23,000 production clusters across AWS, Azure, and GCP, found average CPU utilization at 8 percent — down from 10 percent the year prior. Memory utilization fell from 23 to 20 percent. CPU overprovisioning jumped from 40 to 69 percent year over year.
These numbers reflect clusters running traditional workloads, and the pattern is worsening as environments scale. Agent workloads compound this problem further. An agent holding an open inference connection or waiting on a tool call registers as idle to a scheduler that reads CPU and memory as the only meaningful load signals. The infrastructure responds to the wrong metric, overprovisioning capacity for demand it cannot measure, while the actual bottleneck — environment provisioning latency and state continuity — goes unaddressed.
AI Cloud Infrastructure Companies in India Are Building the Agent-Native Layer
This is where the story gets interesting. While the Kubernetes ecosystem catches up — SIG Apps published an introduction to Agent Sandbox in March 2026, a CRD-based abstraction designed specifically for singleton, stateful agent workloads — a different group of companies is building the execution layer from scratch.
AI cloud infrastructure companies in India are betting heavily on session-native compute. The thesis is simple: agents don't need containers, pods, or VMs. They need sessions — persistent, isolated, stateful execution environments that start with a snapshot, carry their own credentials, can be suspended and resumed, spawn sub-agents, and communicate with other sessions without caring about the underlying infrastructure.
The companies building this aren't waiting for Kubernetes to evolve. They're treating session speed as the only real constraint — limited only by model-provider time-to-first-token — and moving everything else, like cloning and installing dependencies, to happen before the session starts.
This isn't theoretical. Ramp's engineering team published a detailed account of building Inspect, their internal background coding agent. Each Inspect session runs in a sandboxed VM with a full-stack development environment and deep integrations across observability, CI, and deployment tooling. Filesystem snapshots keep sessions starting in seconds rather than minutes. Sessions are isolated and stateful. The agent can run tests, review telemetry, query feature flags, and visually verify frontend changes in a real browser.
Within months of launch, roughly 30 percent of all pull requests merged to Ramp's frontend and backend repositories were written by Inspect. That level of adoption wasn't mandated. It happened because the execution environment was fast enough, capable enough, and well-integrated enough that the agent was strictly better than a local workflow for a meaningful share of tasks.
The key insight from the Ramp case isn't about the model. It's about the execution layer. As their team put it, session speed should only be limited by model-provider time-to-first-token; everything else needs to be done before the session starts. That's a statement about infrastructure, not intelligence.
Security Is Not the Same Problem It Was Before
Agent workloads change the threat model at the infrastructure level, and most teams haven't caught up.
A compromised stateless service exposes a narrow surface defined by its API contracts. A compromised agent exposes every system it can reach, every credential it holds, and every action it is authorized to take on behalf of the user. Agents generate and execute their own code, make non-deterministic tool-call decisions, and accumulate context across long-running sessions. Standard container namespacing does not contain that kind of risk.
Kernel-level isolation, default-deny network egress, scoped credentials per session, and agent-aware observability are not optional hardening steps. They are baseline requirements for running agents in production.
For teams evaluating their options, Red Hat's OpenShell framework offers a useful reference point — kernel-enforced sandboxing across three isolation modes, validated against Anthropic and OpenAI agent frameworks. The progression from code-only sandboxing to full agent isolation maps directly onto the security requirements that agentic workloads introduce.
The infrastructure to close the gap exists now. The prerequisite is recognizing that agent execution is a first-class compute pattern with its own primitives and its own requirements — not a variant of the stateless service model that defined the last decade. Teams that make that shift early will have a meaningful structural advantage. The ones that don't will spend the next two years wondering why their agent systems are unreliable at a scale that should be tractable.