Real-Time Video for Agents
Let's face it: scaling real-time audio and video is a complete nightmare. As an SRE, I've spent far too much time debugging erratic WebRTC connection success rates and bloated CDN buffers. The modern IP stack wasn't built for this. Now, enter agentic AI. We are no longer just streaming Netflix files to passive viewers; we are streaming real-time, interactive, multi-modal feedback from autonomous agents straight to user browsers. When you look at how platform teams evaluate agentic systems—benchmarking them in enterprise workflows like we see with the GTM Bench evaluations—the latency bottlenecks immediately shift from model inference to the networking layer. The moment an agent has to stream a live avatar or high-fidelity voice output, the SRE team starts sweating. We need a protocol that can actually scale real-time media without dropping the connection or buffering.
What is Agentic AI? Definitions and Differentiators
Understanding the architectural requirements for these systems requires defining what we are actually running over the network. There's a lot of hype floating around, so let's parse the concrete definitions from the big players.
What is Agentic AI? - IBM
According to IBM, agentic AI refers to an artificial intelligence system that can accomplish a specific goal with limited human supervision. This isn't your standard prompt-and-response chatbot. It's a system composed of active "AI agents"—machine learning models that mimic human decision-making to solve complex tasks. These agents perceive their environment, run reasoning models over the data, create action plans, and actively use tools like APIs, web search engines, or database queries to hit their target goals.
What is Agentic AI? Definition and Differentiators | Google Cloud
When we look at the official model from Google Cloud, they focus on goal-driven autonomy. The key differentiators from vanilla generative AI center on iterative planning, memory, and active tool use. Google's model highlights that instead of just generating text for a user to act on, agentic AI operates in a loop: observing the environment, reasoning, choosing a tool, and acting. Under their AI and Cloud Computing Services, Google Cloud highlights these capabilities as essential for next-generation enterprise workflows. We're talking about autonomous agents that can manage entire incident response playbooks or analyze network logs without a human holding their hand.
What is Agentic AI? | Software Development Companies
For software development companies, agentic AI is defined by the underlying frameworks and runtimes used to build them. They build autonomous multi-agent systems using tools like LangGraph or a Databricks AI agent. The focus is on long-running state management, deterministic validation gates, and self-healing code execution loops. Instead of writing linear code to handle every possible edge case, developers deploy an agent that orchestrates its own execution path, frequently calling APIs or generating dynamic UIs to resolve tasks.
Why AI Cloud Infrastructure Companies in India Face a Network Wall
So, why does this matter for SREs and infrastructure teams? Let's zoom in on India. We are seeing a massive surge of localized compute hubs. Platforms like Amazon are pouring billions into expansion. For instance, AWS's $13 billion bet on Indian regional datacenters in Mumbai and Hyderabad shows how rapidly the cloud footprint is growing. Yet, AI cloud infrastructure companies in India face a serious bottleneck: the network footprint. You can spin up thousands of Nvidia H100s, deploy high-performance model runtimes, and optimize model latency down to milliseconds. If the egress network routes can't deliver that model’s voice and video feed to a user in Bangalore or Delhi without stuttering, the application fails. Traditional AWS cloud infrastructure configurations are built around robust HTTP/2 and TCP pipelines, but these architectures break down when you try to scale millions of concurrent, ultra-low-latency real-time sessions.
Enter Media Over QUIC: Protocol Mechanics
This is where Media over QUIC (MoQ) steps in. As detailed by The Register, MoQ acts as a bridge between the two traditional bookends of media transport: WebRTC and DASH. WebRTC is what powers Google Meet or Zoom. It's real-time, UDP-based, and has incredibly low latency. However, it lacks scalability. Try scaling WebRTC to a million concurrent viewers; you will drown in the operational costs of managing Selective Forwarding Units (SFUs) that must inspect, decrypt, and replicate streams. On the other side, DASH (Dynamic Adaptive Streaming over HTTP) is what powers YouTube and Netflix. It scales globally by leveraging standard CDNs, but it relies on TCP or reliable HTTP/3 chunks, caching media in seconds-long segments. That caching adds unacceptable latency for real-time human-agent dialogue. MoQ delivers WebRTC-like latency with DASH-like scalability by using a publisher/subscriber relay model.
MoQ Relays and the Replicated Track Architecture
Under the hood, MoQ abstracts media transport into a clean pub/sub model. The core entity here is a "track." A track represents a specific stream of data—say, a live video feed from an agent's virtual avatar or an audio track. Publishers push media into a track, and subscribers subscribe to it. The magic lies in MoQ relays. These are CDN-like overlay nodes deployed across edge networks, such as Cloudflare's regional networks. Publishers send their tracks to an upstream relay, and the relays handle replication down the tree to the edge. The subscribers receive their media without ever establishing a direct peer-to-peer connection to the publisher. Because the relays do not need to decrypt or inspect the media payloads to route them, they remain blazing-fast and highly secure.
The QUIC Layer: Solving Head-of-Line Blocking
The foundational protocol for MoQ is, of course, QUIC. Traditional TCP transport has a fatal flaw: head-of-line (HOL) blocking. If one packet of a TCP stream drops, the operating system kernel halts delivery of all subsequent packets until the dropped packet is retransmitted. In real-time video, this is a disaster—a single dropped frame freezes the entire stream. As Zscaler and ClouDNS point out, QUIC solves this by multiplexing streams. Within a single QUIC connection, you can have thousands of independent, lightweight streams. A dropped packet on stream A has absolutely zero impact on stream B. Furthermore, QUIC delivers connection migration (resilience when switching from Wi-Fi to cellular), default TLS 1.3 encryption, and 0-RTT handshakes. When network congestion hits, MoQ uses this stream awareness to prioritize media. Low-priority streams—such as high-fidelity video tracks—can be discarded immediately, ensuring the high-priority audio track gets through without delay.
Application Layer Framing: Relays Without Decoding
MoQ is also a modern realization of Application Layer Framing (ALF), a network architecture concept proposed by David Clark and David Tennenhouse back in 1990. The concept is simple: let the application define how data is framed, and let the network assist in handling those frames without understanding the underlying payload. MoQ packages media into discrete "objects" (like independently decodable video keyframes). Each object is sent over its own QUIC stream and carries clear metadata (priority, track IDs, object IDs) visible to relays. Relays use this metadata to manage congestion intelligently. If an edge node is bottlenecked, it reads the priority header and drops the low-priority video objects while forwarding the audio objects. There’s no need for expensive, resource-heavy SFUs to decode and re-encode the stream. It’s elegant, fast, and exactly what we need as we scale real-time AI networks.