The 763 Tok/s Benchmark That Changed Everything
I’ve seen a lot of AI benchmarks. Most are theater. A lab in Palo Alto, a few GPUs in a rack, a press release that sounds like a TED Talk. This one? This one cracked the code.
Artificial Analysis didn’t just run numbers—they ran a real, air-cooled system with four Nvidia H200s and sixteen SambaNova SN50 RDUs. And they got 763 tokens per second on MiniMax M2.7. At short context. That’s not a tweak. That’s a redefinition of what’s possible.
And here’s the kicker: they didn’t need liquid cooling. No chiller units. No raised floors. No $200k in infrastructure upgrades. Just a standard datacenter rack, humming away like a server farm from 2018. That’s not innovation. That’s pragmatism.
I’ve watched enterprises hemorrhage budget trying to retrofit liquid-cooled Rubin racks into their aging facilities. They’re stuck. And SambaNova? They didn’t ask for permission. They just built something that fits.
Prefill vs. Decode: Why the Split Isn’t Optional Anymore
Let’s talk about what’s really happening under the hood.
The H200s? They’re doing the heavy lifting on prefill. That’s where the prompt gets parsed, the context gets loaded, the key-value cache gets built. It’s compute-heavy. It’s GPU territory. Nvidia knew this. That’s why they started disaggregating prefill and decode with NVL72.
But decode? That’s where things get ugly.
Decode is memory-bandwidth bound. Every token you generate after the first one? It’s a tiny memory fetch. A tiny latency hit. A tiny bottleneck. And when you’re generating 763 tokens a second? Those tiny hits add up to a wall.
That’s where the SN50 RDUs come in. Not as a GPU replacement. As a decode accelerator. They’re built for this. They’re not trying to do everything. They’re not trying to be general-purpose. They’re a scalpel. And they’re slicing through the decode bottleneck like it’s butter.
This isn’t just a new architecture. It’s a new philosophy. Stop forcing one chip to do two jobs that demand opposite things. Split it. Optimize each half. And then, for god’s sake, make sure it fits in your existing rack.
The Real Win: Air Cooling in a Liquid-Cooled World
Here’s the truth no one wants to admit: most enterprise datacenters are not built for liquid cooling.
I’ve sat in server rooms where the AC unit wheezes like an asthmatic grandpa. Where the power draw is already maxed out. Where the budget for infrastructure upgrades got cut three quarters ago.
Nvidia’s Rubin? It’s a masterpiece. And it’s a death sentence for anyone not starting from scratch.
SambaNova’s SN50? It’s air-cooled. It runs on standard 120V. It fits in a 19-inch rack. It doesn’t need a new floor. It doesn’t need a new power grid. It doesn’t need a new vendor contract with a cooling specialist.
This isn’t just about performance. It’s about adoption velocity. It’s about getting AI into production without a two-year infrastructure overhaul.
JPMorgan Chase didn’t pick SambaNova because it’s the flashiest. They picked it because it’s the only one that doesn’t require them to rebuild their entire datacenter just to run a model.
The $11 Billion Bet: Why the Market Is Already Moving
Let’s talk money.
SambaNova just closed a $1 billion Series F. At $11 billion. Five months after their SN50 launch. After a $350 million Series E.
This isn’t hype. This is conviction.
General Atlantic, T. Rowe Price, Capital Group—they didn’t throw money at a chip startup because it sounded cool. They saw a pattern: enterprises are starving for inference that’s fast, secure, and deployable.
JPMorgan’s on board. SoftBank’s deploying in H2 2026. TogetherAI’s already running the platform. And Vector Core Compute? They’re the bridge between SambaNova’s hardware and the real world.
The market isn’t waiting for the next trillion-parameter model. It’s waiting for the next trillion tokens per second.
And right now? SambaNova’s the only one who’s delivering it without asking for a new datacenter.
I’ve watched too many AI startups chase benchmarks that don’t matter. This one? It matters because it’s already running in production. In air-cooled racks. In banks. In sovereign clouds. In places where the only thing more important than speed is reliability.
This isn’t the future.
It’s already here.
And it’s not liquid-cooled.
The Real Win: Air Cooling in a Liquid-Cooled World
Here’s the truth no one wants to admit: most enterprise datacenters are not built for liquid cooling.
I’ve sat in server rooms where the AC unit wheezes like an asthmatic grandpa. Where the power draw is already maxed out. Where the budget for infrastructure upgrades got cut three quarters ago.
Nvidia’s Rubin? It’s a masterpiece. And it’s a death sentence for anyone not starting from scratch.
SambaNova’s SN50? It’s air-cooled. It runs on standard 120V. It fits in a 19-inch rack. It doesn’t need a new floor. It doesn’t need a new power grid. It doesn’t need a new vendor contract with a cooling specialist.
This isn’t just about performance. It’s about adoption velocity. It’s about getting AI into production without a two-year infrastructure overhaul.
JPMorgan Chase didn’t pick SambaNova because it’s the flashiest. They picked it because it’s the only one that doesn’t require them to rebuild their entire datacenter just to run a model.
The $11 Billion Bet: Why the Market Is Already Moving
Let’s talk money.
SambaNova just closed a $1 billion Series F. At $11 billion. Five months after their SN50 launch. After a $350 million Series E.
This isn’t hype. This is conviction.
General Atlantic, T. Rowe Price, Capital Group—they didn’t throw money at a chip startup because it sounded cool. They saw a pattern: enterprises are starving for inference that’s fast, secure, and deployable.
JPMorgan’s on board. SoftBank’s deploying in H2 2026. TogetherAI’s already running the platform. And Vector Core Compute? They’re the bridge between SambaNova’s hardware and the real world.
The market isn’t waiting for the next trillion-parameter model. It’s waiting for the next trillion tokens per second.
And right now? SambaNova’s the only one who’s delivering it without asking for a new datacenter.
I’ve watched too many AI startups chase benchmarks that don’t matter. This one? It matters because it’s already running in production. In air-cooled racks. In banks. In sovereign clouds. In places where the only thing more important than speed is reliability.
This isn’t the future.
It’s already here.
And it’s not liquid-cooled.
