Developer Platforms & Tooling
Coverage of developer-first AI infrastructure tools, frameworks, SDKs, and product ecosystems that enable rapid model deployment and orchestration.
Eight files, six axes, one session — and the two axes we lose on every file
The claim under test was that the engine is at or above llama.cpp on every axis, for every model it can run. This is the session that tested it across sixteen files: prefill goes one way, quality is a tie, and peak VRAM and decode at 8,192 go the other way on eight of eight — the second one visible only once the baseline is the faster of llama.cpp's two backends.
One session, one binary, every axis: four wins, four losses, one tie
Every earlier article in this series measured one axis on the tree that was current at the time. This one measured all of them in a single window, from a single binary, with the gates run first and the spread recorded for every cell — the losses included.
What is inside a dynamic quant: nine tensor types in one Q4_K_M file
The `UD-` label names a bit budget, not a format. `UD-Q4_K_M` carries IQ4_XS, Q5_K, Q4_K, Q6_K, IQ4_NL, Q3_K and IQ3_S, and an engine that knows only the K-quants can size it without running it.
Serving eight sessions on one 32 GB card: 125 tok/s, a 16-slot tie, and a cache that could not keep its promise
Continuous batching beats llama-server by 32% at eight slots and ties it at sixteen, with better tail latency; the hybrid prefix cache is where the release run found — and then fixed — a real defect.
Roofline first: why neither engine can decode faster on a 640 GB/s card
The useful question is not what is slow but what cannot be fast. Decode on this card has a ceiling of 36.1 tokens per second, and the two engines sit at 82% and 88% of it.
Speculative decoding that is exact: 1.53× to 2.00× from the model's own draft head
The MTP head drafts at most three tokens, the target verifies them in one weight pass, and every accepted stream in the release run matches plain greedy token for token.
The bug that survived every parity gate: 40 of 48 heads running on zeros
One launch argument left five sixths of a projection unwritten, and every correctness check the project owned was blind to it.
Beating llama.cpp at prefill: 11 cells of 12, and a 6% decode bill
On one R9700 with a 16.8 GB Qwen3.8-27B Q4_K_M file, prefill is 4.6% to 17.2% faster in eleven of twelve cells; decode is 6% slower, for a reason no kernel change addresses.
Stripe's OpenRouter Play: Power Over AI Model Routing and Expense Flows
Stripe acquired OpenRouter, a startup that routes prompts between different AI models, for approximately $7.5 billion. The official "singularity" reason is a tongue-in-cheek reference from Stripe's founders, but the real drivers are gaining insight into how developers use AI models, securing leverage over AI suppliers, and moving into AI expense management.