ProBackend
Developer Platforms & Tooling

Developer Platforms & Tooling

Coverage of developer-first AI infrastructure tools, frameworks, SDKs, and product ecosystems that enable rapid model deployment and orchestration.

Eight files, six axes, one session — and the two axes we lose on every file
developer platforms tooling1 week ago9 min

Eight files, six axes, one session — and the two axes we lose on every file

The claim under test was that the engine is at or above llama.cpp on every axis, for every model it can run. This is the session that tested it across sixteen files: prefill goes one way, quality is a tie, and peak VRAM and decode at 8,192 go the other way on eight of eight — the second one visible only once the baseline is the faster of llama.cpp's two backends.

One session, one binary, every axis: four wins, four losses, one tie
developer platforms tooling1 week ago6 min

One session, one binary, every axis: four wins, four losses, one tie

Every earlier article in this series measured one axis on the tree that was current at the time. This one measured all of them in a single window, from a single binary, with the gates run first and the spread recorded for every cell — the losses included.

What is inside a dynamic quant: nine tensor types in one Q4_K_M file
developer platforms tooling1 week ago5 min

What is inside a dynamic quant: nine tensor types in one Q4_K_M file

The `UD-` label names a bit budget, not a format. `UD-Q4_K_M` carries IQ4_XS, Q5_K, Q4_K, Q6_K, IQ4_NL, Q3_K and IQ3_S, and an engine that knows only the K-quants can size it without running it.

Serving eight sessions on one 32 GB card: 125 tok/s, a 16-slot tie, and a cache that could not keep its promise
developer platforms tooling1 week ago4 min

Serving eight sessions on one 32 GB card: 125 tok/s, a 16-slot tie, and a cache that could not keep its promise

Continuous batching beats llama-server by 32% at eight slots and ties it at sixteen, with better tail latency; the hybrid prefix cache is where the release run found — and then fixed — a real defect.

Roofline first: why neither engine can decode faster on a 640 GB/s card
developer platforms tooling1 week ago5 min

Roofline first: why neither engine can decode faster on a 640 GB/s card

The useful question is not what is slow but what cannot be fast. Decode on this card has a ceiling of 36.1 tokens per second, and the two engines sit at 82% and 88% of it.

Speculative decoding that is exact: 1.53× to 2.00× from the model's own draft head
developer platforms tooling1 week ago5 min

Speculative decoding that is exact: 1.53× to 2.00× from the model's own draft head

The MTP head drafts at most three tokens, the target verifies them in one weight pass, and every accepted stream in the release run matches plain greedy token for token.

The bug that survived every parity gate: 40 of 48 heads running on zeros
developer platforms tooling1 week ago4 min

The bug that survived every parity gate: 40 of 48 heads running on zeros

One launch argument left five sixths of a projection unwritten, and every correctness check the project owned was blind to it.

Beating llama.cpp at prefill: 11 cells of 12, and a 6% decode bill
developer platforms tooling1 week ago5 min

Beating llama.cpp at prefill: 11 cells of 12, and a 6% decode bill

On one R9700 with a 16.8 GB Qwen3.8-27B Q4_K_M file, prefill is 4.6% to 17.2% faster in eleven of twelve cells; decode is 6% slower, for a reason no kernel change addresses.

Stripe's OpenRouter Play: Power Over AI Model Routing and Expense Flows
developer platforms toolingSep 18, 20266 min

Stripe's OpenRouter Play: Power Over AI Model Routing and Expense Flows

Stripe acquired OpenRouter, a startup that routes prompts between different AI models, for approximately $7.5 billion. The official "singularity" reason is a tongue-in-cheek reference from Stripe's founders, but the real drivers are gaining insight into how developers use AI models, securing leverage over AI suppliers, and moving into AI expense management.