What this refresh covers
AI model local deployment keeps moving faster than most people can re-benchmark. So when I re-ran a finished like-for-like standard-v2 suite against two packs of the same model, I wanted the raw numbers on the page before anything got smoothed into a single score. They're here, plain.
Both packs are the same architecture: Liquid AI's LFM2.5-24B-A2B, which is native lfm2_moe in mlx-serve. It is not a Qwen model. The tags on both Hugging Face repos confirm it — MLX, Mixture of Experts, LFM2.5. That matters because a mislabeled conversion quietly wrecks throughput and reasoning quality, and people still get these two confused.
The two packs I tested:
- Mixed 4/8-bit: pgmharikrishnan/LFM2.5-24B-A2B-MLX-Serve-mixed-4-8bit — a community pack that raises precision on some layers.
- Stock 4-bit: LiquidAI/LFM2.5-24B-A2B-MLX-4bit — the vendor's uniform 4-bit release.
Same weights underneath, different quantization layout. That's the only variable. If you want the background on what the server layer actually adds on top of plain MLX, read What Is MLX-Serve? first; if you want how we run the numbers, see How We Benchmark AI Model Local Deployment.
Why mixed precision at all
The whole premise of mixed precision is that not all tensors carry the same amount of risk when you crush them down to 4 bits. A uniform 4-bit pack is simple and compact. A mixed pack spends a little extra memory on the layers that lose the most when quantized, and saves on the ones that don't. For a sparse mixture-of-experts model, the router and gate tensors are the sensitive spots, get those wrong and tokens start landing on the wrong experts. The trade-offs are laid out in How MoE Routers and Gates Direct Tokens, and Why Precision Matters and Choosing MLX Weight Precision by Tensor Role and Storage Budget.
This refresh doesn't re-explain the conversion step. If you haven't seen the pack get built in the first place, the prior conversion write-up is LFM2.5-24B-A2B Mixed 4/8-Bit on MLX-Serve, and the general mechanics are in Converting a Hugging Face Model for mlx-serve.
The results, metric by metric
Setup. Apple M4 Max workstation; mlx-serve v26.10.2; pgmharikrishnan/LFM2.5-24B-A2B-MLX-Serve-mixed-4-8bit mixed 4/8-bit and LiquidAI/LFM2.5-24B-A2B-MLX-4bit 4-bit; tested October 2026.
Every figure below is a completed like-for-like standard-v2 result from the same machine and the same server build. Nothing here is an average across runs, and nothing is a composite.
Reasoning and knowledge (MMLU-Pro-400)
The mixed pack scored 65.5% (262/400). Stock came in at 61.75% (247/400). That's a 15-question gap out of 400 on a broad multi-domain reasoning set, real, but you should read it as "mixed pulled ahead on a wide knowledge sweep," not as proof of a better model. Same base weights, different precision.
Math (GSM8K-50)
Both packs hit 94% (47/50). No separation at all. Grade-school word problems are a blunt instrument at this size, when a 4-bit stock pack and a mixed pack tie exactly, the task simply isn't sensitive enough to the precision differences we're chasing.
Coding
The mixed pack cleared 6/6; stock cleared 5/6. One task separated them. I'm deliberately not inflating a single coding pass into a verdict, six tasks is a smoke test, not a benchmark.
Plain decode throughput
Stock generated faster: 72.608 tok/s versus the mixed pack's 69.781 tok/s. A roughly 4% edge. That's the expected direction, stock carries fewer high-precision weights, so decode reads less per token.
Plain prefill throughput
The mixed pack processed input faster: 538.827 tok/s versus stock's 515.597 tok/s. Interesting inversion. On this build the mixed pack wins prefill while losing decode. Whether that holds on other hardware I genuinely don't know, and I'm not going to pretend the suite says otherwise.
Peak memory
Mixed used 18.67 GB; stock used 18.32 GB. The ~0.35 GB premium is the cost of the extra precision, exactly where you'd expect to pay it.
The honest limits here
Fidelity was not measured in this run. Not measured is not zero, and I'm not going to leave the implication that the packs are numerically identical to the originals, that's simply untested, so treat it as unknown. The standard-v2 suite also has a ceiling on how much one pass can settle; see How We Benchmark AI Model Local Deployment for the methodology and its boundaries.
There is no composite winner in this article, and no general recommendation. The mixed pack led on knowledge breadth and one coding pass and on prefill; stock led on decode and used less memory; both tied on math. Those are four different directions. Picking between them is a per-workload decision, not a league-table decision.
Deploying a pack like this locally
For anyone working through how to deploy ai model workflows on Apple Silicon, the practical point is smaller than the marketing usually admits: pick your pack the same way you'd pick any config flag, measure it on your workload. These are single-model, single-machine numbers, and a pack that wins prefill on an M4 Max can lose on a different memory bandwidth. The serving layer behind it is covered in What Is MLX-Serve?, and the precision choices you'd tweak next are in Choosing MLX Weight Precision by Tensor Role and Storage Budget.
Disclaimer. These are measurements from one machine, one server build, and one test suite on the stated date. Results vary with hardware, software version, settings, and load. They are not a general ranking or a recommendation.
