How LFM2.5-24B-A2B Was Converted for MLX-Serve
The public pgmharikrishnan/LFM2.5-24B-A2B-MLX-Serve-mixed-4-8bit repository on Hugging Face is an MLX-Serve-oriented conversion of LiquidAI’s LFM2.5-24B-A2B. This is a specific local deployment artifact, not a generic recipe for quantizing every model. The repository identifies the model family as lfm2_moe and describes it as LFM-MoE; it is not Qwen. Its model card identifies the converted artifact as an affine-quantized derivative of the LiquidAI checkpoint and retains the source repository’s LFM 1.0 license.
For teams asking how to deploy an AI model locally, the important first distinction is model compatibility: this pack targets Apple Silicon and the MLX-Serve software path. It should not be mistaken for a portable checkpoint with identical performance across hardware or runtimes. For background, see What is MLX-Serve? and Converting Hugging Face Models for MLX-Serve. The conversion and architecture details in this article are documented in the linked Hugging Face model card.
What the conversion preserves—and what it changes
The repository’s model card documents an architecture-aware mixed-precision layout. Routed expert weights are affine 4-bit with group size 64. The card describes the expert projections as stacked three-dimensional banks, the tensor layout the repository targets for the MLX-Serve fused expert path. This choice concentrates the lowest precision in the routed expert matrices rather than applying one bit width indiscriminately to every tensor.
Other parts of the network use distinct treatment, as recorded in the card. Embeddings are affine 6-bit, also with group size 64. The non-expert weights use 8-bit affine quantization at group size 64. The audited pack keeps norms, scales, biases, dense/non-expert tensors, and router-gate tensors in BF16; expert_bias is F32. These details matter because the word “4-bit” alone would misleadingly suggest every tensor has the same representation.
This is a conversion tailored to this architecture and the serving implementation documented in the repository, not a universal prescription. The documented bit widths track tensor roles in this specific model: the aggressive 4-bit setting is confined to the routed expert matrices, while embeddings, non-expert weights, and the routing-related tensors are each left in higher precision. The distinctions between 4-bit, 8-bit, and mixed precision are covered in MLX Quantization Overview; the role of gate precision in MoE inference is discussed in MoE Router Precision. These conversion and architecture claims are traceable to the Hugging Face model card linked above.
Recorded benchmark results
The figures in this section come from the recorded benchmark evidence for the conversion repository. The evidence compares the mixed pack against a stock LiquidAI LFM2.5 4-bit MLX pack using its onboarding benchmark harness. The values below are the recorded results for the mixed pack; they are measurements, not a general model ranking. The Hugging Face model card documents the conversion, while the recorded benchmark evidence is the source for these benchmark measurements and setup.
Setup. Apple M4 Max workstation; mlx-serve v26.10.2; pgmharikrishnan/LFM2.5-24B-A2B-MLX-Serve-mixed-4-8bit mixed 4/8-bit; tested October 2026.
| Measurement | Mixed pack result |
|---|---|
| Plain decode | 189.751 tok/s |
| Plain prefill | 733.090 tok/s |
| Best decode | 184.952 tok/s |
| Best prefill | 460.791 tok/s |
The “plain” and “best” rows describe different benchmark conditions in the recorded harness; they should not be collapsed into a single speed claim. The best decode result is close to the plain figure, while best prefill is lower than plain prefill. Benchmark interpretation also depends on workload, prompt length, and measurement conditions. See How We Benchmark Local Models for the evaluation context.
The recorded long-context ladder provides only rounded figures here. At 4K, decode was 143 tok/s and prefill 2644 tok/s; at 16K, 104 and 2441 tok/s; at 32K, 79 and 2156 tok/s; and at 64K, 52 and 1674 tok/s, respectively. Both compared packs failed at the 131,072-token ceiling. No 128K throughput comparison is provided because the ceiling result is a failure rather than a usable comparison row. These figures and the setup above are attributed to the recorded benchmark evidence.
These rows show a declining decode rate as context length increases in this test, while prefill remains substantially higher. They do not establish a universal scaling curve: the figures belong to one machine, one server build, and one benchmark suite.
How to read this pack against your own workload
The repository’s model card states the runtime and hardware family the pack targets. Confirm that your setup matches those requirements and independently check that your application is compatible with the model and serving configuration you intend to use. The recorded figures above are benchmark measurements, not a deployment guarantee: output quality, latency, memory headroom, and context lengths should be checked against your own prompt lengths and interaction pattern before relying on the conversion.
This repository documents a specific LFM-MoE conversion for MLX-Serve, with per-tensor precision choices recorded in the model card. Its quantization details and benchmark evidence have distinct sources: the Hugging Face model card documents the pack, and the recorded benchmark evidence documents the reported measurements. The results pertain only to this model pack and the stated setup.
Further reading
- What is MLX-Serve?
- MLX Quantization Overview
- MoE Router Precision
- How We Benchmark Local Models
- Converting Hugging Face Models for MLX-Serve
Disclaimer. These are measurements from one machine, one server build, and one test suite on the stated date. Results vary with hardware, software version, settings, and load. They are not a general ranking or a recommendation.
