# Four Models Measured Across a Growing Context on M4 Max

- **Canonical URL:** https://www.probackend.com/ai-local-model-deployment/four-local-moe-models-under-a-growing-context-load-m4-max-measurements
- **Section:** News & Analysis
- **Published:** 2026-10-19
- **Summary:** Measured first-token times, peak memory, and cached-prefix results for four local models on an M4 Max.

---

## What this local-model probe measured

A context limit is not a promise that every request at that length will feel usable. We tested what happens as one conversation grows, with a prior prefix available for reuse, and measured the wait for the first generated token. The distinction matters: a cached continuation and a cold request that starts with an entire long prompt are different workloads.

The probe had four turns: approximately 30K tokens, then 62K with the earlier prefix cached, then 118K with the prior prefix cached, followed by a cached follow-up. The configured context capacity was 131K. That is the capacity setting, not a 131K prompt in the test. A 240-second first-token readiness budget was used as a practical boundary. These are individual timings, not guarantees for future requests.

**Setup.** Apple Silicon workstation (M4 Max, 128 GB unified memory); mlx-serve v26.10.2; models: [pgmharikrishnan/MiniCPM-V-4.7-35B-A3B-MLX-Serve-4bit](https://huggingface.co/pgmharikrishnan/MiniCPM-V-4.7-35B-A3B-MLX-Serve-4bit); [mlx-community/KAT-Coder-V2.5-Dev-OptiQ-4bit](https://huggingface.co/mlx-community/KAT-Coder-V2.5-Dev-OptiQ-4bit); [NVIDIA Nemotron-3.5 Bolt 74B-A9B 4-bit experimental conversion](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16); [ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw](https://huggingface.co/ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw); tested October 2026. The model repositories document the model variants; the measured setup and results are from the probe.

## Four local models, four growing-prefix turns

| Model | ~30K first token | ~62K cached-prefix | ~118K cached-prefix | Cached follow-up | Peak memory |
|---|---:|---:|---:|---:|---:|
| MiniCPM-V | 21.4 s | 36.6 s | 103.2 s | 3.3 s | 23.2 GiB |
| KAT-Coder tuned | 28.1 s | 46.7 s | 127.7 s | 1.4 s | 25.4 GiB |
| Nemotron Bolt | 55.5 s | 72.6 s | 164.7 s | 1.5 s | 48.1 GiB |
| Qwen Flash iQ | 76.6 s | 94.9 s | 176.4 s | 0.5 s | 77.9 GiB |

Each model stayed within the 240-second budget in these measured growing-session runs. That is a narrow statement about this test, not a service-level promise. Prefix reuse does not mean a cold full-window request passes: we did not establish that by making this probe. Nor does the configured 131K capacity mean we sent a 131K prompt.

The table makes the trade-off visible. MiniCPM had the shortest first-token waits in the first three stages. Qwen’s cached follow-up was quickest, but its peak memory was also highest among these four measurements. KAT’s tuned configuration returned the cached follow-up in 1.4 seconds. Bolt also returned quickly in that step, but it needed a custom loader, so it was benchmarked and then dropped from our stack. A result can be interesting and still be too awkward to treat as a deployable option.

Memory here means measured peak memory, not model weight size. Don’t compare the GiB column with a model’s nominal weight footprint and assume the difference is unexplained waste; the figures answer a different question, namely how much memory this run peaked at under the test setup.

## Task scores are a separate view

The reported MMLU-Pro scores were 91.0% for MiniCPM-V, 88.75% for KAT-Coder, 81.75% for Bolt, and 91.5% for Flash iQ. Each recorded 98% on GSM8K and 6/6 on the coding sample. These outcomes are useful context, but the small coding set is not a broad estimate of programming ability. They also do not answer the serving question: a model can score well and still have a long wait before it starts responding at depth. The model variants are documented in the [MiniCPM-V](https://huggingface.co/pgmharikrishnan/MiniCPM-V-4.7-35B-A3B-MLX-Serve-4bit), [KAT-Coder](https://huggingface.co/mlx-community/KAT-Coder-V2.5-Dev-OptiQ-4bit), [Nemotron Bolt](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16), and [Qwen Flash Next iQ](https://huggingface.co/ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw) model cards; the scores here are reported test results, not independently validated benchmark claims.

Short depth-decode samples, roughly 60 generated tokens or fewer, are indicative rather than stable throughput measurements. Treat them as a glimpse, not a dependable ranking of sustained generation speed.

## Tuning findings, and what they do not establish

On the measured KAT and Bolt configurations, 8-bit KV was faster than KV off. That evidence is not established for all four models. MiniCPM and Flash iQ lacked a clean A/B comparison, so extending the result to them would overstate what we measured.

For KAT, disabling PLD helped on novel text in a one-run comparison. Its SSM checkpoint stride change had a much larger observed follow-up effect: the cached follow-up moved from 13.5 seconds to 1.4 seconds, with a growth-latency trade-off. This is a measured configuration-specific result, not a universal setting recommendation. Bolt’s chunk size of 4096 was best among the sizes tested, though the loader requirement remains a substantial caveat.

Multi-token prediction (MTP) is relevant only when a model ships with a trained, compatible head. For Flash iQ, a short-prompt sample measured 42.138 tok/s with MTP versus 29.985 tok/s plain, at about 2.4 tokens per step. It helped little at depth. In the mixed t3 result, Flash’s spread was 138.5 versus 273.1 seconds. That spread is not evidence of reliability; it is a warning not to turn a handful of measurements into a stability claim.

## Related measurements

Related series articles include [MiniCPM-V 4.7 35B A3B](/ai-local-model-deployment/performance-profiling-of-minicpm-v-4-7-35b-a3b-131k-context-on-apple-silicon-m4), [KAT-Coder v2.5](/ai-local-model-deployment/benchmarking-kat-coder-v2-5-131k-context-performance-on-apple-silicon-m4-max), [Nemotron Bolt 74B](/ai-local-model-deployment/nemotron-3-5-bolt-74b-a9b-131k-context-performance-on-apple-silicon-m4-max), [Qwen3.8 Flash Next iQ](/ai-local-model-deployment/how-a-cached-prefix-probe-performed-on-a-local-qwen-model), the four-model MoE roundup, and the MiniCPM vs. Qwen3.8 Flash Next comparison. For a live sibling measurement, see [MiniCPM and Qwen3.8 Flash at 131K on M4 Max](https://www.probackend.com/ai-local-model-deployment/how-minicpm-and-qwen3-8-flash-handle-a-growing-131k-session-on-m4-max).

These related articles provide measurement context; the timings, peak-memory readings, benchmark samples, and tuning observations in this article are results from the stated probe and should be interpreted with its setup and caveats.

## Practical conclusion

If the workload is a long-running local conversation, measure the continuation path, not only an isolated prompt. Keep cold-start and cache behavior separate in your notes. And don’t plan around a single successful deep request as if it were a latency guarantee. In this run, every model met the chosen 240-second first-token budget across the growing-prefix turns, but the models differed sharply in time, memory, and practical setup friction.

**Disclaimer.** These are measurements from one machine, one server build, and one test suite on the stated date. Results vary with hardware, software version, settings, and load. They are not a general ranking or a recommendation.

![this local-model probe measured](https://assets.probackend.com/60de3c70-ff3d-407b-a273-a7d177b78955?token=ast_7b9ee8975a4b187969d396286e3f9717c42f2331)

![Abstract visualization of layered data blocks expanding toward a local](https://images.unsplash.com/photo-1518770660439-4636190af475?au&fit=crop&w=1600&q=85)
