AI Models
Model releases, benchmarks, inference and fine-tuning.
Local AI Model Deployment on the Radeon AI PRO R9700: Speed and Tradeoffs
A measured comparison of eight local model configurations on a Radeon AI PRO R9700 for ai model local deployment, emphasizing decode throughput, suite completion time, truncation, and evaluation limits.
Flint on the R9700: AI Model Local Deployment Performance and Reasoning Realities
A measured R9700 evaluation of Flint 40B-A2.5B, examining its strong decode rate alongside suite duration, capped responses, coding results, and long-context setup observations.
Practical AI Model Local Deployment: GSQ-RCO IQ3_S on the R9700
A measured R9700 comparison of Qwen3.8-27B GSQ-RCO IQ3_S and Q4, focused on footprint, task scores and decode speed.
AI Model Local Deployment: What Quantization Results Really Show
Measured MLX and Radeon results reveal trade-offs among quantization, task scores and decode speed—without pretending unlike model packs form a controlled sweep.
Navigating AI Model Local Deployment: What Two MiniCPM Runs Can—and Can't—Tell Us
Measured context-length prefill and decode observations for MiniCPM-V-4.7 on Apple M4 Max and Radeon R9700, exploring the limits of non-like-for-like comparisons in local model deployment.
Configuring and Serving Local AI Model Deployment on a 32 GB Radeon Workstation
Practical guide and measured performance observations for local ai model deployment on a 32 GB Radeon AI PRO R9700 workstation using llama.cpp and Vulkan.
How to Deploy an AI Model Locally: Qwen’s Cached Context Probe on M4 Max
A four-turn growing-prefix probe of local Qwen3.8-Flash-Next on M4 Max, with measured latency, memory and decoding results—and explicit limits.
Four Local Models Through a Growing 131K-Token Session
A measured four-model benchmark of growing-prefix sessions on an M4 Max: first-token latency, memory, and tuning as context grows toward 118K tokens, with cached follow-ups. 131K is configured context capacity, not prompt length.
AI Model Local Deployment: MiniCPM and Qwen3.8 Flash in a Growing 131K Session on M4 Max
A measured comparison of two local MoE models in a four-turn growing-prefix probe, with explicit limits on context, latency, and throughput claims.
KAT-Coder V2.5 at 131K: An AI Model Local Deployment Probe on the M4 Max
Measured four-turn, growing-prefix results for KAT-Coder V2.5 (OptiQ 4-bit) at up to 118K tokens of live context on a single Apple M4 Max, with tuned versus baseline 8-bit KV cache timings.
Converting a Hugging Face Model for mlx-serve: A Practical Guide to AI Model Local Deployment
A step-by-step field guide for adapting a public Hugging Face model to mlx-serve — architecture identification, preserving config and tensor names, quantization choices, validation, and benchmarking for reliable AI model local deployment.
AI Model Local Deployment Refresh: LFM2.5-24B-A2B Mixed 4/8-Bit vs Stock 4-Bit on the Standard v2 Suite
Like-for-like standard-v2 results comparing the LFM2.5-24B-A2B mixed 4/8-bit pack against the stock 4-bit pack on mlx-serve. Raw MMLU-Pro, GSM8K, coding, throughput, and memory numbers — no composite winner.