AI & Local Model Deployment
Articles on deploying and running AI models locally, including open-source model ecosystems (LM Studio, Ollama, etc.), on-device inference, hardware requirements, and the shift from cloud-hosted to self-hosted AI development tools.
LFM2.5-24B-A2B on MLX-Serve: Mixed 4/8-Bit Conversion and Benchmarks
A focused guide to the public LFM2.5 mixed-precision conversion for local AI deployment, its tensor choices, and recorded results.
Optimizing AI Model Local Deployment: Navigating Multimodal Quantization Beyond AutoAWQ Deprecation
A practical guide to preparing fine-tuned multimodal models for local deployment, diagnosing quantization wrapper failures, and validating exports before serving them.
AI Model Local Deployment: Qwen Flash Next Through a Growing 131K Session
A four-turn cached-prefix probe on M4 Max reports measured readiness, evaluation and short-decode results—and why cache reuse is not a cold full-window test.
MiniCPM-V 4.7 for AI Model Local Deployment: R9700 Results and DSpark Tuning
Measured MiniCPM-V 4.7 results on an AMD Radeon AI PRO R9700, including same-recipe M4 Max comparisons and DSpark draft-depth trade-offs.
How to Deploy an AI Model for Local Agents: M4 Max or Radeon?
Measured MiniCPM-V-4.7 results offer a starting point for choosing a local AI deployment path, but agent workloads demand more than decode speed.
Practical AI Model Local Deployment: Benchmarking Qwen3.8-27B Multi-Token Prediction Depths on the R9700
Practical AI model local deployment analysis benchmarking Qwen3.8-27B multi-token prediction depths and quantizations on the AMD Radeon AI PRO R9700.
AI Model Local Deployment: What Quantization Results Really Show
Measured MLX and Radeon results reveal trade-offs among quantization, task scores and decode speed—without pretending unlike model packs form a controlled sweep.
Navigating AI Model Local Deployment: What Two MiniCPM Runs Can—and Can't—Tell Us
Measured context-length prefill and decode observations for MiniCPM-V-4.7 on Apple M4 Max and Radeon R9700, exploring the limits of non-like-for-like comparisons in local model deployment.
Radeon R9700 Qwen3.8-27B Quantization Results: Speed, Scores, and Tradeoffs for AI Model Local Deployment
A measured comparison of three Qwen3.8-27B GGUF configurations on a 32 GB Radeon AI PRO R9700, exploring quantization tradeoffs, deployment practices, and benchmark results.
Local AI Model Deployment on the Radeon AI PRO R9700: Speed and Tradeoffs
A measured comparison of eight local model configurations on a Radeon AI PRO R9700 for ai model local deployment, emphasizing decode throughput, suite completion time, truncation, and evaluation limits.
AI Model Local Deployment: Long-Context Fit on the R9700, Measured
A measured local deployment test of 131K-context memory fit, quantization spillover, batch size, and concurrency on the Radeon AI PRO R9700.
Local Deployment of AI Models on the R9700 Suite: Swift 1.5 vs Stock Qwen3.8-27B
Measured R9700-suite comparison of Swift 1.5 and stock Qwen3.8-27B on MMLU-Pro, GSM8K, and coding tasks, including decode speed, suite wall time, and recommended local run settings.