AI & Open-Weights Self-Hosting
Articles on open-weights and open-source AI models that enable engineering teams to host frontier-level models on their own sovereign infrastructure, eliminating vendor lock-in and enabling self-hosted deployment of large language models.
Four Local Models Through a Growing 131K-Token Session
A measured four-model benchmark of growing-prefix sessions on an M4 Max: first-token latency, memory, and tuning as context grows toward 118K tokens, with cached follow-ups. 131K is configured context capacity, not prompt length.
Quantized Llama Checkpoints for AI Edge Infrastructure: What to Test
A practical guide to jsbaicenter’s FP8 and AWQ Llama checkpoints, edge infrastructure, runtime checks, and deployment evaluation.
Running Llama 2 Chat in Transformers: A Practical Memory Guide
Load NousResearch/Llama-2-7b-chat-hf with Transformers, understand the model’s reported weight footprint, and plan edge deployments without mistaking weight size for a complete VRAM budget.
GPTQ for Huihui-Qwen3-Coder-Next: Feasibility on AI Edge Infrastructure
A source-grounded assessment of preparing and validating a 4-bit GPTQ checkpoint for Huihui-Qwen3-Coder-Next-abliterated.
Beyond the Model Weights: How PBH Applied Systems Makes GGUF Quantization Deployment-Ready
PBH Applied Systems publishes evaluated, quantized open-weight GGUF models and behavioral evidence to help teams plan practical AI edge infrastructure and reliable self-hosted inference.
Performance Realities in Modern AI Edge Infrastructure: MiniCPM-V-4.7 on Apple MLX and AMD Radeon R9700
Measured MiniCPM-V-4.7 performance on Apple M4 Max and Radeon R9700, including task scores and comparison limits.
Why Shallow Speculative Decoding Won in Our AI Edge Infrastructure Reload Tests
Measured speculative-decoding results across Qwen, MiniCPM-V and OrcaCyber-Nano show why shallow draft depths led in these AI edge infrastructure tests.
Mastering AI Model Local Deployment: A Deep Dive into LiquidAI 24B 4-bit on Apple Silicon
Comprehensive performance analysis and technical deep dive of the LiquidAI 24B 4-bit model pack running on Apple Silicon via mlx-serve, covering edge infrastructure readiness, throughput benchmarks, and quality scorecards.
Can a 2.8-Trillion-Parameter Model Fit on AI Edge Infrastructure? Kimi K3’s Deployment Math
A deep dive into the architecture, quantization, and deployment realities of Moonshot AI's Kimi K3, examining how MXFP4 quantization and Stable LatentMoE impact API serving and self-hosting economics on ai edge infrastructure.
Z-AI's GLM-5.2: A Game-Changer for Sovereign AI Infrastructure
Z-AI's open-weights GLM-5.2 model outperforms GPT-5.5 in long-horizon coding benchmarks at 1/6th the cost, enabling engineering teams to host frontier-level AI on their own sovereign infrastructure.