AI consumer hardware, tested locally
A fast local-inference card matters only if the models fit, the software behaves, and the output is useful. In our Vulkan llama.cpp test, the AMD Radeon AI PRO R9700 ran selected workloads at up to 195 decode tokens per second and held the listed models at 131K context. That's a strong showing for this setup, not a universal ranking of AI consumer hardware.
The test also has hard boundaries. It covers one machine, one software build and a limited set of workloads. Model quality varied, a Windows memory-mapping behavior affected system RAM use, and we did not measure power, noise or cost per token. The right verdict has to include those gaps.
Setup
Setup. AMD Radeon AI PRO R9700 (32 GB GDDR6); desktop AM5 platform, DDR5-6000; Windows 11; llama.cpp llama-server, Vulkan backend, build b11836; flash attention on; 131K context unless noted; ubatch 2048; four parallel slots sharing one unified KV cache. Tested October 2026.
The AMD Radeon AI PRO R9700 is the hardware under test. We used Vulkan, not ROCm. Model cards for Qwen3.8-27B, MiniCPM-V-4.7, Sage-A1.5, and Flint describe their architectures, licenses and intended capabilities. Those descriptions don't guarantee the quality or speed of the quantizations we tested.
Where the R9700 worked well
The 32 GB memory budget is the clearest practical strength here. Every listed model fit at 131K context with ubatch 2048. MiniCPM-V used about 24.4 GiB; Flint used about 27.6 GiB. That leaves less slack than the headline capacity might suggest: a larger 27.9 GiB Flint Q5 variant spilled into system memory.
A separate Windows observation revealed a memory-mapping quirk. With memory-mapped loading, MiniCPM-V's GGUF remained resident in system RAM after uploading to VRAM (17.7 GB). Loading without mmap reduced system RAM use to about 1 GB. That is a specific behavior seen in this configuration, not a claim that every Windows setup will behave the same way.
Decode speed was strong on the workloads we measured. Sage-A1.5, with thinking off and built-in MTP depth two, reached 195 tokens per second. MiniCPM-V reached 139 tokens per second and completed the standard suite in 48 minutes. See Local AI Model Deployment on the Radeon AI PRO R9700: Speed and Tradeoffs for the detailed model results.
We also compared the same mixed-quant MiniCPM-V recipe on an Apple M4 Max. The two systems scored the same MMLU-Pro and GSM8K percentages; the R9700 decoded at 139 tokens per second, against 66.8 tokens per second on the M4 Max, approximately 2.1 times faster. The coding result differed by one task. This is a narrow same-recipe speed comparison, not a complete platform ranking.
Setup choices mattered. In the observed 16K case, keeping the integrated GPU as the primary display restored decode performance. Desktop and vendor RGB rendering on the R9700 had coincided with a decline from 112 to 73 tokens per second. Re-Size BAR, UEFI boot, XMP/EXPO memory at DDR5-6000, and ubatch 2048 were part of the tested configuration. Configuring and Serving Local AI Model Deployment on a 32 GB Radeon Workstation covers the broader workstation setup.
Model quality and evaluation limits
Capacity and speed don't make models interchangeable. In the shared MMLU-Pro/GSM8K/coding suite, MiniCPM-V scored 90% and 100%, with 5/6 coding, while Flint scored 71.25%, 92%, and 4/6. Those results apply to the tested models and configurations; they aren't a general measure of every use case.
The Sage-A1.5 comparison showed another practical wrinkle: thinking-on greedy decoding could loop. The thinking-off re-bench scored 86.75% MMLU-Pro, 98% GSM8K, and 6/6 coding; thinking-on greedy scored 55.25%, 26%, and 0/6. Thinking and sampling settings are part of the experience, not small footnotes to throughput.
Our suite used 400 MMLU-Pro items, 50 GSM8K questions and six coding tasks, at temperature zero, with a 2,048-token answer cap. Many MMLU-Pro answers reached that cap, so the scores need to be read with truncation in view. The findings also depend on a particular llama.cpp build, Windows configuration, quantization artifacts and benchmark suite. Results may change with driver versions or another application stack.
Vulkan support worked for this server build. We did not evaluate ROCm, and this article shouldn't be read as a ROCm result. Nor is it an all-purpose statement about local AI software support.
What this test cannot tell you
We didn't measure board power, system power, acoustic output, thermals over an extended run, price, cost per million tokens or long-term reliability. No price appears here because we have no dated public purchase source. We also don't have a direct same-model comparison against a broad set of current GPUs.
Those are material omissions. Without them, we can't make a cost-efficiency or value-for-money verdict, and we can't say how the card would behave in a sustained, quiet workstation or under another user's workload. A useful result isn't the same thing as a complete buying guide.
Who may find this useful
The R9700 is worth considering for someone who needs a 32 GB local model target, has a compatible desktop platform and is comfortable tuning an open inference stack. It's particularly interesting if local model residency and measured decode speed matter more than a turnkey ROCm-specific workflow.
The results also suggest practical options for users able to keep display rendering on an integrated GPU and test prompt batch and context against their actual workload. Buyers who need a verified energy budget, low-noise workstation, simple one-click model support or a proven cost advantage over alternatives have less to go on here. Those needs require measurements or platform comparisons that this test doesn't provide.
Before committing, confirm the model, quantization, software backend, context length and concurrency you intend to use. Small differences in those choices can change both memory headroom and the experience. The companion articles—R9700 capacity table, model report card, speculative decoding results, quantization comparison, Sage loop analysis, and workstation setup—cover those narrower questions in more detail. They are not published yet.
Verdict
For the workloads we measured, the AMD Radeon AI PRO R9700 is good for local AI. It offered a substantial 32 GB memory pool, high decode rates in several configurations, and enough room for the tested models at 131K context. The best results required deliberate setup and model choice.
That supports a favorable technical verdict for Vulkan-based local inference, with caveats around evaluation setup and spillover. It doesn't establish a universal buying recommendation or a complete comparison against competing GPUs. Power, noise, price, cost efficiency, sustained thermal behavior and broader GPU comparisons remain open data gaps.
Disclaimer. These are measurements from one machine, one server build, and one test suite on the stated date. Results vary with hardware, software version, settings, model, and load. They are not a general ranking or a recommendation.
