Key takeaway: ISTA-DASLab’s GSQ-RCO IQ3_S brought Qwen3.8-27B down to an 11.3 GiB target package in our R9700 setup while scoring 85.25% on MMLU-Pro and completing five of six coding tasks. That’s a compact option with useful measured capability. The larger Q4 target decoded slightly faster and scored higher on MMLU-Pro.
Model choices for AI model local deployment
The Qwen3.8-27B model card describes a dense, hybrid-attention vision-language model with 27B parameters, all active, 262K native context and Apache 2.0 licensing. It gives different sampling recommendations for thinking and non-thinking instruction prompts. For thinking mode, the card recommends temperature 1.0, top-p 0.95, top-k 20, min-p 0, presence penalty 0 and repetition penalty 1. For non-thinking instruction prompts, it recommends temperature 0.7, top-p 0.80, top-k 20, presence penalty 1.5 and repetition penalty 1.
The ISTA-DASLab Qwen3.8 GSQ-RCO model card describes a family of per-tensor GGUF quantizations. GSQ is Gumbel-Softmax Quantization, a post-training scalar quantization method that optimizes discrete low-bit assignments. RCO chooses a mixture of quantization types across tensors to meet a target whole-model bit budget. Instead of assigning every tensor the same precision, this method can allocate bits differently by tensor, and the repository includes per-tensor allocation data and an importance matrix for auditing the package. The IQ3_S target in that repository is listed as 3.50 bits per weight and 11.8 GB on its model card; our local measurement was 11.3 GiB. The card’s GB listing and our GiB measurement use different size units. The base Qwen license applies to the quantized weights, while the quantization tooling has its own repository license. The card calls IQ3_S its recommended operating point and reports its own comparisons on AIME25, LiveCodeBench and GPQA-Diamond; those are separate from our R9700 results.
What we measured
The R9700 suite used the GSQ-RCO IQ3_S target with the built-in MTP d2 head. As with the other rows, we ran 400 MMLU-Pro items, 50 GSM8K questions and six coding tasks at temperature 0, a 2,048-token answer cap and parallelism four. Parenthetical counts denote results that reached the cap. Setup. AMD Radeon AI PRO R9700 (32 GB GDDR6); desktop AM5 platform, DDR5-6000, Windows 11; llama.cpp llama-server, Vulkan backend, build b11836; flash attention on; 131K context; ubatch 2048; four parallel slots sharing one unified KV cache. Tested October 2026.
| Target and run | Footprint | MMLU-Pro | GSM8K | Coding | Decode | Suite wall time |
|---|---|---|---|---|---|---|
| GSQ-RCO IQ3_S, built-in MTP d2 | 11.3 GiB | 85.25% (39 at cap) | 92% (3 at cap) | 5/6 | 57 tok/s | 149 min |
| Unsloth UD-Q4_K_XL, MTP d2 | 17.6 GB file listing | 86.75% (31 at cap) | 94% (1 at cap) | 5/6 | 62 tok/s | 149 min |
The Q4 row provides a nearby reference, not a fully controlled quality-only comparison: both used MTP depth 2, but they use different quantization methods and the file size column follows the public listing units. In this suite, the IQ3_S package used less storage and was five tok/s slower. Its MMLU-Pro score was 1.5 points lower; GSM8K was two points lower; coding completion matched. The suite wall time was equal at 149 minutes. That combination makes IQ3_S an interesting compact target when storage or memory pressure matters more than a small rate advantage. A short decode sweep found the same depth relationship on GSQ: code/prose decode was 66/61 tok/s with MTP depth 2 versus 57/44 tok/s at depth 4. The full standardized suite's median decode number was lower at 57 tok/s. These measurements differ in prompt shape and run length, so they answer different questions. We should not substitute a short decode sweep for end-to-end task results.
Recommended run settings
Use the IQ3_S GGUF with its built-in MTP head at depth 2 as the starting configuration for a compact local R9700 setup. The tested context was 131K, ubatch 2048, four parallel slots and shared KV cache. Our accuracy suite used temperature 0 and a 2,048-token answer cap; Qwen’s model-card settings for natural generation are temperature 1.0/top-p 0.95/top-k 20 in thinking mode, or temperature 0.7/top-p 0.80/top-k 20 with presence penalty 1.5 in non-thinking instruction mode. Change the sampling to match the intended task and evaluate quality again rather than assuming the suite score applies. GSQ-RCO may also be attractive when a reader wants to inspect how precision was allocated. The public card lists several lower and higher points, from IQ2_XS through IQ3_S, and explicitly describes IQ3_S as task-lossless on its own selected trio of benchmarks. Our local score suite is different and not lossless against Q4. Use the phrase “compact with strong measured results” rather than extending the model card’s claim to tasks we did not test.
Verdict
IQ3_S offers a practical lower-footprint Qwen3.8 route on the R9700. Our run preserved five of six coding completions and kept MMLU-Pro above 85%, with a decode rate close to the larger Q4 target. If every point of benchmark score or a few extra tokens per second matters, choose Q4 and compare against your own tasks. If the 11.3 GiB measured package better fits your available memory budget, GSQ-RCO IQ3_S is the more compelling trade-off in this series. Related reading: “MiniCPM-V 4.7 on the R9700: a fast multimodal MoE, with a DSpark caveat”; “Sage-A1.5 on the R9700: why thinking mode changed the score”; “Swift 1.5 versus stock Qwen3.8-27B on the R9700”; “Qwen3.8-27B on the R9700: MTP depth and quantization trade-offs”; and “Flint 40B-A2.5B on the R9700: fast decode, longer work”.
Disclaimer. These are measurements from one machine, one server build, and one test suite on the stated date. Results vary with hardware, software version, settings and load. They are not a general ranking or a recommendation.
