AI general intelligence definition: a score needs its settings
A benchmark can execute exactly as configured and still leave readers with the wrong impression. In our Sage-A1.5 test, thinking mode with temperature-zero decoding kept checking its own work on a simple math question until it consumed the entire 8,192-token budget—without answering. The model’s recommended sampling settings changed what happened.
That’s not proof that greedy decoding is generally bad, or that a benchmark should conceal an ugly result. It is a reminder that the score describes a particular configuration, not a context-free measure of intelligence. If we’re asking what a benchmark says about a model’s useful behavior, sampling, thinking mode, prompt format and answer budget all matter.
There is no single AI general intelligence definition hiding inside a percentage. An evaluation measures performance on its prompts under its rules. Those results can be useful, but only if the rules stay visible.
Setup
Setup. AMD Radeon AI PRO R9700 (32 GB GDDR6); desktop AM5 platform, DDR5-6000; Windows 11; llama.cpp llama-server, Vulkan backend, build b11836; flash attention on; 131K context unless noted; ubatch 2048; four parallel slots sharing one unified KV cache. Tested October 2026.
The Sage-A1.5 model card describes the public build as a Qwen3.5-MoE-derived 35B-A3B class model and documents changes that include refusal-direction editing. It recommends temperature 1.0, top-p 0.95, top-k 20 and presence penalty 1.5. That recommendation is relevant context for a temperature-zero result, not a reason to erase it. The card also warns about factual-reliability limitations; its claims are not a substitute for independent evaluation.
Our standard-v2 suite used temperature zero and capped answers at 2,048 tokens. The looping observation was a separate prompt-level check: a trivial GSM8K item with an 8,192-token generation budget. Keep those two budgets distinct. One describes the suite rule; the other describes the test that exposed the loop.
What the loop tells us about model behavior
With thinking on under greedy decoding, Sage-A1.5 repeatedly self-verified instead of producing the requested answer. On the simple GSM8K item, it used all 8,192 generated tokens and gave no answer. The larger prompt-level budget did not rescue that run; it only let the loop continue longer.
We then used the model card’s recommended sampling settings. The model answered correctly in about 3,400 tokens. That response would still exceed the standard-v2 suite’s 2,048-token cap and be truncated under that rule. With thinking off, it answered correctly in about 230 tokens.
Those are three observed examples, not an average and not a promise about what every prompt will do:
| Mode and sampling | Observed response |
|---|---|
| Thinking on, temperature 0 | Used all 8,192 tokens without answering |
| Thinking on, model-card sampling (temperature 1.0, top_p 0.95, top_k 20, presence penalty 1.5) | Correct answer in ~3,400 tokens |
| Thinking off | Correct answer in ~230 tokens |
We did not repeat these prompt-level checks. So the table is evidence about these runs, not a universal behavior rule. We also did not record an exact generation time for the long loop, and token count alone doesn’t let us calculate one. No time estimate belongs here.
The report-card impact
The wider shoot-out recorded Sage-A1.5 Q4_K_M with thinking on and greedy decoding at 55.25% MMLU-Pro, 26% GSM8K and 0/6 coding. There were 170 MMLU-Pro answers at the cap, and the suite took 189 minutes.
With thinking off, the re-bench scored 86.75% MMLU-Pro, 98% GSM8K and 6/6 coding. It had 29 MMLU-Pro items at the cap; the standard row decoded at 195 tokens per second and completed the suite in 59 minutes.
That’s a large measured difference. It also changes the model mode and benchmark behavior, so it doesn’t isolate one hidden capability or prove that thinking should always be disabled. Explicit reasoning traces may be useful for a given task. In this suite, the greedy thinking loop was damaging. The settings belong next to the score, not in a footnote that readers never see.
Why the benchmark can mislead without lying
The benchmark didn’t fabricate the poor result. It recorded what happened under the declared temperature and thinking configuration. The misleading step comes later: seeing only the metric and treating it as a fixed, context-free ranking of Sage-A1.5.
An evaluation can be numerically correct and still say less about intended use than a reader assumes. A useful local-model report should state the sampling settings, thinking mode, answer cap and prompt set, and say whether outputs were truncated. When a model card recommends non-greedy sampling, disclose that before comparing a temperature-zero baseline with the recommended setup. If a mode spends its entire token budget without answering, keep the failure visible. Don’t discard it as an outlier without evidence.
This is one part of a broader AI research methodology problem: measurements depend on what was held constant and what changed. The R9700 quantization comparison looks at specific model builds and draft settings. The R9700 speculative-decoding investigation concerns target-draft matching. And the R9700 model shoot-out reports truncation counts because a percentage by itself can hide a lengthy answer that ran into the cap. Those related articles are not published yet, so we name them here without linking to them.
Practical guidance for local model tests
For a Sage-A1.5 deployment, start with the model card’s sampling settings and make an explicit choice about thinking mode. Check that the generation budget can accommodate the answer length you expect. Include representative prompts, including tasks with short, verifiable answers, and inspect the generated text rather than trusting completion status or aggregate accuracy alone.
For a controlled benchmark, keep temperature-zero results when comparability requires them. Label them clearly, and report recommended-sampling behavior separately. Don’t blend the two modes into one score. Two results can explain behavior better than one, but only when the settings difference is explicit.
That’s the narrow, useful conclusion here. This case identifies the configuration that triggered a loop and compares it with two alternatives. It does not establish that greedy decoding is always wrong, or that thinking should be switched off for every reasoning task. The result is about this model, these prompts and these settings—not a universal verdict on AI models’ advancements, nor a shortcut to an artificial intelligence general intelligence definition.
Disclaimer. These are measurements from one machine, one server build, and one test suite on the stated date. Results vary with hardware, software version, settings, prompt, and load. They are not a general ranking or a recommendation.
