`, no footer and no links — which is why the compliance score collapses. At 800 tokens **neither** config repeats a line, so whatever this is, it is length-dependent. **It is rare, and our first write-up of it was wrong.** The identical greedy build was re-run four more times on the same prompt: ``` run 1 10,785 tok finish=stop max repeated content line 3 clean run 2 11,006 tok finish=stop max repeated content line 3 clean run 3 12,252 tok finish=stop max repeated content line 4 clean run 4 14,000 tok finish=length max repeated content line 1 clean (long, not looping) ``` **0 of 4.** Pooled with the original, the observed rate is **1 in 5** — not something greedy does, something greedy sometimes does. Greedy decoding is **not bit-reproducible on this stack** (NVFP4 GEMM variance on sm_121 — the same effect behind the 10× token spread documented in the thinking section above), so `temperature: 0` names a *distribution*, not one trajectory. A small slice of that distribution lands in a basin greedy cannot leave. A sampled decoder can land in the same basin and still escape by chance — that asymmetry, not the loop itself, is the finding. The two sampled arms are **one run each**. n=1 bounds nothing; treat their rate as unmeasured, merely lower. ### Why we are NOT claiming "temperature 0 causes this" ⚠️ **There is an uncontrolled confound, and it is a big one.** We run `--sampling-backend flashinfer`. [tonyd2wild's recipe](https://github.com/tonyd2wild/qwen3.8-flash-next-nvfp4-dgx-spark) for the same model and hardware documents a degenerate-output loop and attributes it to that exact kernel — his fix is `--sampling-backend pytorch`, described as ruling out "the FlashInfer kernel arg-maxing a stale row to token 0." With his four-part stack he reports the loop **clean at temp 0.0 / 0.2 / 0.7**, with a residual edge only at temp 1.0. We ship **two of his four** loop-fix elements (`enable_thinking: false`, `--disable-cuda-graph-padding`) and not the other two (`--sampling-backend pytorch`, `--disable-radix-cache`). So the honest statement is: > A long greedy generation looped **on a stack missing the sampling-backend fix that a published > recipe says prevents exactly this class of failure.** Temperature is correlated with the failure > in our three runs; it is not established as the cause. Our manifestation also differs from his — a whole CSS line repeated 507 times, not a token-0 `!` loop — so they may be different bugs. **Unresolved.** Testing it properly means restarting the engine with `--sampling-backend pytorch` and re-running all three arms, which we have not done. (Do not reach for `--disable-radix-cache` casually as the other half of his stack: his own 2026-08-27 update reports it *silently collapses the mamba/SSM state pool* to `max_running_requests`. Our workload is also prefill-dominated, which is precisely where a prefix cache pays.) ### And no, we cannot tell you greedy is faster An earlier version of this section claimed temperature 0 was **+6.9%** faster (48.2 vs 45.1 tok/s), citing higher speculative acceptance (58.3% vs 50.8%) as the mechanism. **That claim is withdrawn.** It came from **n=3 per arm**, against a measured inherent CV of **~6.3%** on this cluster — the "difference" was the same size as the noise, and the defaults arm contained a 42.3 outlier of exactly the shape this log has previously root-caused to page-cache pressure. This repo's own standard, set after an earlier bad call, is that **n=5 is not enough** to report a config win. n=3 is not close. The acceptance figures (2.75 vs 2.525 accept-length) are real per-run gauge reads and the direction is mechanically plausible — greedy tokens are more predictable, so the drafter hits more often. Plausible is not measured. If you want this number, it needs n≥16 with page cache evicted and a named prompt. **Practical guidance, as far as it is actually supported:** - **Long generation on a flashinfer-sampling stack** → send no sampling parameters, or cap temperature at ≤0.7 per tonyd2wild. Both completed cleanly here — *one run each*, so this is a completion, not a rate. The reason to prefer them is the escape asymmetry above, not a measured difference in loop frequency. - **A 1-in-5 chance of losing the whole document is worth engineering around** even though it is rare. If you run greedy on long output, treat `finish_reason == "length"` plus a high repeated-line count as a retryable failure. - **On the server-side default there is a real trade, and we have not resolved it.** We leave `sampling_defaults='model'`, which serves temp 1.0 to any client that sends nothing — and temp 1.0 is precisely where tonyd2wild reports his residual edge, with an explicit recommendation to cap agent temperature at ≤0.7. Keeping `model` preserves the diagnostic signal and honours the checkpoint's own config; it also defaults silent clients into the one regime the cited source calls risky. Pick deliberately rather than inheriting it, as we did. - **Benchmarks** → always say which sampling config produced the number. The direction is plausible; the size, and whether it exists at all, is unmeasured. --- ## Capability evaluation 13 tasks across backend Python, backend Node/TS, SQL schema design, debugging, three frontend stacks (vanilla, React, Next.js App Router), Sanity CMS schemas, and multi-file cross-file debugging. Two passes per arm, temp 0. **8 of the 13 are graded by executing held-out tests in a sandbox** (backend Python ×2, Node ×2, SQL, debugging ×2, and the cross-file task). The three frontend tasks, the Sanity schema and one large-codebase task are graded by **structural checks on the output text** — weaker, and the negative controls validate only the executing verifiers. | arm | scored PASS | INVALID | wall clock¹ | |---|---|---|---| | thinking **OFF**, pass 1 | **13 / 13** | 0 | **3 m 44 s** | | thinking **OFF**, pass 2 | **13 / 13** | 0 | **3 m 45 s** | | thinking **ON**, pass 1 | 12 / 12 | 1 | **81 min** | | thinking **ON**, pass 2 | 11 / 11 | 2 | **84 min** | _{¹ Wall clock between arm-start markers in the run log — this is what you actually wait for.
It is much larger than the sum of per-task `elapsed`, because retried attempts are not counted in
the per-record figure and one frontend task alone burned ~3 × 400 s per thinking-ON arm.} **Thinking off is ~22× faster in wall clock, with equal correctness.** Both INVALIDs in pass 2 were thinking-budget exhaustion at 20,000 tokens on long-output tasks. **A fourth independent measurement of the runaway, from the campaign itself:** retries fired in **10 of 30 thinking-ON cells and 0 of 32 thinking-OFF cells**. The thinking-ON scoreline is therefore *retry-dependent* — retries only fire on INVALID (truncation or empty output), never on FAIL, so they cannot turn a wrong answer into a pass, but they do resample a nondeterministic coin-flip. Without the retry policy the thinking arm would show ~30% failures that are not capability failures. ⚠️ **A clean sweep measures the suite, not the model.** 13/13 bounds the failure rate; it does not locate the ceiling. The 95% Wilson interval on 13/13 is roughly **77–100%** — wide, because n is small. The honest reading is "this suite sits below the model's capability", not "this model does not fail". These tasks were written by us and are not a public benchmark. **Disclosure:** first-pass results under two buggy verifiers were 12/13. `fe-01` (both OFF passes) and `dbg-02` (ON pass 1) were **re-run after the verifier fixes described below** — fresh generations, not re-grades. The headline includes those re-run cells. The hardest task — a four-file service with a cross-file contract bug (a heap negating priority while the constants documented the opposite convention) — passed in **18 s** with thinking on (under 4 s with it off), changing only the file that needed changing, fixing the misleading comment that caused it, and satisfying a held-out three-part test covering ordering, FIFO tie-break, and untouched retry semantics. ### Verifier validation Every run includes negative controls whose tests are **deliberately unsatisfiable**: a Python task asserting `2+2==5`, and a SQL task whose table is pre-created so the model's own DDL must collide. Both failed correctly in all four arms, and real tasks pass — so the verifiers genuinely execute and are not merely always-fail. ### ⚠️ Three of our own checks failed *correct* output This is the part worth copying if you build something similar. In the first pass, three verifiers produced confident, specific, **false** results: | check | what it did | reality | |---|---|---| | mutable-default fix | asserted the caller's list must not be mutated | the prompt never asked for a defensive copy; taking ownership is a normal contract | | self-contained HTML | banned the substring `http://` | flagged `xmlns="http://www.w3.org/2000/svg"` — a namespace URI browsers never fetch | | token budget | 6,000 max_tokens | thinking consumed it, so truncation looked like failure | Uncorrected, the writeup would have claimed this model fails the classic mutable-default bug and cannot produce self-contained HTML. Both are the opposite of true. **A verifier is a claim about the world and needs its own negative controls** — ours caught the model's failures fine; what they could not catch was themselves. The tell each time was a *surprising* failure that turned out, on reading the actual output, to be correct. --- ## Dead ends — documented so you don't spend the time **Clock headroom does not exist.** `clocks.max.sm` reports 3003 MHz; the GPU runs 2528 under load. Locking `-lgc 2800,3003` yields **2528 MHz** — the floor does not take — and prefill changes by **0.07%**: | | clock | prefill | |---|---|---| | default | 2528 MHz | 3,055 tok/s | | locked 2800–3003 | 2528 MHz | 3,053 tok/s | GB10 is **memory-bandwidth bound**, not clock bound, for both prefill and decode. This also explains the low power draw — the GPU is mostly waiting on memory. **Raising the draft budget is impossible.** See §3. **`--load-format dummy` should not be used on GB10** — the rule and the >150 GB transient figure are Mia's; our contribution is only that it explains one of our own wedges. A "safe rehearsal" is more dangerous than the real load. --- ## Thermals — much cooler than a comparable dense-ish MoE Measured at concurrency 4, 1 Hz telemetry: ``` idle 42.0 °C · 10.4 W · 2398 MHz peak (load) 52.0 °C · 35.5 W · 2522 MHz ``` For scale, a previous model on identical hardware peaked at **88 °C / 65 W** uncapped. Qwen runs ~36 °C cooler at ~45% the power — at *higher* clocks. Cause: ~6B active params/token (10 of 512 experts) and only 12 of 48 layers are full attention; the rest are linear-attention GDN. **Practical effect:** thermal guard stages sized for the older model are effectively unreachable (43 °C of margin), and a clock cap intended to control thermals has nothing left to control. --- ## Benchmark discipline Two spectacular false results were produced and caught during this work. Both were **prefix-cache artifacts**: ``` "72,000 tok/s prefill" -> identical prompt repeated, radix cache hit "46,388 tok/s prefill" -> a shell function that never passed its seed argument true prefill: 3,050 tok/s (unique prompt per run, cached_tokens=0 asserted) ``` **Always assert `usage.prompt_tokens_details.cached_tokens == 0`** when measuring prefill. A 24× speedup that appears without a config change is a cache hit, not a discovery. Likewise, **`ignore_eos` benchmarks are a floor, not real-world throughput.** They force generation past the natural stopping point into degenerate text: | measurement | tok/s | |---|---| | real generation (10.7k tokens of HTML) | **62.9** | | harness, `ignore_eos`, hard prompt, 800 tok | 47.6 | Both correct; they measure different things. Name the prompt, token count and clock state on every number, or it is not comparable to anything. --- ## Files | | | |---|---| | [`scripts/cache-warden.py`](scripts/cache-warden.py) | bounds page cache during and after load; no root, no engine patch | | [`bench/decode-bench.py`](bench/decode-bench.py) | decode benchmark that waits for idle, discards contended runs, reports medians, and names its conditions | ## Credit This work stands on two recipes published first, and would not exist without them: - **[MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks](https://github.com/MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks)** — the orchestration and the SM121 QSA Triton fallback kernel that makes this model run on sm_121 at all. Our deployment *is* this stack. - **[tonyd2wild/qwen3.8-flash-next-nvfp4-dgx-spark](https://github.com/tonyd2wild/qwen3.8-flash-next-nvfp4-dgx-spark)** — `--disable-cuda-graph-padding`, NCCL channel pinning (which fixed a 64-channel init hang for us), KV pinning, and the rule that any fix making the model text-only is off the table. - **[bird/GLM-spark](https://github.com/bird/GLM-spark)** — published the `posix_fadvise(DONTNEED)` page-cache mechanism first, as an in-loader vLLM patch. We arrived at it independently and measured it before finding theirs; `cache-warden.py` is an out-of-tree, no-root variant that also bounds cache *during* and *after* load. ## License MIT.