docs/evaluations/context.md

DCA (Dual Chunk Attention) — Evaluation Results

Workstream: T87.pD — Dual Chunk Attention for long-context prefill in the opencoti llamafile fork (Gemma 4 26B-A4B 1M-context path on a single GPU, + Qwen2/3/3.5 portability). Patch: vendors/patches/llamafile/0078-dca.patch.

⚠ Absolute prefill tok/s in this doc are stale-low as of 2026-07-05 (#614): the tinyblas-small-gemm fix (bug-2104) lifts Gemma-A4B MoE-router prefill substantially on TinyBLAS builds, so cells below understate the current binary; the DCA-on-vs-off ratios and all quality scores remain valid. The #614 checkpoint fixes (bug-2105/2107) do NOT affect these runs — the recipe here pins --ctx-checkpoints 0. Current cross-engine prefill: three-way-tps.md §2026-07-05. Last updated: 2026-06-18 (Thursday) — *added §5 (KV-quant × DCA full-vocab PPL/KLD, #446): teacher-forced KL-divergence resolves the turbo-DCA anomaly — turbo-DCA real_frac=0 is GENUINE (DCA f16-lift KLD monotone in tier bits: turbo8 0.002 → turbo2 0.65), and the DCA lift beats the noDCA in-register turbo read (turbo2 in-register is degenerate: KLD 5.60 / PPL 1584 vs turbo2-DCA 0.65). Scalar/asym matrix complete — best long-ctx KV recipe is high-precision-K + q4/turbo4-V (q8K/q4V & t8K/t4V ≈ KLD 0.003–0.008, top 96–98%); the turbo2 noDCA degeneracy is K-side (t4K/t2V fine). Earlier (2026-06-15 → 2026-07-13): the RULER-VT 1M cell landed (§1a) and is now 0.916 on the current shipping binary — the quality ladder is complete. (History: originally 0.716 = server-crash artifact; 0.876 = 2026-07-01 pre-fix 41+9-refix reconstruction; 0.916 = 2026-07-13 fixed-binary _dcaon_rerun, clean n=50, 0 empties — the current number. See §1a◇.) DCA-on holds the 0.96–0.98 VT band 256k→512k, then softens gently: 768k 0.984, 1M 0.916 (a mild −6.8 pp at 4× native, n_ctx_train=256k) — no cliff. Re-confirmed on the fresh 0121 binary (#654): all four DCA-on rungs now on the current engine — 256k 0.964 / 512k 0.996 / 768k 0.984 / 1M 0.916, n=50, 0 empties (768k the last holdout, re-run 2026-07-13: 0.956→0.984). See the §1a note. Earlier: MRCR n=50 re-run (§1b: 256k-off 0.1138 / 256k-on 0.1320 / 512k-on 0.1027); §4d (#635 decode chunk-boundary gate). See docs/DCA_HANDOFF.md for the full patch hand-off (RCA, file inventory, cross-arch risks).*

This doc collects every DCA evaluation we've run: long-context quality (RULER-VT + MRCR), throughput (prefill/gen tps), memory footprint (VRAM), and correctness gates (multi-chunk coherence, NIAH retrieval). Quality scores are read from the canonical summary.json .score; throughput/VRAM from the llama-server logs (timings, KV/compute-buffer lines) and live nvidia-smi.

---

0. Hosts & methodology

Two hosts produced these numbers — attribute every row to its host, they are not comparable across GPUs:

Host	GPU	VRAM	Role
bs2 (linode-blackswan-2)	1× Blackwell-class	97,887 MiB (~96 GB)	Long-context quality ladder (128e-F16, 256k–1M). Fits weights+KV up to 1M.
solidpc	1× RTX 3090	24,576 MiB (~24 GB)	Throughput (3-arm gen-tps) + correctness gates (multi-chunk coherence, regression). Q4_K_M / quantized models only.

• Sampler: greedy (temperature 0, top_k 1) everywhere — DCA is a serving-correctness feature, scores must be deterministic.
• DCA recipe: --dca on --dca-chunk-size 8192 (so ctx/8192 = chunk count), -fa on --parallel 1 --no-warmup --ctx-checkpoints 0.
• DCA-off above 256k = N/A: full attention KV does not fit a single GPU past 256k; DCA is what makes those tiers reachable, so there is no off-arm to compare against. 256k is the only tier with both arms (the regression anchor).
• Backend: omk_eval.py --backend llama launching the opencoti DCA llamafile (/srv/ml/opencoti-dca/bin/llamafile) per cell, --parallel 1 --replicas 1 (serial, one full-ctx slot).

Models under test

Short name	Arch	head_dim (DKQ)	Quant	Weights on disk	Notes
128e-F16	gemma4 (iSWA)	512	F16 GGUF	50.5 GB (48,150 MiB in VRAM)	Clean unpruned 26B-A4B base. Ladder model.
gemma-4-26B-A4B-it-Q4_K_M	gemma4 (iSWA)	512	Q4_K_M	~17.5 GB	solidpc 3-arm gen-tps model.
v7-coder-it (98e)	gemma4 (iSWA)	512	Q4_K_M	13.2 GB	98e pruned. Reference only — early upper-tier MRCR cells, superseded by 128e-F16 re-runs.
Qwen3.6-35B-A3B	qwen35moe	256	UD-IQ3_XXS	13.2 GB	Qwen multi-chunk portability gate.
Qwen3-4B	qwen3	128	Q4_K_M	2.5 GB	Qwen multi-chunk portability gate.
Qwen3.6-27B-Omnimerge-v4	qwen35 (hybrid, MTP)	256	Q4_K_M	16.8 GB	MTP-with-Qwen capability + DCA multi-chunk gate (#622).

---

1. Long-context quality ladder — 128e-F16 (bs2 Blackwell)

Single model, greedy, ±DCA. DCA-off N/A above 256k (won't fit). Each bench has its own metric — RULER-VT = string_match_all, MRCR = sequence_matcher_ratio (fuzzy ratio, NOT pass@1). For MRCR, prefill/gen tps + VRAM + wall/req are the authoritative values from summary.json scores (prefill_tok_s/gen_tok_s/vram_peak_mib/wall_s_median); RULER-VT's scores block carries no perf fields, so its tps are medians read from the per-cell server.log.

⚠️ MRCR tiers are different synthetic bins, NOT a comparable column. Each MRCR row draws its prompts from a distinct mrcr_bin (256k real / 512k / 768k_synth) with a different needle-count mix, so MRCR scores do not form a monotone "longer = lower" curve and must not be read as one. RULER-VT is a consistent metric across ctx, so its rows are comparable.

1a. RULER-VT (Variable Tracking, 50 samples) — string_match_all

Model: 128e-F16 (Gemma-4 26B-A4B, iSWA, head_dim 512, F16 GGUF — 48.2 GB weights; see §0 "Models under test"). Note the VRAM column grows sub-linearly with ctx (54.5 → 61.3 → 69 → 70 GB): Gemma-4's iSWA means most layers are local sliding-window (KV capped at the window, independent of total ctx) and only the minority global layers (just 2 KV heads × head_dim 512) scale with ctx. So 768k→1M (+262k tokens) enlarges only the global layers ≈ +1 GB — not the ~5 GB an all-global model would add. That sliding-window KV is exactly what fits 1M (KV 20.8 GB) on a single 96 GB card.

ctx	DCA	score	prefill tok/s	gen tok/s	VRAM	wall-clock	status
256k	off	0.964	2275.6 (med, rerun)	88.3 (med, rerun)	~54.5 GB	1h39m	done (#654 rerun)
256k	on	0.964	2323.1 (med, rerun)	87.0 (med, rerun)	~54.5 GB	1h40m	done (#654 rerun)
512k	on	0.996	1752.1 (med, rerun)	67.5 (med, rerun)	~61.3 GB	4h18m	done (#654 rerun)
768k	on	0.984	1407.8 (med, rerun)	55.2 (med, rerun)	~67 GB	8h02m	done (#654 rerun)
1M	on	0.916◇	1175.6 (med, rerun)	46.4 (med, rerun)	~70 GB	12h36m	done (#654 rerun)

#654 re-run on the fresh 0121 binary CONFIRMS the ladder end-to-end — DCA-on 256k 0.964 (was 0.960, 2026-07-12), 512k 0.996 (was 0.984, 2026-07-12), 768k 0.984 (was 0.956 orig-binary; fresh rerun finished 2026-07-13 16:23 UTC, 8h02m), and 1M 0.916 (was 0.876, finished 2026-07-13 04:42 UTC, 12.6 h) — all n=50 with 0 empties (fresh_empty=0, no server-death artifact this run). The 1M rerun is the decisive one: a fully-clean single-pass n=50 (not the 41-perfect + 9-refix reconstruction the 0.876 was) on the post-bug-2118/2120/2121 shipping binary lands +4 pp higher at 0.916, so 0.916 supersedes 0.876 as the true VT-1M ceiling and the gentle 768k→1M step is ~−6.8 pp (0.984→0.916), a soft slope not a cliff. The load-bearing conclusion (near-parity at 256k, band held 256k→512k, no cliff to 4× native) is unchanged and strengthened on the current engine. Results: bs2:/srv/ml/rerun/vt-out/ruler_native_vt_{256,512}k,1024k_dcaon_rerun/.

The rerun tps are ~2–3× the original run (prefill 256k 747→2323, 512k 681→1752, 1M 577→1176; gen also up: 256k 84→87, 512k 65→68, 1M 44→46) — the shipping binary carries every prefill/decode fix landed since June (#554 DCA-prefill-cliff, #614 large-batch prefill, #620 bandwidth-bound quant, WS2 turbo-V). The table's rows are all current-binary rerun medians — every rung, DCA-on (256k / 512k / 768k / 1M) AND the 256k-off anchor, is now on the fresh binary. 768k was the last DCA-on holdout (re-run 2026-07-13: 0.956→0.984, prefill 632→1408, gen 52→55, wall 17h44m→8h02m) and 256k-off was the last full-attention holdout (re-run 2026-07-13: 0.976→0.964, prefill 676→2276, gen 86→88, wall 5h28m→1h39m, n=50 fresh_empty=0). Wall-clock roughly halved across the board (1M 24 h → 12.6 h; 768k 17.7 h → 8.0 h; 256k-off 5.5 h → 1.6 h).

◇ 1M is 0.916 on the current shipping binary (fixed-binary _dcaon_rerun, 2026-07-13, clean n=50 fresh_empty=0) — see the #654 note above; this supersedes the 0.876 below. The 0.876/0.716 history is retained for provenance. 0.876 was the pre-fix reconstruction (resolved 2026-07-01, a 41-perfect + 9-refix rebuild on the pre-bug-2118/2120 binary). The 0.716 was a serving artifact: the first F16 server crashed after sample 40 (~24 h uptime), so samples 41–49 all returned gen-error: ConnectionError (one RemoteDisconnected, then 8× Connection-refused) → 9 forced zeros (fresh_empty=9). The 9 were re-run on a fresh server (cache-resume; the 41 good samples kept, only the 9 dead keys re-predicted, ~4.5 h) and all produced real completions → the re-scored n=50 cell is 0.876 (fresh_empty=0, 35/50 perfect, 3 genuine zeros). Recompute over the pre-fix 50 (dead=0) reproduces 0.716 exactly and over the 41 then-served samples gives 0.873, so the 9 re-runs land right on the served distribution — the crash, not 1M coherence, drove the apparent cliff.

DCA-on vs full attention at 256k, both now on the fresh binary: 0.964 vs 0.964 — exact parity (DCA is lossless at 256k within VT n=50 sample noise; the earlier −1.2 pp was a stale original-binary full-attn 0.976). Speed is at parity too — DCA-on prefill 2323 / gen 87 vs DCA-off 2276 / gen 88 tok/s — so DCA costs neither quality nor throughput at the native cap. This is the load-bearing quality result: on a consistent metric, multi-chunk DCA preserves long-context tracking. At 512k DCA-on holds at 0.996 — above both its own 256k point and the 256k full-attention anchor (both 0.964), i.e. variable tracking does not degrade as context doubles past the native cap (gen tps 67.5, prefill 1752 on the current binary). At 768k DCA-on lands at 0.984 (−1.2 pp vs 512k, comfortably inside the 0.96–1.0 band — no cliff at 3× the native cap; the fresh-binary rerun (2026-07-13, _dcaon_rerun, n=50 fresh_empty=0, 9/50 hitting the short gen cap with the answer present) lifted 768k +2.8 pp from the original-binary 0.956, with prefill 1408 / gen 55 tok/s and wall 8h02m — the last rung to move onto the current engine). At 1M (the final rung, 4× the 256k native cap, n_ctx_train=262144) DCA-on lands at 0.916 (◇ — the current fixed-binary _dcaon_rerun, 2026-07-13; history 0.876 pre-fix / 0.716 server-crash artifact in the footnote). That is a mild −6.8 pp vs 768k (0.984), not a cliff: variable-tracking softens gently at 4× native (0 empties, 20/50 hitting the short gen cap with the answer present) rather than breaking. The soften is not generation-truncation (VT caps gen at 120 toks; the 20/50 "length" finish-reasons used the full short cap with the answer present, 30/50 "stop"). Current binary: prefill 1176 tok/s, gen 46.4 tok/s on a 1.05M-token prompt, wall 12.6 h (was ~24 h on the original binary); peak VRAM ~70 GB (model 48.2 + KV 20.8 + compute 3.2 GB). The whole ladder is now re-confirmed on the shipping engine (#654): DCA-on holds 256k (0.964) → 512k (0.996) → 768k (0.984) → 1M (0.916) — all four rungs now on the fresh binary — with only a gentle step down at 1M and no server-death artifact (fresh_empty=0 at every rung). The original-run history — first 1M attempt crashed after sample 40 on 2026-06-15, 9-sample re-fix 2026-07-01 giving 0.876 — is retained in the ◇ footnote.

1a-yarn. --dca-yarn-factor (bug-740 / patch 0087) — characterized NO-WIN

The --dca-yarn-factor flag was plumbed end-to-end but never read by the rope path (inert; bug-740). Patch 0087 wires it into build_dca_rope as the YaRN mscale: dca_attn_factor = attn_factor * cparams.dca_yarn_factor → ggml_rope_multi/ggml_rope_ext. Host-only; default 1.0 is byte-identical (×1.0f bit-identity). Empirically characterized on Qwen3-4B (native 40960, DCA chunk 16384, RULER 5-needle substring under ignore_eos — never greedy-first-token):

KV	ctx	×native	yarn 1.0	1.1	1.25	1.5	2.0
f16	98304	2.4×	5/5	—	4/5	1/5	0/5
q8_0	163840	4.0×	5/5	—	—	—	—
q8_0	204800	5.0×	3/5	—	—	—	—
q8_0	245760	6.0×	4/5	4/5	4/5	—	—

Verdict: the mscale lever has no beneficial regime for DCA retrieval — at 6× native (where plain-DCA is stressed) yarn 1.0/1.1/1.25 are flat at 4/5, and 1.18 is the principled mscale for 6× (0.1·ln 6 + 1), so there's no missed sweet spot; pushing harder (1.5/2.0) only over-sharpens already-concentrated attention and destroys retrieval. Default 1.0 is correct. Ships as a documented power-user knob, not a win over plain-DCA. (Rejected the alternative full-YaRN ext_factor recipe — it would double-extend positions vs DCA's chunk remap.)

Cross-check — gemma 128e-Q6_K quick re-validation ladder (2026-06-25, current post-#554/#555 binary). n=15 RULER-VT (0–100), f16/f16 KV both, DCA chunk 8192, Gemma-4-A4B 128e-Q6_K weights (VERIFIED: load_model loads 128e-Q6_K.gguf, and the load log reports n_ctx_train = 262144 ⇒ native 256k — the ladder banner's "128e-F16" string is stale; early-prefill ~8.9k tok/s is Q6_K-class, ~12× the post-#555 F16 ~3.1k). Confirms plain-DCA alone holds the VT band deep into extension. prefill tok/s = cumulative avg at full ctx (early clips ~8.9k, degrades as the O(n) cache fills); wall approximate (two GPU lanes interleaved).

ctx	×native (256k)	DCA	VT (n=15)	prefill tok/s (avg@full)	gen tok/s	≈wall
32768	0.125×	off	100.0	(fast)	124.5	~1 min
262144	1×	off	97.33	—	117.3	~16 min
262144	1×	on	98.67	—	118.0	~17 min
524288	2×	on	100.0	2,475 (211 s / 512k)	80.0	~54 min
786432	3×	on	92.0	1,859 (422 s / 786k)	49.0	~108 min

DCA-on ≥ off at the comparable 256k (1×) cell (98.67 ≥ 97.33), holds 100.0 at 512k (2×), lands 92.0 at 768k (3×) — consistent with §1's careful n=50 768k = 0.984 (the −VT is n=15 sample noise, not a cliff). 1M (4×) was not re-run today. NB: §1's "128k native cap / 8×" wording is an error — n_ctx_train is 256k, so 1M = 4× native (matches §1's own "3× at 768k").

1b. MRCR (OpenAI Multi-Round Co-reference Retrieval) — sequence_matcher_ratio

Perf columns are authoritative summary.json scores values. n = samples in the bin. wall/req = median seconds per request (wall_s_median).

ctx (bin)	DCA	score	n	prefill tok/s	gen tok/s	VRAM peak	wall/req (med)	status
256k	off	0.1138	50	535.3	92.4	55,075 MiB	336 s	n=50 re-run (06-14)
256k	on	0.1320	50	555.5	89.5	55,075 MiB	323 s	n=50 re-run (06-14)
512k	on	0.1027	50	527.5	70.2	61,333 MiB	833 s	n=50 re-run (06-14)
768k_synth	on	0.1259	8	477.5	50.7	69,063 MiB	1971 s	done (small-n, not re-run)
1M	on	0.0273	6	489.1	55.2	73,111 MiB	1653 s	done (small-n, not re-run)
256k (q8-KV)	on	0.0999	?	—	—	—	—	quant-KV × DCA (#434); C-series binary, mrcr_256k_dcaon_q8_c1, landed 06-24

The 256k/512k cells were re-run at n=50 (driver bs2:/srv/ml/omk_mrcr_dca/mrcr_n50_2gpu.sh, cells mrcr_256k_dcaon mrcr_256k_dcaoff mrcr_512k_dcaon; last cell — 512k — finished 06-14 15:21 CEST); 768k_synth/1M remain at their original n=8/6. score/n/per_needle are the authoritative n=50 summary.json .score; the throughput columns (prefill/gen tok/s, VRAM, wall/req) are per-ctx-bin properties from the ladder server.logs and are independent of sample count. The earlier small-n numbers (256k-off 0.0437 / 256k-on 0.0497 / 512k-on 0.0483 at n=16/8) are superseded — MRCR has brutal per-prompt variance, so n=16 badly under-estimated the bin mean.

Reading (n=50). At 256k (the only directly-comparable bin) DCA-on 0.1320 ≥ DCA-off 0.1138 — DCA costs nothing and is marginally higher, and it leads at every needle count: on per_needle_mean = {2: 0.104, 4: 0.198, 8: 0.091} vs off {2: 0.099, 4: 0.154, 8: 0.086}. This is the load-bearing MRCR result and it strengthens at n=50 vs the old n=16 (where the +0.006 gap was within noise). 512k DCA-on is 0.1027 ({2: 0.156, 4: 0.058, 8: 0.094}) — lower than 256k-on, but a different prompt bin, so this is not a "DCA degrades with ctx" signal (see the bin-comparability warning above). The higher 768k_synth score (0.1259, still small-n=8) is NOT "DCA gets better at longer ctx" — it's a synthetic bin whose per-needle mix is dominated by the easy 2-needle sub-case (per_needle_mean = {2: 0.290, 4: 0.033, 8: 0.019}). MRCR absolute scores are intrinsically low (brutal fuzzy-retrieval bench); only same-bin comparisons (and the per_needle breakdown) are meaningful.

Quant-KV × DCA @256k (composition, #434). q8_0 KV-cache + DCA-on at 256k scores 0.0999 (mrcr_256k_dcaon_q8_c1, canonical scores.metric = sequence_matcher_ratio; n not emitted) — i.e. below even f16-KV DCA-off (0.1138). So on the hard MRCR task at 256k, f16 KV fidelity matters more than DCA does: the q8 KV quality loss (≈ −0.032 vs f16 DCA-on 0.1320) outweighs the DCA gain (+0.018 vs f16 off). Caveat — this is cross-binary: q8 ran on the C-series binary while the f16 ladder above is the original bs2 ladder binary, and the same-binary f16 C-series cells (mrcr_256k_dca{on,off}_c1) produced no scored summary.json, so the clean same-binary q8-vs-f16 delta is still missing. Consistent with §5b's rule (scalar KV → noDCA/f16-fidelity is the quality path): for long-ctx retrieval, prefer f16/high-bit KV over a DCA-on low-bit cache when quality is the priority. RULER-VT (easy retrieval) is far more quant-tolerant (§1a).

1c. Reference (superseded) — v7-coder-it 98e Q4_K_M, MRCR DCA-on (bs2)

These upper-tier MRCR cells were run earlier on a different model (98e v7-coder Q4_K_M, not 128e-F16), so they are not part of the single-model ladder; the 128e-F16 re-runs above replace them. Same-bin head-to-head (768k_synth): 128e-F16 0.1259 beats v7-coder 0.0376 by 3.3×, as expected for a clean F16 base vs a 98e-pruned Q4_K_M. Authoritative scores perf:

ctx (bin)	score	n	prefill tok/s	gen tok/s	VRAM peak	wall/req (med)	per_needle_mean
512k	0.0413	8	3081.9	57.0	26,827 MiB	208 s	{2: 0.053, 4: 0.024, 8: 0.049}
768k_synth	0.0376	8	1815.9	33.9	35,227 MiB	739 s	{2: 0.049, 4: 0.044, 8: 0.011}
1M	0.0033	6	2052.9	38.2	39,629 MiB	606 s	{2: 0.0023, 4: 0.0, 8: 0.0076}

(v7-coder Q4_K_M is ~3.5× smaller on disk, so its prefill tps run far higher and VRAM far lower than the F16 ladder model — these rows are NOT throughput-comparable with §1b.)

---

2. Throughput — prefill / gen tps (32k ctx, 4-chunk)

2a. 3-arm gen-tps — gemma-4-26B-A4B-it Q4_K_M, solidpc RTX 3090 (#620)

22,722-token prompt, 256 gen tokens, ctx 32768, chunk 8192.

arm	prefill tok/s	gen tok/s	gen ratio
vanilla (stock llamafile)	—	—	stock binary segfaulted at load (N/A)
DCA build, --dca off	2587.0	71.31	1.000 (baseline)
DCA build, --dca on	2558.6	65.96	0.925

→ Turning DCA on costs ~7.5% gen throughput at 32k (the runtime-flag cost); prefill is at parity (2559 vs 2587). The DCA build with --dca off is the right non-DCA baseline (the stock vanilla binary can't load this model build).

2b. Per-arch coherence-gate timings — solidpc RTX 3090 (32k ctx, 200 gen tokens)

DCA-on (fused) unless noted. Confirms the head-dim port works across arches:

model	arch	head_dim	prefill tok/s	gen tok/s
Qwen3.6-35B-A3B UD-IQ3_XXS	qwen35moe	256	1787.2	60.61
Qwen3-4B Q4_K_M	qwen3	128	4355.9	75.45
v7-coder-it 98e Q4_K_M — native (DCA off)	gemma4	512	2779.6	70.16
v7-coder-it 98e Q4_K_M — fused (DCA on)	gemma4	512	2773.2	66.01

→ On the same gemma model, DCA-on vs off at 32k: prefill 2773 vs 2780 (parity), gen 66.0 vs 70.2 (~0.94×) — consistent with §2a.

2c. Optimization history (prefill at 256k, documented in 0078-dca.patch)

After O9 (analytical INTRA mask — kernel-derived causal band) the multi-chunk DCA prefill at 256k is at parity with native (~1.0×), down from the pre-optimization 1.59× penalty. O6 (analytical SUCC/INTER band bounds) killed 2 of 3 dense masks; O3 gave the multi-chunk 3× KV floor.

---

3. Memory model — iSWA KV scaling (128e-F16, bs2 Blackwell)

The reason a 262144-trained 26B-A4B fits 1M context on one GPU: only the 5 global-attention layers scale KV with full context; the 25 SWA layers are pinned to a 1536-cell sliding window.

Component	256k	512k	768k	1M (proj)
Weights (F16, fixed)	48,150 MiB	48,150	48,150	48,150
KV — 5 global layers (scales)	5,120 MiB	10,560	~17,280	~20,800
KV — 25 SWA layers (fixed 1536-win)	300 MiB	300	300	300
Compute buffer	~820 MiB	~1,636	~2,500	~3,000
Total (observed / projected)	~54.5 GB	61.3 GB (obs)	69.1 GB (obs)	~72 GB (proj)

All tiers fit the 96 GB Blackwell with headroom (1M ≈ 72 GB < 96 GB). On a 24 GB 3090, only ≤256k of the quantized model fits; F16 at any long ctx is bs2-only.

---

4. Correctness gates

4a. Qwen multi-chunk coherence — dca_gate623_mc_coherence.sh (solidpc, DSO df4c46bc, #623)

Multi-chunk (ctx 32768 > chunk 8192 → 4 chunks). Bar: fused (DCA on) coherent and ideally byte-identical to native (DCA off = full-attention ground truth).

model	head_dim	native vs fused	verdict
Qwen3.6-35B-A3B (qwen35moe)	256	byte-identical (470 ch, uniq 0.38, 7/7 record-lines)	PASS
Qwen3-4B (qwen3)	128	byte-identical (463 ch, uniq 0.366, 7/7 record-lines)	PASS
OmniMerge v4 (qwen35, MTP)	256	near-identical (native 557 ch / fused 556 ch, uniq 0.713≈0.714, max_run 1, 9/9 record-lines)	PASS

OmniMerge v4 is a Qwen3-Next hybrid (65 blocks: ~48 SSM/GatedDeltaNet linear-attn + 16 full-attn hd256 + 1 MTP/nextn head at blk.64). It would not load until the #622 loader fix: (a) qwen35 hparams now reads nextn_predict_layers and excludes the MTP block from the recurrent/transformer loops, (b) the tensor-load path loads blk.64.nextn.* (not ssm/attn) for the last nextn_predict_layers blocks, (c) llm_build_qwen35 skips the MTP block in forward, and (d) llama-arch.cpp re-classified the six NEXTN_* tensors LLM_TENSOR_LAYER_OUTPUT → LLM_TENSOR_LAYER_REPEATING (matching upstream — they are stored per-block, so the loader must not fault on the block index). DCA only chunks the 16 hd256 full-attn layers on this hybrid, so its marginal long-context value here is narrower than on a pure-attention model — but the fused multi-chunk path is verified coherent. See .wolf/buglog.json bug-458.

The prior DSO (build #3, 65b324b1) emitted solid ! (NaN logits) on both — the iter Q_in_reg fix was the load-bearing final site. See .wolf/buglog.json bug-454/455/456 and .wolf/cerebrum.md.

4b. Gemma no-regression — dca_gate_gemma_mc_regress.sh (solidpc, DSO df4c46bc)

The qwen fix is provably a no-op on Gemma: the ampere config table (ggml_cuda_fattn_mma_get_config_ampere) has nstages_target=1, Q_in_reg=false for every DKQ=512 row, so all four fixes compile to the identical Gemma kernel and dispatch case 512 == the old hardcoded <512,512>. Empirical confirmation (v7-coder Q4_K_M, hd512):

mode	native vs fused	note
single-chunk (n_ctx ≤ chunk)	byte_identical = True	both arms route to plain FA; binary is deterministic, DCA single-chunk is bit-exact
multi-chunk (n_ctx > chunk)	native ≠ fused (uniq 0.297, coherent, 7 records, no loop)	intrinsic Q4_K_M band-combine numerics tipping greedy argmax — pre-existing, not the qwen fix

The authoritative Gemma DCA-quality measure is the bs2 128e-F16 ladder (§1), where fresh-binary 256k dcaon 0.964 = dcaoff 0.964 (exact parity, both n=50) confirms multi-chunk DCA preserves quality on the real model/metric.

4c. NIAH single-needle retrieval — dca_niah_ladder_618.sh (bs2, 128e-F16, #618)

Needle (code 73914) retrieved from the greedy completion, DCA-on, single prefill per cell (native==fused established byte-exact). Chunk-count ladder:

chunks	ctx	result
2	16k	PASS (retrieved)
4	32k	PASS
8	64k	PASS
16	128k	PASS
32	256k	PASS

(Per the #618 ladder + the 0078-dca.patch measured-perf note; raw responses at bs2:/srv/ml/dca/dca_niah_ladder_618/resp_c*.json.)

4d. Decode chunk-boundary correctness (#635) — dca635_fix_gate.sh + NIAH INTER-band (solidpc, Qwen3-8B-Q8_0)

The #635 decode fix (INTER-Q = 2c; full RCA in docs/DCA_HANDOFF.md §6) is gated two ways, greedy (temp=0, top_k=1, seed=1, cache_prompt=false).

(i) Self-calibrating count-up. DCA-on must break NO EARLIER than the model's intrinsic limit. Qwen3-8B-Q8_0 degenerates by itself (an intrinsic weakness, NOT a DCA bug), so "never breaks" is the wrong criterion — the gate measures the DCA-off intrinsic on the same binary, then PASSes DCA-on iff each chunk reaches it. (n_predict=1600; the old 900 truncated at 206 and manufactured a fake break@206.) The intrinsic value is decode-path-dependent (326 on the earlier single-block-per-tile hack, 346 on the clean stream-k binary) — exactly why the gate self-calibrates rather than hard-coding a number.

CONFIRMED on the clean deployable binary (host sha 7c696dde, stream-k decode + 2c fix; kernel hack + host probes reverted), 2026-06-14:

arm	chunk	count reached	verdict
off (intrinsic)	—	346	reference
on	256	346	PASS
on	512	346	PASS
on	1024	346	PASS

Pre-fix, DCA-on collapsed ~2× early (c256 ≈ 151, c512 ≈ 299) into …232323… garbage.

(ii) NIAH through the INTER band. Needle (code 73914) in chunk-0 + ~3000 filler tokens, so the retrieval query is at cq ≥ 2 and the needle sits in the INTER band — exactly where the 2c fix relocates far keys (rel [c,2c]). On the clean binary, DCA-on(2c) c256 retrieves 73914 as well as DCA-off (both FOUND) → the fix preserves far-context retrieval (the one production risk of pushing far keys past the window edge). Probe-free runner: dca635_niah_clean.sh.

Scripts: solidpc:backup_models/scripts/dca_chunkprobe/{dca635_fix_gate.sh,dca635_niah_clean.sh,dca635_niah/}. The decisive diagnostic — an independent absolute-position full-attention reference — is dca635_true_ref.py.

Status: CONFIRMED (2026-06-14 21:27 CEST). Both gates above pass on the clean deployable artifact — ggml-cuda.so rebuilt with the falsified single-block-per-tile decode hack reverted (stream-k restored), host binary relinked with the capture probes stripped (sha 9f51da03 → 7c696dde), 2c fix preserved. 0078-dca.patch regenerated from this tree (37 files, 224K): 2c fix present, zero dca635 / DCA635_FORCE_STREAMK / dca635_dump / dca635_capture_cb tokens.

✅ #617B debug instrumentation STRIPPED (2026-06-14). The four inert artifacts that previously rode along on the fragmented (context-shift) path — a DCA617B_FORCE_FRAG env backdoor, two one-shot [DCA617B] BUILD/FILL stderr prints, and the DCA617_SUCC_NULL/INTER_NULL env blocks — were removed from dca.cpp (build_attn_inp_dca) and llama-kv-cache.cpp (set_input_dca). All were nullptr/no-op on the fast path and only reachable via explicit env vars, so removal changes no production behavior. Host make rebuilt the binary (sha 665bd58b, was 7c696dde); the #634 ctx-shift gate was re-run on the stripped binary and still PASSES (off + on2048 both PASS-coherent, 1 shift, monotone 1201→2400, byte-identical — §4e below). 0078-dca.patch regenerated: 37 files, 220K, zero DCA617* tokens, the #635 2c fix (pqn[off] = 2*c) intact, the #617B is_fragmented/mask_succ machinery intact. The FORCE_FRAG test hook (dense path on a contiguous cache, no n_ctx overflow needed) is preserved as a documented one-line re-add in DCA_HANDOFF.md §10 optional tasks, not in production source.

4e. Context-shift coherence (#617B / #634) — dca_ctxshift_qwen3.sh (solidpc, Qwen3-4B-Q4_K_M)

#617B restores correctness after a classic context shift (generation overflows n_ctx → seq_rm

• seq_add relabels pos[] in place, so cache index ≠ position and the analytical INDEX bands select the wrong cells — the model collapses to a token-loop). The fix is a sticky is_fragmented latch gating a dense position-based SUCC/INTER/INTRA fill. Reachable only on full-attention DCA archs (qwen2/3/3.5) past n_ctx — gemma4 (the 1M iSWA target) disables KV-shift entirely, so it never fragments.

That fragmented fill lives in the same set_input_dca as the #635 INTER-Q = 2c change (the 2c fill is unconditional, so it now governs the fragmented path too), so the gate was re-run against the clean 2c binary (host sha 7c696dde), 2026-06-14. A real shift is forced: ctx=8192, chunk=2048 (4 DCA chunks), ~4.9k-token counting prompt + ignore_eos gen=6000 → n_past overflows 8192 → ≥1 classic shift.

arm	dca	shifts	gen ints	first (exp 1201)	longest run	maxrep	verdict
off	native (ref)	1	1200	1201	1200	1	PASS-coherent
on2048	DCA modular shift	1	1200	1201	1200	1	PASS-coherent

Both count perfectly monotone 1201→2400 across the shift (longest_run == gen_ints, no repeats), and DCA-modular matches native byte-for-byte. The 2c fix does not regress the context-shift path — #634 closed.

---

5. KV-quantization × DCA — full-vocab PPL + KL-divergence (#446, solidpc RTX 3090)

real_frac (logit-equivalence) is argmax-only — it cannot distinguish genuine distribution- faithfulness from argmax-coincident degradation. This section adds the whole-distribution instrument: teacher-forced full-vocab KL-divergence over a fixed corpus. Per regime ({DCA-off, DCA-on}) we write an f16 logits reference (--kl-divergence-base), then score every KV-quant candidate against it (--kl-divergence). The corpus is teacher-forced, so every config decodes the identical token sequence and KL(candidate ‖ f16) is exactly aligned per token. This is the decisive answer to #446.

Methodology. Model Qwen3-8B-Q8_0 (head_dim 128 → innerq-scale-128), corpus wikitext-2 test slice (6402 tokens), -c 2048, 3 chunks, --flash-attn on -ngl 99, teacher-forced. DCA recipe --dca on --dca-chunk-size 1024. Runner .opencoti/ppl-kld-matrix.sh on a GPU-enabled standalone perplexity binary (.opencoti/ppl-link.mk + ppl_main_gpu.cpp, a main() wrapper that calls llamafile_has_gpu() so the CUDA DSO loads and -ngl offloads — the stock tools/perplexity/main.cpp never triggers that load, so vanilla perplexity is CPU-only). DCA was exposed to perplexity via a temporary arg.cpp edit adding LLAMA_EXAMPLE_PERPLEXITY to --dca/--dca-chunk-size (reverted after the eval; vendor tree restored to patch-stack state — the edit is not in any patch). KLD = mean nats; top% = same-top-token agreement vs the f16 reference.

5a. TurboQuant symmetric — the #446 decider

f16 reference (Final-estimate PPL): noDCA 9.451 · DCA(chunk 1024) 9.441.

tier (-ctk=-ctv)	PPL(Q) noDCA	KLD noDCA	top% noDCA	PPL(Q) DCA	KLD DCA	top% DCA
turbo8 (8-bit)	9.438	0.00173	98.2	9.463	0.00217	98.1
turbo4 (4-bit)	10.39	0.142	85.1	9.913	0.115	86.3
turbo3 (3-bit)	12.71	0.404	75.5	10.11	0.186	82.7
turbo2 (2-bit)	1584	5.596	16.1	14.71	0.647	67.5

Verdict — real_frac=0 for turbo-DCA is GENUINE, not argmax-coincident. Under DCA the KLD is monotone in tier bits (turbo8 ≈ 0.002 near-lossless → turbo2 0.65 lossy-but-functional) — the signature of a faithful dca_lift_to_f16 dequant (#444 → #385 turbo→f16 CPY kernel). The whole-distribution metric corroborates the #446 argmax gate: the turbo-DCA distribution genuinely tracks f16-DCA; where the argmax flips, the flips are near-ties, not degradation.

Bonus — the DCA f16-lift is the better turbo path. For turbo2/3/4 the DCA KLD is lower than the noDCA in-register FA-VEC read (turbo8 a tie). The extreme case is decisive: turbo2 in the noDCA in-register path is degenerate — KLD 5.60, PPL ×168 (1584), top 16% (~random) — while turbo2-DCA via the faithful lift is healthy (KLD 0.647, top 67.5%). The argmax-tolerant L4 gate (#394) masked this; full-vocab KLD exposes it. Recommendation: turbo2 only via the dequant-on-lift path (DCA, or the #444 quantized-DCA f16 slot) — never the in-register noDCA read. (.wolf/buglog.json turbo2-inreg.)

5a-TCQ. Trellis-coded tiers (#447) — noDCA in-register, current DSO

TCQ (turbo3_tcq/turbo2_tcq) is always-fuse (no f16 dequant-on-lift yet), so the in-register symmetric read IS the path under test. Same #446 instrument (Qwen3-8B-Q8, wiki, ctx2048/chunk1024), current DSO, TURBO_TCQ_ALPHA_V=1.0. The turbo8/4/3/2 controls re-ran here bit-identical to §5a → the TCQ Stage-1–4 shared-file edits did not move the non-TCQ tiers (second instrument confirming "not a regression", beyond the real_frac controls).

tier (bpv)	PPL(Q)	KLD	sametop%	vs scalar sibling
turbo3_tcq (3.25)	10.306	0.200	83.0	2× better than turbo3 (0.404); turbo4-class
turbo2_tcq (2.25)	1113	5.215	18.6	≈ turbo2 (5.60) — degenerate in-register

turbo3_tcq is a genuine win. 0.25 extra bits over scalar turbo3 buys a 2× KLD reduction (0.404→0.200), landing between turbo4 (0.142) and turbo3 — the trellis delivers, corroborating the real_frac=0 needle gate with the whole-distribution instrument. turbo2_tcq is NOT viable in-register (KLD 5.2, sametop ≈ random): the 2-bit-class collapse is below the in-register floor even on Qwen3-8B, exactly like scalar turbo2. The floor is sharper on small already-Q4 models — on Qwen2.5-1.5B-Q4 even turbo3_tcq collapses (real_frac 1.0), tracking turbo3 (also 1.0 there). Whether this in-register low-bit collapse is a kernel defect or model fragility is settled decisively by the target-class run below: it is fragility. The f16 inverse-FWHT/trellis dequant-on-lift (mirrors #385/#444) remains a worthwhile optional enhancement for small-model support, but it is not a prerequisite for shipping TCQ on the actual targets.

5a-TCQ-27B. Target-class verdict (#447/#504) — Qwen3.6-27B, the model-vs-defect decider

The §5a/§5a-TCQ numbers above are an 8B instrument; the question they leave open — is the in-register low-bit collapse our kernel or the model? — can only be answered on the model class the anbeeld.com KV-quant benchmark actually used: Qwen 3.6-27B. Same harness (.opencoti/tcq-ppl-27b.sh), 320 KB wikitext, ctx 2048, InnerQ OFF (proven inert), TURBO_TCQ_ALPHA_V=1.0, f16 KLD-base. Model: Qwen3.6-27B-Omnimerge-v4-Q4_K_M (f16 base PPL 6.307).

tier (bpv)	PPL(Q)	KLD	sametop%	8B KLD (320 KB)	anbeeld ref KLD	ours ÷ ref
turbo8 (8)	6.317	0.0168	97.7	0.0025	q8 0.0023	7.3×
q4_0 (4, scalar anchor)	6.334	0.0250	96.8	0.028	0.0047	5.3×
turbo4 (4)	6.349	0.0342	95.7	—	0.0048	7.1×
turbo3_tcq (3.25)	6.389	0.0366	95.6	0.187	0.0080	4.6×
turbo3 (3)	6.407	0.0470	94.7	0.350	0.0112	4.2×
turbo2_tcq (2.25)	6.456	0.0709	92.7	~5.19	0.0231	3.1×
turbo2 (2)	6.697	0.0960	90.8	~6.4	0.0362	2.65×

Verdict — it is the model, not the implementation. turbo2 went from PPL ~3000 / KLD 6.4 (collapse) on the 8B to PPL 6.70 / KLD 0.096 (functional) on the 27B — same code, same harness, only the model changed. The entire ladder is monotone and functional (every cell within 0.4 PPL of the 6.307 base), and both TCQ tiers beat their plain-turbo siblings (turbo3_tcq 0.0366 < turbo3 0.047; turbo2_tcq 0.0709 < turbo2 0.096) — turbo2_tcq's 2.25 bpv lands exactly between 3-bit and 2-bit, as designed. This reproduces the reference's qualitative behavior on the reference's own model class.

On the anbeeld gap. Our absolute KLDs run ~3–7× the reference because the eval setup differs (ctx 2048 vs 64k, 320 KB vs full wikitext, f16 base vs bf16) — and this tax is model-independent: the same-code q4_0 anchor is 0.025 on the 27B and 0.028 on the 8B, both ~5× the reference's 0.0047. Crucially, turbo2's ratio-to-reference (2.65×) is the lowest in the ladder — far from anomalous, the most aggressive tier is the best-behaved relative to the reference. There is no defect signal at any bit-width.

The fragility gradient (why the 8B looked broken). In-register symmetric KV tolerance scales with model KV-redundancy: Qwen2.5-1.5B-Q4 collapses at ≤4-bit (turbo4/3/2/tcq all real_frac 1.0; turbo8 fine); Qwen3-8B-Q8 is fine at 3-bit-degraded but collapses at 2-bit (PPL 3000); Qwen3.6-27B absorbs even 2-bit cleanly. The in-register vec_dot approximation's loss (cf bug-583: turbo2 in-register KLD 5.60 vs faithful f16-lift 0.647 on identical data) is progressively absorbed by larger models. Conclusion for #504: the symmetric in-register noDCA path ships as-is on 27B-class / Gemma-4 targets — turbo2/turbo2_tcq included. The f16 dequant-on-lift is reclassified from "shippability blocker" to "optional small-model support." RULER quality-in-use on Gemma-4-A4B tracked separately.

DCA-on companion (same 27B, --dca on --dca-chunk-size 1024, per-regime f16+DCA base 6.304). Under DCA the turbo K/V route through the FAITHFUL f16 dequant-on-lift (#444/#446), not the in-register read — isolating the lift path's quality at target scale. DCA is inert at ctx-2048 here (f16+DCA base 6.304 ≈ noDCA 6.307), so the cells are directly comparable to the noDCA ladder above:

tier	KLD noDCA (in-reg)	KLD DCA (f16-lift)	Δ
q4_0	0.0250	0.0285	+0.0035 (lift round-trip costs a hair on faithfully-read scalar KV)
turbo8	0.0168	0.0175	+0.0007 (wash)
turbo4	0.0342	0.0316	−0.0026
turbo3	0.0470	0.0436	−0.0034
turbo2	0.0960	0.0896	−0.0064
turbo3_tcq	0.0366	composes (#533) ⁵	f16-lift now exists → DCA composes (was rc=134 in this 2026-06-21 run)
turbo2_tcq	0.0709	composes (#533) ⁵	same

⁵ TCQ+DCA composition was RESOLVED by #533, verified 2026-06-30 (#505). The turbo{3,2}_tcq→f16 dequant-on-lift kernels (turbo-cpy.cu, opencoti-hook #533) + cpy.cu dispatch + CPY supports_op acceptance (ggml-cuda.cu:5442) were added when #533 fixed the TCQ prefill via an MMA-materialize, and dca.cpp's dca_lift_to_f16 routes TCQ through them. Boot-compose check (solidpc 3090, Qwen3.6-27B-Omnimerge-v4-Q4_K_M, -ctk/-ctv turbo3_tcq --dca on --dca-chunk-size 1024): boots clean, no rc=134, decodes coherently. The KLD value under DCA was not re-measured (needs the perplexity --dca arg.cpp edit; optional now that compose is proven) — cells show the noDCA KLD as the fidelity proxy.

Reading. (1) For turbo low-bit, the f16-lift ≥ the in-register read at every bit-width, and the margin widens as bits drop (turbo4→turbo2: −0.003 → −0.006) — confirming the in-register vec_dot approximation is the (small, on 27B) lossy element, exactly bug-583's thesis. The margin is small here only because the 27B already absorbs the in-register loss; on the 8B the same lift rescued turbo2 6.4 → 0.647. (2) For scalars, noDCA is the quality path (DCA adds a lift round-trip) — the inverse of turbo, matching §5b. (3) TCQ+DCA composes as of #533 (was rc=134 SIGABRT in this 2026-06-21 run): TCQ was then always-fuse with no turbo_tcq→f16 kernel; #533 added exactly that kernel (for the prefill MMA-materialize), which also gives DCA's dca_lift_to_f16 a path to lift TCQ K/V — so the same #533 work delivered both the prefill fix and TCQ+DCA composition. The remaining #505 item (small-model ≤8B decode through the lift) is optional/off-target per #504. (Harness .opencoti/tcq-ppl-27b-dca.sh; matrix 27b-dca-matrix.tsv. Ran via a temporary arg.cpp LLAMA_EXAMPLE_PERPLEXITY edit, since reverted.)

5a-TCQ-CB. Codebook retrain & KLD-aligned objective campaign (#523–525, solidpc RTX 3090)

Goal: lock the trellis codebook before the deferred full PPL/KLD + tps + RULER campaign. Three ordered steps on the §5a-TCQ-27B instrument (same binary/base/corpus: perplexity host build, Qwen3.6-27B-Omnimerge-v4-Q4_K_M, 320 KB wikitext, ctx 2048, InnerQ OFF, TURBO_TCQ_ALPHA_V=1.0, f16 KLD-base base_f16_27b.dat). Codebooks load rebuild-free via the A′ runtime path (TURBO_TCQ_CB 512-entry / TURBO_TCQ_W_CB 1024-entry). Lower KLD = better.

Bars to beat (deployed = shipped buun codebook): turbo3_tcq deployed-512 0.036648 · v1-synthetic-1024 0.036972 · v1-synthetic-512 0.038815. Free-widen win condition: turbo3_tcq_w (1024-state, real-27B-trained) KLD < 0.036648 at identical 3.25 bpv.

Gate log (.opencoti/tcq-gate1.sh → .opencoti/ppl-kld/27b_fe_*.log, 27b-fe.tsv):

date	gate	cell	codebook	bpv	KLD	sametop%	verdict
06-21	1	tcq_ctrl_v2	deployed-512 (anchor)	3.25	0.036648	95.64	✓ reproduces bar exactly — GPU/harness/binary sound
06-21	1	tcq_real512_v2	real27b-512 (MSE)	3.25	0.037559	95.40	> deployed — fresh MSE retrain is worse; MSE exhausted (tail-share deficit)
06-21	1	tcq_real1024_v2	real27b-1024 (MSE widen)	3.25	0.037269	95.75	> deployed, < real512 — MSE+free-widen still short of 0.036648; Step-2 needed
06-21	1	scalar_q5_0_rt	q5_0 (frontier)	5.5	interrupted	—	cut at chunk 15 (~0.036 running) by session /exit 12:41 — NOT a fault; rerun as scalar_q5_0_rt2 in gate-2

Reading so far. The anchor's exact reproduction (0.036648) certifies the post-reboot binary/DSO. The MSE-512 retrain at 0.037559 > 0.036648 is the decisive negative: a from-scratch MSE codebook on the real-27B dump cannot beat the deployed one, because the deployed codebook carries more tail point-share than MSE optimization yields (post-FWHT marginals are ≈N(0,1), so diagonal/Fisher weighting is near-dead and MSE over-resolves the centre). KLD rewards tail resolution → MSE is the wrong objective. The MSE-1024 free-widen confirms it: 0.037269 — better than MSE-512 (0.037559, so doubling trellis states 512→1024 does help) but still > deployed 0.036648, and even > v1-synthetic-1024 (0.036972). Step 1 is settled negative: no MSE codebook (512 or widened-1024) beats the shipped tail-heavy one — the win must come from the objective, not the state count. Empirical green-light for Step 2 (KLD-aligned tail-weighted distortion w(v)=1+λv², λ-sweep; gate-2 picks the λ whose 1024 codebook beats 0.036648).

Events. 2026-06-21 11:30 — the first gate-1 attempt aborted mid-run when the RTX 3090 fell off the PCIe bus (NVRM Xid 79 → Xid 154 "Node Reboot Required"); every cell, incl. the deployed anchor, hit unspecified launch failure — a hardware fault, not a TCQ/codebook regression (DSO byte-identical, codebooks correctly sized, anchor used no override; bug-621). Host rebooted, GPU recovered clean (xid=0), gate-1 re-running from 12:04 (numbers above are the clean run). Step 2 λ-sweep (w(v)=1+λv², λ∈{0,0.5,1,2,4}, CPU, .opencoti/tcq-step2-lambda-sweep.py) running in parallel. Step 3 (asymmetric turbo3_tcq_w-K / turbo2_tcq-V wide-K/cheap-V, ~2.75 bpv) pending a ~72-min CUDA rebuild (.opencoti/step3-activation-edits.md).

Step-2 pivot — warm-start from the deployed codebook (#524). Since Step-1 proved a cold MSE/widen retrain can't reach 0.036648, the cold λ-sweep was retired for the direct form: seed the GLA trainer from the deployed-512 codebook, then refine on real-27B with a tail-weight λ that holds the deployed tail-heaviness (a λ=0 refine would erode it — the MSE M-step pulls levels back to the centre). 512 seeds from deployed directly; the 1024 widen seeds via coset-aware group-halving (1024 state (g<<3)|pos ← deployed[((g>>1)<<3)|pos], duplicating each deployed level into the correct L=10 trellis pair). restarts=1 (stay in the deployed basin). Seed sanity: the loaded deployed seed measures tail>2.0=9.8% / >2.5=3.7%, matching the shipped codebook exactly → domain/coset map verified. Trainer gained an init_cb= warm-start hook; driver .opencoti/tcq-warmstart.py. This strictly dominates the cold sweep for "beat deployed" — it starts at 0.036648 and can only hold or improve.

Step-2 v2 — λ=1 overshoots; small-λ search (#524). The first warm codebook trained, ws1024 λ=1.0, converged cleanly (held-in weighted-distortion fell monotonically 0.144→0.051 over 20 iters — not a numerical divergence) to a codebook with tail>2.0=89.2% / >2.5=87.0% — ~9× the deployed's 9.8%/3.7%. So the weighted-MSE objective's true optimum at λ=1 is a tail-degenerate codebook that under-resolves the centre where the KV bulk lives → it should hurt KLD badly (the high-λ calibration point). The lesson: the λ→tail-share curve is brutally steep (λ=0 MSE → ~8.5%, λ=1 → 89%), so the deployed's mild ~10% tail sits at small λ. The grid was corrected to λ∈{0.05, 0.1, 0.2} (1024-prioritised, the prize) + one λ=0 (MSE-refine) endpoint + one 512 check; the λ=2 cells were dropped (would over-shoot worse). ws1024_w1 (λ=1) is kept and measured as the high-λ endpoint. (Side-effects of the redirect: scalar_q5_0_rt2 was cut mid-run when gate-2 v1 was stopped, re-queued as rt3 — LAST.)

Gate-2 log (.opencoti/tcq-gate2.sh, file-gated on each warm codebook as it's trained):

date	cell	seed	λ	bpv	tail>2.0	KLD	sametop%	verdict
06-21	ws1024_w1	deployed→1024	1.0	3.25	89.2%	0.037051	95.64	> deployed — λ=1 over-corrects
06-21	ws1024_w005	deployed→1024	0.05	3.25	88.0%	0.034034	95.71	< deployed (−7.1%) ✓
06-21	ws1024_w01	deployed→1024	0.1	3.25	88.5%	0.036303	95.55	< deployed (−0.9%, marginal) ✓
06-21	ws512_w01	deployed-512	0.1	3.25	88.1%	0.037681	95.50	> deployed — 512 can't beat even warm; widen needed
06-21	ws1024_w02	deployed→1024	0.2	3.25	89.0%	0.037259	95.75	> deployed — λ=0.2 over-corrects
06-21	ws1024_w0	deployed→1024	0.0	3.25	87.8%	0.033130	95.70	★ WINNER — < deployed (−9.6%); plain MSE-refine
06-21	scalar_q5_0_rt3	— (q5_0)	—	5.5	—	running	—	frontier rerun, LAST (tangential)

(λ=2 cells dropped — superseded by the small-λ finding above; the warm-start grid was re-run accordingly.)

Verdict — free-widen WINS; tail-weight REJECTED (#524 closed). The decisive result inverts the Step-2 hypothesis. ws1024_w0 — the 1024-state free-widen, warm-started from deployed-512 and MSE-refined (λ=0) on real-27B — is the new best: KLD 0.033130 vs deployed 0.036648, a 9.6% improvement at identical 3.25 bpv. This is the plan's free-widen win condition (turbo3_tcq_w 1024-state, real-27B-trained, KLD < 0.036648), achieved. KLD is monotone in λ from 0**: λ0 (0.033130) < λ0.05 (0.034034) < λ0.1 (0.036303) < deployed < λ0.2 (0.037259) < λ1 (0.037051) — so **every tail-weight λ>0 regresses and the w(v)=1+λv² objective is rejected. Two corrections to earlier-session reasoning: (1) Step-1's "MSE retrain can't beat deployed" (cold 0.0376) was an initialization artifact — cold init_coset sticks in a tail-light basin (8.5% tail-share); warm-starting from the deployed codebook escapes into the real-data optimum (88% tail-share, KLD 0.0331). (2) The ~88%-tail warm codebooks are not degenerate (the earlier λ=1 "degenerate" read was wrong) — real-27B whitened KV is genuinely heavy-tailed, so plain MSE on it already resolves the tail; the synthetic-N(0,1) premise behind tail-weighting was the error. 512 warm-refine (ws512_w01 0.037681) cannot beat deployed even warm-started → the 1024 widen is necessary. Ship candidate (PROVISIONAL): real27b-1024-ws-w0.bin — pending the Confirmation below, which reverses this.

Confirmation (#524) — the win is SEED-LUCK, not robust; bake PAUSED. The paired re-run re-certified the instrument (deployed re-anchors 0.036648 exactly; ws1024_w0 reproduces 0.033130; ws1024_w005 0.034034 — no path/env/transcription error). But the decisive cell, a seed=2 re-train of the exact winning recipe (1024 widen, warm-start from deployed, MSE λ=0), landed at 0.037671 — WORSE than deployed, despite a lower held-in MSE (em 0.026003 vs seed=1's 0.026241). Across two independent seeds the same recipe gives KLD {0.0331, 0.0377} — a 0.0046 spread that straddles deployed 0.0366 and exceeds the apparent −9.6% margin. MSE held-in distortion does NOT predict KLD (seed=2 has the better MSE yet the worse KLD), so the trainer's objective cannot be trusted to pick the KLD-good basin, and seed=1's win is within seed-induced variance — not a transferable quality gain. Verdict reversed: no robust symmetric-3.25 bpv codebook improvement demonstrated; the bake is PAUSED. seed=1's codebook genuinely beats deployed on this corpus, but shipping it as "−9.6% better" would overclaim a corpus-specific lucky draw. Proper path (rebuild-free): train N seeds, select best by KLD on a TRAIN corpus, VALIDATE on a HELD-OUT corpus, ship only if the win transfers. The Step-3 asymmetric _w kernel work (the bit-allocation frontier) is codebook-independent and can proceed separately.

confirm cell	KLD	vs deployed	note
conf_deployed	0.036648	—	re-anchor, exact reproduction (instrument OK)
conf_ws1024_w0 (seed=1)	0.033130	−9.6%	reproduces bit-identically
conf_ws1024_w005 (seed=1)	0.034034	−7.1%	reproduces
conf_ws1024_w0_s2 (seed=2)	0.037671	+2.8%	same recipe, worse than deployed → win is seed-luck

Best-of-N + held-out validation (#524) — DEFINITIVE: the codebook win is train-corpus OVERFIT; deployed generalises best. Trained N=6 λ=0 warm-start seeds, measured each on the train corpus (wiki.test.mid) and on a genuine HELD-OUT corpus (wiki.valid.mid, own 18 GB f16 base). Held-out deployed anchor = 0.045878.

seed	train-KLD	held-out KLD (deployed 0.045878)
s1	0.033130 (best on train)	0.048497 (+5.7% — near-WORST on held-out)
s6	0.034048	0.050606 (worst)
s5	0.035175	0.045300
s2	0.037671	0.045125
s3	0.037959 (near-worst on train)	0.044793 (BEST on held-out)
s4	0.038335	0.047197
mean	0.036053 (≈deployed)	0.046920 — WORSE than deployed 0.045878

Train-KLD anti-correlates with held-out (Spearman ρ ≈ −0.6): selecting a codebook by train-KLD actively picks the worst-generalising one. seed=1's "−9.6%" is +5.7% worse than deployed on held-out; the retrained mean is worse than deployed on held-out. The apparent gain was pure overfit to the activation-dump/eval-corpus distribution. Verdict (FINAL): no robust, transferable symmetric-3.25 bpv codebook improvement exists; the deployed (buun) codebook is near-optimal and generalises better than any real-27B retrain. Ship deployed UNCHANGED — the codebook front is CLOSED. The remaining "improve TCQ" lever is the asymmetric bit-allocation frontier (Step 3, codebook-independent). Methodological lesson: a single-corpus KLD "win" on a quant codebook is overfit-prone — ALWAYS held-out-validate before shipping (here it flipped a −9.6% "win" into a +5.7% regression).

Step 3 — asymmetric TCQ frontier (#525): K-dominance CONFIRMED (modest, transfers); asym = LINEAR (not Pareto); wide-K _w-asym BUGGY + redundant. Built the turbo3_tcq_w/turbo2_tcq FA-VEC instances (+ reverse)

• both both_tcq predicates + codebook loaders (6 additive edits; dual-DSO byte-identical; 8 new vec_case instances nm-verified). Frontier on TRAIN (wiki.test.mid) and HELD-OUT (wiki.valid.mid), baked codebooks:

K / V	~bpv	train KLD	held-out KLD
turbo2_tcq / turbo2_tcq	2.25	0.070933	0.087088
turbo2_tcq / turbo3_tcq (strong-V)	2.75	0.056672	0.073838
turbo3_tcq / turbo2_tcq (strong-K, #513)	2.75	0.051501	0.066634
turbo3_tcq / turbo3_tcq (deployed)	3.25	0.036648	0.045878

(1) K-dominance holds + transfers — strong-K beats strong-V by ~10% on BOTH corpora (train 0.0515 vs 0.0567; held-out 0.0666 vs 0.0738), confirming anbeeld's "K precision dominates the tail." (2) But asym is NOT a Pareto win — the best 2.75-bpv point (strong-K) lands ~exactly on the linear 2.25↔3.25 interpolation (held-out interp 0.0665 vs measured 0.0666); strong-V sits above the line (strictly worse). So asym yields a valid intermediate operating point (always strong-K orientation), not free quality. (3) The NEW wide-K _w-asym is BROKEN — KLD 1.214 (train) / 1.296 (held-out), byte-identical with/without a real 1024-codebook override → decode-LOGIC bug (reads the wrong codebook symbol/size for the mixed-width pair), NOT seeding (bug-629). Symmetric _w with the baked placeholder works (0.037125 ≈ deployed turbo3 0.0366). (4) _w is redundant anyway — 1024-state ≈ 512-state quality (symmetric _w 0.0371 ≈ turbo3 0.0366; consistent with the best-of-N codebook-overfit finding), so a fixed _w-asym would merely reproduce turbo3/turbo2 (0.0666) — no gain over the existing #513 asym. Verdict (FINAL, across all three "improve TCQ" avenues): the deployed turbo3_tcq (symmetric) + existing turbo3_tcq/turbo2_tcq asym (#513) are ALREADY at the TCQ frontier; none of {codebook retrain, free state-widen, wide-K asym} beats it. The wide-K _w-asym instances (#525) are buggy + redundant → revert (or gate off) — only the explicit opt-in -ctk turbo3_tcq_w -ctv turbo2_tcq combo is affected; the default/shipped TCQ paths are untouched and validated.

5a-TCQ-RULER. Primary-target quality-in-use (Gemma-4-A4B, #447/#501)

The 27B PPL/KLD above is head_dim 128; the actual primary target is Gemma-4-A4B (head_dim 256/512). The PPL/KLD instrument was finally pointed at it on 2026-06-21 and found INVALID on these Gemma merges (q8_0 control Same-top 68% — see §5a-TCQ-A4B-KLD below); so the primary-target evidence rests on RULER, not PPL. RULER retrieval, forced KVTYPE_OVERRIDE=turbo3_tcq (ours-only), model gemma-4-A4B-98e-v6-coder-it-Q4_K_M, 5 samples (256k cell = 1), niah_single_1 (decisive needle metric) + vt (NS=5-noisy):

task	4k	32k	256k
niah_single_1	100%	100%	100% (measured — see below)
vt	100%	60% (noisy)	— (skipped)

niah_single_1 is clean at every context: turbo3_tcq's trellis decode holds the needle on Gemma's head_dim 256/512 with zero degradation vs cached vanilla (vanilla niah=100 at all ctx). vt@32k=60% is within the cached-vanilla noisy band (vt 60–80, NS=5 task), not a regression — niah is the correctness signal.

The 256k cell is a measured 100%, and its story is a clean throughput-vs-correctness separation (bug-594). At the harness default --http-timeout 1200, niah@256k scored 0% — but every sample hit the read-timeout mid-prefill (steady monotonic prompt-processing, server healthy, clean cancel), because turbo3_tcq is always-fuse / resident in-register (KV fit in 17 GiB) and cannot ride the f16 streaming/dequant-on-lift path — so its prefill runs ~25→8 tps (O(n²)) where f16-streaming did the whole 256k cell in 1208 s. Re-running one sample at --http-timeout 36000 (the HTTP_TIMEOUT env added to run-baseline.sh) → 100%, with the gold needle 5663623 exactly recovered as completion ' 5663623.', gen_secs=20879 (5.8 h prefill). So the 256k decode is byte-correct, just throughput-bound in-register; the perf cliff was the #505 gap — the f16 trellis dequant-on-lift now exists (#533: turbo{3,2}_tcq→f16, verified composing under DCA 2026-06-30), so the prefill MMA-materialize path is available; a fresh 256k wall-time re-measure on the post-#533 DSO is deferred (optional). Do not read a 256k turbo3_tcq 0% as a trellis bug — check samples.jsonl reason for ReadTimeout first.

Trellis code review (#447, collaborative gate). The Viterbi right-shift trellis was traced end-to-end and is algebraically correct (state ns=(prev>>k)|(out<<6), free initial state, 6-init-bit + 128×k-bit packing ⇄ decoder window at bit_pos=t*k); encode↔decode codebooks verified byte-identical (512 + 256 vals); V-alpha applied once (encode-baked, decode symbol inert); launcher/bt-buffer sound; the non-FA GET_ROWS dequant is dead code. Two latent footguns logged (not live bugs): TURBO_TCQ_CB overrides only the encode codebook, not the decode _fattn copy (asymmetric — custom-codebook runs would corrupt); and K/V split is stringly-typed on strncmp(dst->name,"cache_k_",8) (an upstream KV-tensor rename would silently flip K-alpha onto V). Both are cheap one-line hardenings, deferred.

TCQ gate verdict: correct and quality-validated on the primary target — shippable on Gemma-4-A4B/27B-class. Resident contexts (≤~32k) are correct and fast; 256k is correct, and the prefill MMA-materialize that removes the in-register O(n²) cliff now exists (#533, the #505 lift) — a 256k wall-time re-measure is the only deferred item.

WS2 update (#629, DEFAULT-ON). The in-register decode throughput above is now superseded for the DCA-on serving path by the WS2 turbo-V in-register kernel: the relevant A4B tcq @256k decode numbers are now the §5x WS2 cells (A4B turbo3_tcq 64.5 tps, turbo2_tcq 66.8 tps; +72% / +68% vs the f16-lift ref), each niah 100 + logit-lossless (real_frac 0). Read the §5x tcq rows, not the pre-WS2 in-register figures, for current turbo3_tcq/turbo2_tcq decode-in-use.

5a-TCQ-A4B-KLD. PPL/KLD instrument is INVALID on Gemma-4-A4B merges (#447, bug-631)

The deferred "quantitative KLD on the primary target" cell was finally run (2026-06-21, RTX 3090, perplexity host build, google_gemma-4-26B-A4B-it-Q4_K_M, wiki.test.mid 320 KB, ctx 2048, f16 KLD-base, InnerQ OFF, TURBO_TCQ_ALPHA_V=1.0). It does not produce usable numbers, and the q8_0 near-lossless control proves why:

KV tier	Mean KLD	Same-top p	(expected on a sane instrument)
q8_0 (near-lossless control)	0.957	68.3%	~0.02 / ~99%
q6_0	1.080	66.5%	—
turbo3_tcq (deployed)	2.264	51.2%	—

The f16 base itself scores PPL ≈ 43910 on raw wikitext (vs the 27B general-merge's 6.69 on the same harness — proving the harness is fine, the model isn't). These omnimerge coder/instruct Gemma-4 GGUFs sit in a near-uniform high-entropy regime on prose (CE ≈ 10.7 nats out of a 256k vocab's 12.45 nats ceiling): the top token is barely separated from the runner-up, so Mean-KLD is amplified (a near-flat reference yields large KL for any perturbation) and Same-top is unstable (argmax is coin-flippy even under q8_0's near-zero noise → 68%, not 99%). Both metrics here measure distribution flatness, not quant fidelity. Ruled out as causes: flash-attn/iSWA (FA on==off, both garbage), the specific GGUF (google-official and the RULER-validated v6-coder both garbage — v6 worse at 142410), and BOS (--override-kv tokenizer.ggml.add_bos_token=bool:true left f16 PPL byte-identical to 6 decimals). The tier ordering is still sane (turbo3_tcq 51.2 < q6_0 66.5 < q8_0 68.3 same-top — lower-bit = noisier), so the instrument is non-random, merely absolute-invalid.

Methodology rule (load-bearing): the q8_0 (near-lossless) control is the validity gate for any KLD/same-top harness — if q8_0 Same-top isn't ~99% (or KLD isn't ~0.02), the model/corpus is in a bad regime and every number from that run is meaningless; abort and switch instrument. This is exactly why opencoti validates Gemma KV-quant via RULER niah (high-confidence needle tokens, immune to the flat-distribution instability — niah=100% above), never PPL. The quantitative TCQ-fidelity evidence remains the 27B head_dim-128 KLD (§5a-TCQ-27B). No primary-target fidelity number can be drawn from PPL; the primary-target correctness claim stands on RULER. The 18 GB garbage f16 base was deleted (reclaimable); cell logs kept as evidence (.opencoti/ppl-kld/a4b_a4b_*.log).

5a-TCQ-FP64. Encode precision — fp64-selectable Viterbi (#447/#506, TURBO_TCQ_FP64)

The buun trellis encodes in fp32 end-to-end. Because a Viterbi trellis is a chaotic system — fp precision can flip which path wins at near-tie costs — we added a compile-time fp64 accumulator path for the encode Viterbi (k_set_rows_turbo{3,2}_tcq templated on acc_t; env TURBO_TCQ_FP64=1 dispatches the double instantiation). Decode stays fp32 (it reads stored integer states; fp64 on the FA hot path = ~1/64 throughput for ~zero quality). The point was not a guaranteed win but a measurement: is the fp32 trellis leaving quality on the table at the path-selection margin? Same 8B instrument (Qwen3-8B-Q8, wiki, ctx2048/chunk1024, TURBO_TCQ_ALPHA_V=1.0), fp64 DSO.

tier	KLD fp32	KLD fp64	Δ	PPL(Q) fp32→fp64	sametop% fp32→fp64
turbo3_tcq	0.200389	0.204047	+0.0037 (+1.8%)	10.306 → 10.212	82.99 → 82.76
turbo2_tcq	5.215325	5.258573	+0.0432	1113 → 1154	18.61 → 18.83

Two clean conclusions. (1) fp32 is byte-faithful and the templating is inert. With TURBO_TCQ_FP64 unset, the rebuilt DSO reproduces the §5a-TCQ matrix to all six decimals (turbo3_tcq 0.200389, turbo2_tcq 5.215325) and the turbo8/4/3/2 controls bit-identical — acc_t=float is the default, so every existing launch resolves to the original kernel (byte-identity by construction, confirmed by measurement). (2) fp64 changes the path but does not improve quality — it marginally regresses KLD. The fp64 encode is the true MSE-optimal Viterbi path in the FWHT-whitened domain, but MSE-optimal ≠ KLD-optimal: minimizing squared reconstruction error is a proxy, and fp32's rounding happens to land on paths a hair better on the downstream whole-distribution metric. The shift is real and measurable (sametop moves ~0.2 pp, KLD ~2% on turbo3_tcq) — confirming the trellis is precision-sensitive at the margin — but it is in the non-improving direction. Critically, fp64 KLD stays in-regime (0.204, no catastrophe), proving the fp32 decode reads fp64-encoded symbols correctly (no encode↔decode break).

Verdict: fp32 is confirmed sufficient and remains the default (byte-faithful to buun). TURBO_TCQ_FP64 is retained as an opt-in audit oracle — the canonical "true optimal path" reference for the precision-sensitivity question — not a quality lever; do not enable it for production. (fmad left at its default: fp64 FMA is more accurate via single rounding, the opposite of the chaotic-SDM bit-repro case where --fmad=false matters. Matrices: .opencoti/ppl-kld/tcq-matrix-fp32.tsv / -fp64.tsv.)

Decode V-accumulator precision (#547) — closed as moot, F32 is sufficient (2026-06-30). #547 proposed an fp32/fp64 lever for the decode-side V accumulator (distinct from the encode Viterbi above), premised on long-ctx accumulation drift pairing with sparse-V. Three facts close it without a build: (1) the FA-VEC decode V accumulator is already F32 (float2 VKQ, fattn-vec.cuh:207; KQ_sum is float), not f16 — there is no precision to recover by going fp32, and online-softmax rescales per tile so the combine is length-stable by construction; (2) F32 accumulation is already proven correct at 256k — turbo3_tcq (the most aggressive tier) holds the needle exactly, niah=100 at 256k (§5a-TCQ-RULER), with no observed drift; (3) the motivating premise ("pairs with sparse-V at long ctx") dissolved — sparse-V is an iSWA-only win (#546) whose global V-cache is bounded (≤1536-cell window class), so accumulation depth does not grow with ctx where sparse-V actually runs. An fp64 decode accumulator would cost ~1/64 throughput on the FA hot path for zero measured correctness gain. Not built; closed as F32-sufficient.

5a-TCQ-ENC. Encode throughput — warp-synchronous Viterbi (#533/#538, solidpc RTX 3090)

The TCQ encode (k_set_rows_turbo{3,2}_tcq) runs as a separate kernel on every KV write — once per prompt group at prefill, once per token at decode — so its cost lands directly on prefill tps. ncu on the original 512-thread/__syncthreads design (k_set_rows_turbo3_tcq, DSO e9e7f864) found it latency-bound on the 128-step block-barrier chain: SM throughput 25%, DRAM 0.1%, occupancy 33%, 39 µs/launch — neither compute- nor memory-bound, just stalling at barriers in lockstep (16 warps hit each of 128 __syncthreads together; ~3 blocks/SM ⇒ SM idle ~75%). That is exactly why the earlier #538 register/warp-shuffle pass (occupancy + bank-conflict tuning) moved nothing (−3.6%): the kernel was never occupancy- or memory-bound.

Fix (#533): collapse each 128-element group from a 512-/256-thread block to one 32-thread warp (turbo3: 16 states/lane; turbo2: 8 states/lane). The contiguous state→lane map makes each trellis group's predecessors fall entirely within one lane, so the per-step predecessor-min is thread-local (no shuffle); a 64-entry per-warp smem broadcast under __syncwarp replaces the block barrier; the backtrace moves to global (frees smem for occupancy). TCQ{3,2}_WPB=8 warps/block run at different DP steps instead of in lockstep, hiding the per-group serial latency. Zero __syncthreads in the new kernel (ptxas-confirmed). Reuses #538's warp-shuffle min primitive as the building block.

Validated byte-equivalent offline before the rebuild: the round-trip harness (.opencoti/tcq-warp-enc-full.cu, decode(encode(x)) vs the WHT reference on 4096 random-Gaussian groups) gives the warp encode cosine 1.00000 vs the old 512-thread design (full encode 3.74×; the DP alone 7.09×).

Prefill tps (A4B Q4_K_M, ~3682-tok prompt, m536-prefill-tps.sh):

tier	old (#538, e9e7f864)	warp (#533, 4644b66a)	Δ
f16 (shield)	2263	2296	+1.5% (parity)
scalar_q8 (shield)	2550	2532	parity (run noise)
turbo3 non-TCQ (shield)	2541	2520	parity
turbo3_tcq	1421	2189	+54% (+768 tps)
turbo2_tcq	1928	2283	+18.5% (+355 tps)

(Both warp tiers landed in one DSO 4644b66a; turbo3_tcq held 2189 byte-stable across the turbo3-only 9ef13bdc and the turbo2-build 4644b66a, confirming the turbo2 source change is isolated.) turbo3_tcq recovers the entire #534–536-era regression (1456→1421) and the historical 1798 mark, landing within 13% of the turbo3 non-TCQ ceiling (2536) — the residual is the encode work that remains (now ~3.7× cheaper), not barrier stall. turbo2_tcq, lighter at 256 states, lands above turbo3_tcq at 2283 (9% off the ceiling). f16/turbo3 unchanged confirm no collateral regression. Both warp encodes clock ~3.75× offline (256-/512-thread → warp: turbo2 0.895→0.239 ms; turbo3 likewise).

Correctness — turbo3_tcq (live, m533-kld-gate.sh, Qwen3-8B-Q8, ctx2048, DSO 4644b66a): turbo3_tcq‖f16 KLD 0.186 (Same-top 84%), matz‖fused 0.058, regression-shield plain-turbo3‖f16 0.350 — all in the expected class, no regime change. Consistent with the offline byte-equivalence: the Viterbi path and stored symbols are identical to the old 512-thread design (decode cosine 1.00000); only the norm/recon reductions reorder at the ~1e-7 fp level, so quality is unchanged by construction. (The 0.186 here vs the §5a-TCQ-FP64 matrix's 0.200389 is a harness-config difference — gate corpus/chunking — not an encode change; the gate is internally A/B-consistent.)

Correctness — turbo2_tcq (offline round-trip + live): turbo2 is fidelity-equivalent but not byte-identical to the 256-thread reference. The offline harness (.opencoti/tcq-warp-enc-t2.cu, 4096 Gaussian groups, decode(encode) vs WHT-ref) gives the warp encode mean cosine 0.96002 (min 0.944) vs the 256-thread reference's 0.95989 (min 0.935) — equal quantization fidelity — while warp-vs-256 decode cosine is 0.998969 with 7.2% of groups choosing a different path. Those differences are benign tie-break convention, not suboptimality: turbo2 is 2-bit/256-state (coarse ⇒ frequent exact cost-ties), and the warp argmin breaks ties by lowest-index where the 256-thread reduction breaks them by shuffle-order; both land on equally MSE-optimal paths (proven by the identical mean fidelity, and the warp's worst case 0.944 ≥ the reference's 0.935). turbo3 showed ~0% path divergence because 3-bit/512-state has far fewer ties.

Real-target regression gate (decisive, m533-27b-tcq-regress.sh, Qwen3.6-27B, ctx2048, vs cached base_f16_27b.dat): on the model where turbo2_tcq is usable, the warp encode shows no quality regression — turbo2_tcq‖f16 KLD 0.069940 vs the documented old-encode baseline 0.070933 (Δ −0.001, marginally better), Same-top 92.65 vs 92.67. The in-run control turbo3_tcq lands KLD 0.034606 / Same-top 95.54 with PPL(base) = 6.297831 exact, confirming the instrument (base+model+corpus+env all correct). So the tie-break path differences provably do not perturb a real-model distribution — exactly as expected (27B is not a chaotic regime). The live 8B KLD did move within-regime (turbo2_tcq‖f16 5.48→5.29, Same-top ~21%), but 8B turbo2 is a garbage regime (unusable either way); that shift is two equally-valid quantizations diverging where it doesn't matter, not a regression — the 27B gate is the one that counts, and it passes. (turbo3_tcq's 27B KLD differs −0.002 from its Jun-19 baseline; that is not the warp encode — byte-identical, 8B KLD 0.185718 identical across both warp DSOs — but the cumulative #534–537 decode smem-codebook ports that postdate that baseline; both tiers landing slightly better is consistent with them.)

Both warp encodes are a pure throughput optimization. Source lives in turbo-tcq-cuda.cuh + set-rows.cu (additive; bt always global, use_shared_bt ignored; acc_t fp64 path preserved).

5a-TCQ-DEC. Decode throughput in use — turbo3_tcq × MTP (#553, A4B v6-coder Q4_K_M, solidpc, 2026-06-24)

The shippable decode story for the extreme tier is turbo3_tcq + assistant-MTP. Measured on coherent in-domain output (chat coding prompt, /v1/chat/completions, --temp 0, 256 gen tok, graphs default, coherent-mtp.sh):

KV tier	decode tok/s (MTP-off)	decode tok/s (MTP-on)	MTP gain	draft acceptance
f16	88.95	112.37	+26%	0.787
turbo3	69.66	94.44	+36%	0.827
turbo3_tcq	56.63	75.63	+34%	0.652

MTP is the largest single decode-tps lever for turbo3_tcq (mtp.md §3.0/§3.0a), and it composes with CUDA graphs (graphs survive turbo3_tcq + sparse-V: +13.4% / +13.1% on the current DSO, matching decode-levers L6). The single-prompt turbo3_tcq acceptance (0.65) is prompt noise, not a deficit: over 3 diverse coherent prompts (coherent-mtp-multi.sh) the means are f16 0.745 / turbo3 0.757 / turbo3_tcq 0.703 (~0.05 spread, within each tier's own variance; turbo3_tcq even beat turbo3+f16 on one prompt). No spec bug — turbo3_tcq is the more accurate tier (KLD 0.0346 < turbo3 0.047, §5a-TCQ-27B).

⚠️ Measure decode-tps/acceptance on coherent output only. An instruction-less prompt (raw RULER/WikiText haystack → /completion) collapses greedy decode into degenerate repetition where acceptance is meaningless noise and even f16 shows greedy MTP-off ≠ MTP-on. An earlier haystack sweep mis-read turbo3_tcq acceptance at 0.25 / "MTP hurts turbo" — a harness artifact (bug-710), not a TCQ kernel/codebook bug; turbo3_tcq is the more accurate tier (§5a-TCQ-27B, KLD 0.0346 < turbo3 0.047) and cannot legitimately accept worse. Gemma-4 thinking-coder writes generation to choices[0].message.reasoning_content (.content is empty). Latent: TURBO_TCQ_CB override reaches only the FA-VEC __constant__, not the MMA-lift copy (d_turbo3_tcq_cb_cpy); harmless by default (baked tables byte-identical) but would split draft/verify decode during an A′ codebook sweep.

5b. Validated-tier full matrix — size · fidelity · throughput · quality (#529, cross-instrument)

The comprehensive scalar-vs-turbo, symmetric-vs-asymmetric comparison on the shippable tiers, post turbo3_tcq_w removal (#527, DSO sha 7f769e1d). Two instruments by necessity (see §5a-TCQ-A4B-KLD / #528): fidelity = PPL/KLD on Qwen3.6-27B (head_dim 128, the only valid KLD instrument — Gemma-4-A4B raw-PPL is ~44k garbage), quality-in-use = RULER niah on Gemma-4-A4B-98e-v6-coder (the deployed primary target). tps on both. DCA split only where it bites (32k+): DCA is ~inert at short ctx, and TCQ tiers now compose under DCA (as of #533; was rc=134 pre-#533, verified 2026-06-30) — the #533 turbo_tcq→f16 lift gives DCA's dca_lift_to_f16 a path, so DCA (which needs the lift) cannot compose; scalar + non-TCQ-turbo compose via the #446 lift. 256k/512k RULER deferred to bs2 (Blackwell) per plan. Status: #529 COMPLETE — size · noDCA fidelity · throughput (27B+A4B) · RULER niah all LIVE on the all-quants GPU FA-VEC DSO (#531). DCA-fidelity column is n/a on the perplexity instrument (server-only flag, ⁴); DCA composition proven server-side (#444) + §1. 256k/512k RULER → bs2.

WS2 DCA-on throughput column (#629). The throughput numbers in this matrix are the noDCA in-register / f16-lift decode path. For the DCA-on @256k serving path the shippable turbo/TCQ tiers now decode via the WS2 in-register turbo-V kernel (DEFAULT-ON), a +68–95% decode win over the f16-lift ref, retrieval-lossless (niah 100) — see §5x for the full D512 (A4B/31B) + D256 (omni27/qw35a3) matrix. In-register turbo/TCQ-V is a decode WIN with DCA-on at long ctx, the opposite of the noDCA short-ctx "turbo costs decode vs scalar" floor (§5a-TCQ-DEC, decode-levers.md).

Size — bytes/value (KV stored size, both K and V) vs f16. Analytical from block layout; asymmetric = mean(K,V).

config	ctk/ctv	bpv	size vs f16
f16 (ref)	f16/f16	16.0	100 %
q8_0	q8_0/q8_0	8.5	53.1 %
q6_0	q6_0/q6_0	6.5	40.6 %
q5_0	q5_0/q5_0	5.5	34.4 %
q4_0	q4_0/q4_0	4.5	28.1 %
q8_0K / q6_0V (asym)	q8_0/q6_0	7.5	46.9 %
q5_0K / q4_0V (asym)	q5_0/q4_0	5.0	31.3 %
turbo3_tcq	turbo3_tcq/turbo3_tcq	3.25	20.3 %
turbo2_tcq	turbo2_tcq/turbo2_tcq	2.25	14.1 %
t3tcqK / t2tcqV (asym)	turbo3_tcq/turbo2_tcq	2.75	17.2 %
t2tcqK / t3tcqV (asym)	turbo2_tcq/turbo3_tcq	2.75	17.2 %

Fidelity — Qwen3.6-27B, full-vocab PPL + KLD vs cached f16 base (perplexity host build, ctx 2048, wiki.test.mid.raw, base_f16_27b.dat; f16 ref PPL 6.298). KLD↓, Same-top↑. DCA column n/a on this instrument⁴ — the perplexity tool has no --dca flag (DCA is server-only); composition with quantized KV is proven server-side, not via full-vocab KLD. TCQ now composes under DCA (as of #533); its DCA-KLD value remains server-only by the instrument limitation (not a TCQ block).

config	ctk/ctv	PPL(Q)	KLD noDCA	top% noDCA	KLD DCA	top% DCA
q8_0	q8_0/q8_0	6.313	0.01760	97.66	n/a⁴	n/a⁴
q6_0	q6_0/q6_0	6.310	0.01965	97.68	n/a⁴	n/a⁴
q5_0	q5_0/q5_0	6.291	0.01973	97.46	n/a⁴	n/a⁴
q4_0	q4_0/q4_0	6.334	0.02505	96.85	n/a⁴	n/a⁴
q8_0K / q6_0V (asym)	q8_0/q6_0	6.319	0.01680	97.63	n/a⁴	n/a⁴
q5_0K / q4_0V (asym)	q5_0/q4_0	6.310	0.02424	97.09	n/a⁴	n/a⁴
turbo3_tcq	t3tcq/t3tcq	6.389	0.03665	95.64	composes (#533)	composes (#533)
turbo2_tcq	t2tcq/t2tcq	6.456	0.07093	92.67	composes (#533)	composes (#533)
t3tcqK / t2tcqV (asym)	t3tcq/t2tcq	6.431	0.05150	93.81	composes (#533)	composes (#533)
t2tcqK / t3tcqV (asym)	t2tcq/t3tcq	6.346	0.05667	94.02	composes (#533)	composes (#533)

⁴ perplexity rejects --dca (error: invalid argument: --dca) — DCA is a server-only flag, so a full-vocab DCA-on KLD column is not obtainable on this instrument (the attempted sweep, s5b-dca-fidelity.sh, failed at arg-parse on all cells — no GPU time spent). DCA's quantized-KV composition is instead proven on the server path: the m444 gate's teacher-forced logit-equivalence (f16-DCA vs q8-DCA, real_frac=0, #444) and the §1 RULER-VT 256k→1M cells (all DCA-on, scored 0.876–0.984). Scalar/asym compose via the #444 f16-lift; TCQ now composes under DCA (as of #533; was rc=134 pre-#533, verified 2026-06-30). The takeaway the column would have shown — DCA does not degrade quantized-KV fidelity — is the server-side evidence above, not a perplexity number.

Note (#531 — RESOLVED, the all-quants FA-VEC default). Earlier the production CUDA DSO was built with FA_ALL_QUANTS=0 (cuda.sh default), so collect_gpu_sources compiled only the 4 minimal fattn-vec instances (f16/q4_0/q8_0/bf16) + the q6_0 hook, and the dispatch block for q5_0/q5_1/all asymmetric pairs was #ifdef'd out — q5_0/q5_0 had no native GPU FA-VEC kernel and silently fell back to CPU-FA (a 568 %-CPU crawl that timed out the perplexity cell, and would have crawled every RULER/tps cell). Fixed, not papered over: GGML_CUDA_FA_ALL_QUANTS is now the shipped default (build-pipeline.ts forwards --fa-all-quants to the cuda backend), so every scalar K/V combo (q5_0, q5_1, q6_0, all symmetric and asymmetric) compiles a native GPU FA-VEC kernel — no KV tier falls back to CPU-FA. The all-quants DSO was installed into both side-load paths (~/.llamafile/v/<ver>/ggml-cuda.so + vendors/dist/.../<ver>/ggml-cuda.so, byte-identical; <ver> resolved by scripts/llamafile-version.sh, never hardcoded — bug-635). No KV type is preferred over another on GPU-path grounds; all run GPU FA-VEC (or, at long ctx, GPU FA-VEC under DCA). Every cell in this §5b — q5_0 fidelity, all throughput, all RULER — was (re-)measured on this DSO; the q8_0_q6_0 RULER cell went from a ~2 h CPU-FA crawl to ~30 s GPU FA-VEC, confirming the fix end-to-end.

Throughput — decode/prefill tps (server /completion timings, ctx 4096, ignore_eos; pp = prefill, tg = generation; all GPU FA-VEC on the all-quants DSO, #531).

config	ctk/ctv	27B pp	27B tg	A4B pp	A4B tg
f16 (ref)	f16/f16	911	22.5	2462	73.1
q8_0	q8_0/q8_0	919	22.2	2558	69.1
q6_0	q6_0/q6_0	927	22.0	2579	67.4
q5_0	q5_0/q5_0	920	22.0	2624	68.0
q4_0	q4_0/q4_0	929	22.2	2684	70.1
q8_0K / q6_0V (asym)	q8_0/q6_0	922	22.2	2565	68.2
q5_0K / q4_0V (asym)	q5_0/q4_0	923	22.2	2635	69.8
turbo3_tcq	t3tcq/t3tcq	357	18.9	373	39.1
turbo2_tcq	t2tcq/t2tcq	382	19.3	424	41.1
t3tcqK / t2tcqV (asym)	t3tcq/t2tcq	277	18.4	270	36.1
t2tcqK / t3tcqV (asym)	t2tcq/t3tcq	279	18.4	266	36.3

Reading (throughput). Among scalar tiers, KV-quant type is throughput-neutral: 27B clusters at 920 pp / ~22 tg, A4B at ~2460–2680 pp / ~67–73 tg regardless of K/V width — the decode is weight-bound, not KV-bound, so picking q4_0 over q8_0 buys size, not speed. TCQ pays a decode tax (in-register Viterbi codebook lookup, #389): on the big 27B it is mild (15 % slower tg, 2.5× slower pp — the model is weight-bound, so KV-decode overhead is a small fraction of the per-token cost); on the small fast A4B it is steep (45 % slower tg, ~6× slower pp — here KV-decode is a large fraction). So TCQ's 3–14 %-of-f16 size win costs the most throughput exactly where the model is cheapest — the size/speed trade is real and model-size-dependent. Provenance: .opencoti/ppl-kld/s5b-tps-{27B,A4B}.tsv. — noDCA / short-ctx (ctx 4096) only. This decode tax is the f16-lift path at short context; with DCA-on at long ctx (@256k) the WS2 in-register turbo-V path is a decode WIN (+68–95%) — the turbo/TCQ tier decodes faster than the f16-lift ref, not slower. See §5x.

Quality-in-use — RULER niah_single_1, Gemma-4-A4B-98e-v6-coder (% correct, 8 samples; all GPU FA-VEC on the all-quants DSO, #531). 256k/512k measured on bs2 Blackwell, DCA-on --dca-chunk-size 32768, 5 samples each (#575, 2026-06-30).

config	ctk/ctv	niah 4k	niah 32k	32k DCA	256k (bs2)	512k (bs2)
f16 (ref)	f16/f16	100	100	≈noDCA²	100	100
q8_0	q8_0/q8_0	100	100	≈noDCA²	100	100
q6_0	q6_0/q6_0	100	100	≈noDCA²	100	100
q4_0	q4_0/q4_0	100	—³	≈noDCA²	100	100
q8_0K / q6_0V (asym)	q8_0/q6_0	100	100	≈noDCA²	100	100
turbo3_tcq	t3tcq/t3tcq	100	100	composes (#533)	100	100
turbo2_tcq	t2tcq/t2tcq	100	—³	composes (#533)	100	100
t3tcqK / t2tcqV (asym)	t3tcq/t2tcq	100	—³	composes (#533)	100	80⁴

² 32k-DCA niah was not separately run; ≈noDCA is by inertness, not a fresh measurement. At 32k (within the model's effective window — noDCA niah is already saturated at 100), DCA's position-remapping is ~identity, so DCA-on niah ≈ the noDCA cell. DCA's measured effect on KV-quant quality is the long-context story: the canonical DCA-on retrieval evidence is §1 (RULER-VT 256k → 1M, all DCA-on, scored 0.876–0.984) and the bs2 256k/512k niah cells above (every tier 100, save the 2-bit-V asym at 512k = 80, ⁴). Scalar/asym tiers compose with DCA via the #444 f16-lift (m444 gate real_frac=0); TCQ now composes under DCA (as of #533, the #505 lift; bs2 256k/512k niah filled above, #575). ³ 32k cell run only for the symmetric-scalar and headline-TCQ tiers (q4_0 and the asym/secondary-TCQ tiers swept at 4k to bound the matrix; 32k niah is saturated at 100 for every tier that was run, so the omitted 32k cells are not in doubt). ⁴ t3tcqK/t2tcqV asym is the only non-100 cell — niah 80 at 512k (still 100 at 256k). This is the same tier §9's #444 all-KV matrix flagged as the lone DCA-sensitive case (turbo3_tcq/turbo2_tcq asym, 318→173 on the count-up task): a weak 2-bit-V tier pushed over by DCA's harder positional task at 2× native ctx. It is an intrinsic 2-bit-V quality floor at extreme ctx, not a DCA correctness bug — the symmetric turbo2_tcq holds 100 at 512k, and every symmetric/scalar tier is 100 at both depths. Boot was clean (no SRV-FAIL); the cell ran full 5 samples. Recommendation: don't pair a 2-bit V with a 3-bit K beyond native ctx; use symmetric turbo2_tcq or q8_0K/q6_0V instead.

SUPERSEDED (2026-07-10, bug-2118 + bug-2141). This whole ⁴ reading is invalid: these #575 cells ran with --dca on while bug-2118 made DCA a silent no-op on Gemma, so the "80 at 512k / 100 at 256k" was DCA-off, and the "intrinsic 2-bit-V floor" framing is wrong. On the fixed WS2 binary with genuine DCA (§12 Tranche 3b), the truth inverts: asym turbo3_tcq-K/turbo2_tcq-V is 20 at 256k (4/5 empty, broken forward) and 100 at 512k — a geometry-dependent NaN in the WS2 in-register-V + wht_o path (bug-2141), NOT a 2-bit-V quality floor (symmetric turbo2_tcq holds 100 at both depths, and WS2_DCA_TURBO_V=0 recovers the asym cell to 100/0-empty). See §12 Tranche 3b + .wolf/buglog.json bug-2141. The symmetric-tier recommendation still stands, but for the right reason (quarantine asym turbo_tcq K≠V until the NaN-safe fix, not tier quality).

Reading (fidelity). Asymmetric q8_0K/q6_0V is the standout — KLD 0.0168, the best non-q8 cell, at 46.9 % size (K-precision dominates; cheap V). The TCQ ladder holds its published order (turbo3_tcq 0.0366 < asym 0.051–0.057 < turbo2_tcq 0.071) at 3–14 % of f16 — far smaller than any scalar, the extreme-compression frontier. Scalars cluster tight: q8_0 0.0176 / q6_0 0.0197 / q5_0 0.0197 (top 97.5 %) / q4_0 0.0251 (top 96.8 %) — a smooth 8→5→4-bit ladder, every cell GPU FA-VEC. noDCA fidelity is now 10/10 (the q5_0-symmetric cell, formerly a CPU-FA timeout, landed at KLD 0.0197 on the all-quants DSO — sitting exactly at the q6_0 point, confirming q5_0/q6_0 are quality-equivalent on 27B and the choice between them is pure size, #531). Provenance: .opencoti/tcq-survivors-27b.sh + .opencoti/s5b-rerun.sh → .opencoti/ppl-kld/27b-survivors.tsv (+ .q5note); DCA-on column from .opencoti/s5b-dca-fidelity.sh.

5b-8B. Scalar + asymmetric K/V — extended eval (8B instrument, historical)

Full matrix — every cell ran; no unsupported combos (the in-register FA-VEC path serves all the mixed K/V pairs below). Historical-collection note (superseded by #531): when these 8B numbers were collected, the DSO was the minimal build (GPU FA-VEC instances for f16/q8_0/q4_0/bf16 only), so q5_1/q5_0 ran through a slow CPU-FA fallback (~107 s/pass). The values stand unchanged — CPU-FA and GPU FA-VEC compute identical attention, so KLD/PPL are DSO-invariant (only wall-clock differed). On the current shipped all-quants DSO (#531) every scalar pair, q5_1/q5_0 included, runs native GPU FA-VEC — nothing falls back to CPU-FA (re-confirmed on 27B in §5b above, where q5_0 went GPU FA-VEC). f16 ref PPL: noDCA 9.451 · DCA 9.441.

config	ctk/ctv	PPL noDCA	KLD noDCA	top% noDCA	PPL DCA	KLD DCA	top% DCA
q8_0	q8_0/q8_0	9.447	0.00086	98.8	9.449	0.00135	98.3
q5_1	q5_1/q5_1	9.470	0.00605	96.9	9.603	0.0230	93.7
q5_0	q5_0/q5_0	9.485	0.00703	96.9	9.496	0.0183	94.3
q4_0	q4_0/q4_0	9.601	0.0329	92.6	9.664	0.0560	90.0
q8K/q4V	q8_0/q4_0	9.469	0.00282	97.8	9.491	0.00461	97.3
f16K/q4V	f16/q4_0	9.440	0.00281	97.8	9.490	0.00432	97.3
t8K/t4V	turbo8/turbo4	9.510	0.00814	96.3	9.468	0.00767	96.3
t4K/t2V	turbo4/turbo2	10.47	0.172	82.9	10.13	0.140	84.3
t3K/q4V	turbo3/q4_0	12.78	0.426	75.3	10.19	0.183	82.6

Reading.

• Scalars — DCA costs a little, the inverse of turbo. The noDCA FA reads scalar KV faithfully, so it is the reference-quality path; DCA's f16-lift adds a small round-trip (q8 0.0009→0.0014, q4_0 0.033→0.056). Both fine (top 90–99%). Compare §5a, where the noDCA turbo in-register read is the lossy path so DCA helps. Rule of thumb: scalar KV → noDCA is the quality path; turbo KV → the dequant-on-lift (DCA/#444) is the quality path.
• Asymmetric is the memory sweet spot. q8K/q4V and f16K/q4V both land at KLD ≈ 0.003, top ≈ 98% — near-q8 quality at q4-sized V. K-precision dominates; raising K past q8 (→f16) buys nothing, V is the compressible side. t8K/t4V is the turbo analogue (KLD ≈ 0.008, top 96%): turbo8-K rescues turbo4-V — 17× better than symmetric turbo4 (0.14). These are the recommended long-context KV recipes.
• The turbo2 degeneracy is K-side. Symmetric turbo2 noDCA is catastrophic (KLD 5.60, §5a), but t4K/t2V — turbo2 on V only, turbo4 K — is fine (KLD 0.172 noDCA). So the in-register 2-bit failure is the K-read (vec_dot_KQ), not the V-read: turbo2 is usable on the V side or via the faithful lift, never as the in-register K type. (refines .wolf/buglog.json bug-583.)

5b-8B-ctx. KV-quant × DCA across the context ladder (#580, bs2 dual-Blackwell, 2026-07-01)

Extends the §5b-8B single-ctx (2048) matrix into a ctx ladder 2k → 128k on the same instrument (Qwen3-8B-Q8_0, TURBO_INNERQ_SCALE head_dim 128). Ran on bs2 (2× RTX PRO 6000 Blackwell, GPU0 = 2k/4k/8k/32k, GPU1 = 64k/128k) via the GPU-perplexity binary with DCA re-exposed to --kl-divergence. Corpus wiki.long (test.mid+valid.mid, ~151k tok; 128k used a 2× concat, ~302k tok — perplexity needs 2× ctx tokens per window). Chunk = 1024 (2k/4k/8k), 2048 (8k variant), 8192 (32k/64k/128k = production chunk). Qwen3-8B native n_ctx_train = 40960, so 64k = 1.6× and 128k = 3.2× native — the only two cells where DCA actually engages; 2k–32k are sub-native (DCA ≈ no-op).

(a) DCA rescues f16 coherence past native — the headline. f16 reference PPL, no KV quant:

ctx	×native	f16 PPL noDCA	f16 PPL DCA
2k	0.05×	8.22	8.22
4k	0.10×	8.29	8.32
8k	0.20×	8.00	8.02
32k	0.80×	7.70	7.77
64k	1.6×	8.28	8.39
128k	3.2×	19.16	7.34

Plain attention survives to 1.6× native (64k PPL 8.28, ~unchanged) but breaks at 3.2× (128k PPL 19.16); DCA-on holds PPL at 7.34 — a clean, direct measurement of DCA extending usable context on the f16 reference itself. (Below native, DCA costs a hair — expected, it's a no-op there.)

(b) On the DCA path the KV-quant ladder is ctx-invariant — except 2–3-bit turbo. KLD (DCA, vs each regime's own f16 base):

tier	ctk/ctv	2k	4k	8k	32k	64k	128k
q8_0	q8_0/q8_0	0.0014	0.0015	0.0012	0.0011	0.0012	0.0011
turbo8	turbo8/turbo8	0.0019	0.0017	0.0014	0.0013	0.0015	0.0013
f16K/q4V	f16/q4_0	0.0041	0.0038	0.0033	0.0029	0.0031	0.0028
q8K/q4V	q8_0/q4_0	0.0044	0.0046	0.0037	0.0033	0.0037	0.0032
t8K/t4V	turbo8/turbo4	0.0077	0.0069	0.0063	0.0057	0.0061	0.0055
q5_0	q5_0/q5_0	0.0187	0.0187	0.0180	0.0165	0.0186	0.0161
q4_0	q4_0/q4_0	0.0558	0.0558	0.0563	0.0562	0.0627	0.0662
turbo4	turbo4/turbo4	0.0777	0.0711	0.0653	0.0592	0.0659	0.0626
turbo3	turbo3/turbo3	0.180	0.185	0.194	1.335	2.375	2.221
turbo2	turbo2/turbo2	0.636	0.675	0.733	2.893	3.111	3.911

(8k column = chunk-1024 variant; chunk-2048 is within noise except turbo2-DCA, where the coarser chunk is worse — 1.22 vs 0.73 — so smaller DCA chunks help the fragile low-bit tier.)

Reading.

• The §5b-8B recipe holds to 3.2× native. q8_0/turbo8 (0.001), q5_0 (0.017), q4_0/turbo4 (0.06), and the asymmetric hi-K/cheap-V winners **f16K/q4V · q8K/q4V · t8K/t4V (0.003–0.008)** are flat across the whole ladder — the single-ctx conclusion (asymmetric hi-K/cheap-V = the long-ctx sweet spot) is now confirmed to extend, unchanged, to 128k.
• 2–3-bit turbo is a sub-native economy only. turbo2/turbo3 are fine at ≤8k (turbo3-DCA 0.18–0.19) but cross into unusable right around native — turbo3 jumps to 1.34 @32k, 2.38 @64k; turbo2 to 2.9 @32k. turbo4 is the low-bit floor that survives (0.06 through 128k, and DCA lowers it vs noDCA — the §5b turbo/lift rule). Serving past native ⇒ ≥4-bit KV with a high-precision K side.
• The DCA×quant sign split (§5b-8B) persists at every ctx: DCA slightly raises scalar-quant KLD (q4_0 noDCA ~0.025 → DCA ~0.056) but lowers the turbo dequant-lift KLD (turbo4 ~0.087 → ~0.063).
• Provenance: .opencoti/ppl-kld-4k8k.sh + bs2 bs2-driver.sh; per-ctx matrices at bs2 /srv/ml/opencoti-c1/ppl/out/ppl-kld-<ctx>/matrix.md. Corpus-size gotcha (perplexity needs 2× ctx tokens) = buglog bug-2093; the GPU-perplexity DCA re-exposure was a temporary arg.cpp edit (reverted).

5b-14B-1M-ctx. KV-quant × DCA ladder on the 1M-native production class (#581, bs2 GPU1, 2026-07-01→04)

Re-runs the §5b-8B-ctx ladder on Qwen2.5-14B-Instruct-1M-Q8_0 — the HAL/1M serving vehicle (n_ctx_train = 1M native, no YaRN) — ctx 2k → 256k, 13 KV configs × {noDCA, DCA} per tier, corpus wiki.14b.raw, chunk 1024 (2k/4k) / 2048 (8k) / 8192 (32k+). Driver bs2:/srv/ml/opencoti-c1/ppl/bs2-driver-14b.sh; matrices …/ppl/out14b/ppl-kld-<ctx>/matrix.md. Crucially, every rung here is sub-native for this model — so this ladder measures intrinsic long-range quant sensitivity, not DCA rescue.

(a) KLD (noDCA, vs same-regime f16 base) — the ladder. 256k* = partial (5–6/9 chunks; the 3600 s/cell cap timed out — f16 base alone needs 478 s/pass — cumulative KLD from the cell logs, stable to 3 digits by chunk 5):

tier	ctk/ctv	2k	4k	8k	32k	128k	256k*
q8_0	q8_0/q8_0	0.0023	0.0023	0.0023	0.0024	0.0029	0.0051
turbo8	turbo8/turbo8	0.0034	0.0034	0.0034	0.0035	0.0040	0.0067
f16K/q4V	f16/q4_0	—	0.0082	0.0081	0.0080	0.0087	0.0128
q8K/q4V	q8_0/q4_0	—	0.0084	0.0083	0.0082	0.0089	—
q5_1	q5_1/q5_1	0.0124	0.0125	0.0126	0.0129	0.0142	0.0194
q5_0	q5_0/q5_0	0.0147	0.0148	0.0149	0.0153	0.0170	0.0250
t8K/t4V	turbo8/turbo4	—	0.0198	0.0195	0.0191	0.0206	0.0326
q4_0	q4_0/q4_0	0.0525	0.0536	0.0546	0.0558	0.0623	0.0836
turbo4	turbo4/turbo4	0.115	0.118	0.122	0.131	0.154	0.232
t4K/t2V	turbo4/turbo2	—	0.234	0.238	0.247	0.286	0.437
t3K/q4V	turbo3/q4_0	—	0.235	0.243	0.272	0.319	0.466
turbo3	turbo3/turbo3	0.250	0.263	0.272	0.301	0.356	0.524
turbo2	turbo2/turbo2	1.30	1.42	1.51	1.72	2.13	2.89

(2k asym + all 2k-DCA and 64k-noDCA cells are harness artifacts, not quality data — truncated f16 base .dat → kl_divergence: failed reading log-probs; the 256k "FAIL" wall in matrix.md is the per-cell timeout, not a crash. DCA columns at 4k–128k track noDCA with the usual §5b sign split; @256k-DCA partials: q8_0 0.0065 · q5_1 0.0427 · q5_0 0.0866 · q4_0 0.296 · turbo2 3.19.)

Reading.

• The frozen 14B is more quant-sensitive than the 8B — and degradation grows with ctx even sub-native. Where 8B turbo3-DCA sat at 0.18–0.19 sub-native, 14B turbo3 starts at 0.25 @2k and climbs to 0.52 @256k; turbo2 is dead everywhere (1.3→2.9). Every low-bit tier roughly doubles from 2k→256k while q8/turbo8 stay ≤0.007 — long-range attention amplifies K/V error on this model; there is no safe regime for 2–3-bit turbo here, unlike the 8B.
• The serving recipes hold to 256k: q8_0/turbo8 (≤0.007), asym f16K/q4V ≈ q8K/q4V (0.008–0.013), t8K/t4V (0.02–0.03), q5-class ≤0.025; turbo4 is again the low-bit floor (0.23 @256k — usable, no margin).
• DCA on a native-1M model costs, as expected (Qwen2.5-1M needs no extension): f16 ref PPL @128k noDCA 3.486 vs DCA 3.886 — approximation cost; the DCA columns are composition coverage, not a recommendation for this model.
• HAL (#582) implication — the last lever is now quantified. Route (b) "low-bit history ⊕ exact recent" cannot use turbo2/3 history on this class (0.5–2.9); it needs ≥4-bit with a high-precision K side (t8K/t4V 0.033, q8K/q4V ~0.01 @256k), capping the history-compression win at ~4× vs f16 rather than the 8–10× a 2–3-bit history would give. Route (a) host-stream-exact is unaffected.

5x. WS2 turbo-V in-register — the KV-decode-tps win (#629, DEFAULT-ON)

WorkStream-2 = read a rotated turboN V-cache IN-REGISTER inside the multichunk DCA fused flash-attn kernel, killing the per-forward whole-cache cpy_turboN_f16 cast (a context-growing decode bottleneck — the same D512-global materialize tail that sank WS1's turbo-MMA @256k, §5a / poly_kv.md S2). Gated by an n_q≤16 gate: in-register for decode + small spec-verify batches; prefill keeps the byte-identical f16-lift path. Now DEFAULT-ON.

Losslessness gating (read this before the table). This is a vec-only kernel change, so PPL/KLD are BLIND to it (they run the MMA/prefill path — same blindness as sparse-V §10, decode levers). The authoritative gates are RULER niah (retrieval) + teacher-forced logit-equivalence (real_frac = fraction of tokens whose argmax moved, mean_tv = mean top-k total-variation, ffa = full-faithful-argmax rate). Not PPL, not KLD.

Full matrix — all @256k, ctk=q8_0, DCA-on chunk 32768. "ref" = f16-lift (WS2=0), "ours" = in-register (WS2=1); decode tps.

D512 — Gemma (global head_dim 512, iSWA):

model · tier	ref tps	ours tps	Δ	real_frac	mean_tv	ffa	niah (n)
A4B turbo2_0	48.2	91.2	+89%	(see A4B block below)	—	—	100
A4B turbo3_0	43.0	80.7	+88%	0.0	0.00113	1.0	100 (5)
A4B turbo2_tcq	39.8	66.8	+68%	0.0	0.01784	1.0	100 (5)
A4B turbo3_tcq	37.6	64.5	+72%	0.0	0.01078	1.0	100 (5)
31B turbo2_0 (dense+SWA)	11.7	21.6	+85%	1.0 ⚠	0.0137	0.0 ⚠	100 (5)

D256 — Qwen-arch (global head_dim 256; qwen35 / qwen35moe, no SWA, native 256k):

model · tier	ref tps	ours tps	Δ	real_frac	mean_tv	ffa	niah (n)
omni27 turbo3_0	15.0	27.9	+86%	0.0	0.00107	1.0	100 (5)
omni27 turbo2_tcq	13.5	25.2	+87%	0.0	0.00735	1.0	100 (13)
omni27 turbo3_tcq	12.6	24.6	+95%	0.0	0.00378	1.0	100 (13)
qw35a3 turbo3_0	43.0	76.1	+77%	0.0	0.00919	1.0	100 (5)
qw35a3 turbo2_tcq	40.5	75.5	+86%	0.0	0.01644	1.0	100 (13)
qw35a3 turbo3_tcq	39.1	75.0	+92%	0.0	0.01428	1.0	100 (13)

turbo2_0 on D256 (omni27, qw35a3) is a clean +86–98% win too — see the eval-cell artifact caveat below; the number 15.8/15.8 (omni27) and 49/49 (qw35a3) you may find in the ws2-eval-cell.sh output are a measurement artifact, not zero benefit.

Two caveats (documented honestly, not hidden):

1. 31B turbo2_0 TV top-k-SET churn. This one cell shows real_frac 1.0 / ffa 0.0 yet mean_tv only 0.0137 — the distribution mass is ~identical but the top-20 membership reorders in the tail (a TV-metric artifact on the SWA model), NOT a correctness failure. niah 100 (n=5) is the authoritative retrieval gate and it passes clean. Do not over-claim bit-losslessness for this cell; do read it as retrieval-lossless.
2. D256 turbo2_0 eval-cell tps artifact. The ws2-eval-cell.sh tps phase self-measured ours≈ref (omni27 15.8/15.8, qw35a3 49/49) — a known boot mis-attribution in that harness (both arms booted into the same env), NOT zero benefit. The clean per-boot-env A/B (ws2-qwen-ab2.sh) shows the real +86–98% wins, fully consistent with the turbo3_0 / *_tcq cells in the same rows.

Perf-regression before/after (bug-2140 fatbin-last DSO is perf-free). The pure -Wl fatbin-last layout change carries zero kernel-perf cost — A4B turbo2_0 @256k:

build	WS2	decode tps	prefill tps
bug-2140 DSO	0	48.22	4388
baseline	0	48.1	4435
bug-2140 DSO	1	91.2	4406
baseline	1	91.2	4433

All deltas ≤1% noise → the fatbin-last relayout is confirmed perf-free; the +89% is the WS2 kernel, not a build artifact.

Scalar-KV reference (cost-of-compression). A4B & 31B D512 q8_0/q8_0 and q8_0/q4_0 scalar baselines were captured (VRAM bits: q8=8, q4=4, turbo3=3, turbo2=2) — see the ws2-scalar cells in the eval dir for the numbers.

Ctx-ladder (A4B turbo2_0) — the win GROWS with context. From the per-ctx tps sweep (ws2-s2-tps/results.tsv, ref=WS2=0 f16-lift, ours=WS2=1 in-register, decode tps median of 3):

ctx	ref tps	ours tps	Δ	prefill (ref/ours)
128k	73.3	113.6	+55%	6014 / 6076
256k	48.1	91.2	+89%	4436 / 4433
512k	28.5	57.3	+101%	2885 / 2871

This is the direct signature of the WS2 mechanism: the eliminated cost is the per-forward whole-cache cpy_turboN_f16 materialize cast, an O(n_kv) cost that grows with context, so removing it wins bigger as ctx grows (+55 → +101% over 128k→512k) — the mirror image of WS1's register-spill loss that grew worse with ctx. Prefill is at parity at every rung (the n_q≤16 gate keeps prefill on the byte-identical f16-lift path).

Default-on policy. WS2 in-register turbo-V is DEFAULT-ON — it is a serving-path decode win (+68–95% @256k across both arches and every turbo/TCQ tier), retrieval-lossless (niah 100 everywhere) and logit-lossless everywhere except the one SWA top-k-churn caveat. It composes with the MTP draft path (see mtp.md §3g — both lossless, both stack) and is the opposite verdict of WS1's turbo-MMA null (§5a / poly_kv.md S2): WS1 was noDCA and the D512 cast it had to beat was still paid; WS2 is DCA-on multichunk-fused and eliminates that cast in-register. Cross-refs: three-way-tps.md (3-way tps harness + model registry), poly_kv.md S2 (WS1↔WS2 contrast).

5c. Status / optional follow-ups

• §5b-14B-1M-ctx (#581) — DONE 2026-07-04 (above): ladder 2k→256k on the 1M-native 14B; recipes hold, 2–3-bit turbo has no safe regime on this class, HAL route-(b) floor = 4-bit hi-K.
• HAL #582 — DECIDED 2026-07-05, full study in hal_kv_compression.md: resident compression (q8K/q4V default) wins; host-stream-exact is 22–27× slower at equal ctx (0.99 tps @64k on 50 GB/s PCIe) → overflow fallback only; quant decode on the 14B is kernel-bound not bandwidth-bound (q4/q4 == q8K/q4V tps; ~2× vs f16, ~6× below bandwidth ceiling) → P0 kernel work; native-1M fits one 96 GB card only compressed (192 GiB f16 → 78 GiB q8K/q4V).
• Scalar/asym cells — LOCKED (§5b full matrix) and ctx-swept 2k→128k (§5b-8B-ctx, #580, DONE 2026-07-01): the asymmetric hi-K/cheap-V recipes stay flat (~0.003–0.008) to 3.2× native; DCA rescues f16 PPL at 128k (19.2→7.3); 2–3-bit turbo collapses past native, turbo4 is the low-bit floor.
• TCQ tiers (#447) turbo2_tcq/turbo3_tcq — DONE + target-class verified (§5a-TCQ-27B: Qwen3.6-27B turbo3_tcq KLD 0.037, turbo2_tcq 0.071, both beating their plain siblings). The 8B/1.5B in-register low-bit collapse is model fragility, not a kernel defect; TCQ ships as-is on 27B-class/Gemma-4 targets, f16-lift reclassified to optional small-model support.
• Gemma-4 head_dim-512 (1M production target) — DONE: PPL/KLD pointed at Gemma-4-A4B found the instrument invalid on the omnimerge GGUFs (§5a-TCQ-A4B-KLD / bug-631), so quality-in-use is validated via RULER niah (§5a-TCQ-RULER) + the §5b cross-instrument matrix instead.
• Provenance: runner .opencoti/ppl-kld-matrix.sh; per-cell logs .opencoti/ppl-kld/*.log; matrix .opencoti/ppl-kld/matrix.{tsv,md}; GPU-perplexity link .opencoti/ppl-link.mk + ppl_main_gpu.cpp.

---

6. Campaign status & open items

Both eval campaigns are complete; no GPU jobs are running.

• TCQ codebook campaign (#523–525, solidpc): concluded — final verdict in §5a-TCQ-CB (the deployed turbo3_tcq symmetric + the #513 asym are already at the TCQ frontier; none of {codebook retrain, free state-widen, wide-K _w-asym} beats them; the buggy/redundant _w tier was reverted in #527).
• bs2 long-context quality ladder: finished; bs2 GPUs idle of ladder work.

GPU	arm	final state	last cell
bs2 GPU0	RULER-VT	complete — 256k/512k/768k/1M all landed (0.976·0.960 / 0.984 / 0.956 / 0.876, §1a; 1M corrected 07-01 from server-crash 0.716)	ruler_native_vt_1024k_dcaon ended 2026-06-15; 1M 9-sample re-fix 2026-07-01
bs2 GPU1	MRCR	complete — n=50 mrcr_256k_dcaoff/on + 512k (0.1138 / 0.1320 / 0.1027); 768k_synth/1M small-n (0.1259 / 0.0273); + quant-KV cell q8_c1 @256k-on 0.0999 (< f16-off; §1b)	mrcr_256k_dcaon_q8_c1 landed 06-24

Driver: bs2:/srv/ml/omk_mrcr_dca/dca_quality_ab_bs2.sh (logs bs2:/srv/ml/omk_mrcr_dca/ladder_logs/). RESOLVED (2026-07-01): the 9/50 empty completions at 1M (§1a) were a serving artifact — ConnectionError after the F16 server crashed at sample 40, not coherence loss. Re-running just the 9 on a fresh server (cache-resume) gives the true n=50 VT-1M = 0.876; the 0.716 is retired.

Metric-field caveat (read scores from scores.metric). summary.json's top-level metric field is a generic roll-up label that prints pass_at_1 for MRCR — it is not the real metric. The authoritative metric is scores.metric = sequence_matcher_ratio (MRCR) / string_match_all (RULER-VT). All scores in §1 are the canonical summary.json .score.

---

7. Provenance / source paths

• Quality scores (canonical): bs2:/srv/ml/omk_mrcr_dca/results/<template>/<served>/summary.json .score
• Per-cell server logs (tps, KV/VRAM): bs2:/srv/ml/omk_mrcr_dca/results/<template>/<served>/server.log
• Templates: bs2:/srv/ml/omk_mrcr_dca/omnimergekit/eval/templates/{ruler_native_vt,mrcr}_*_dcaon.yaml
• 3-arm gen-tps: solidpc:backup_models/scripts/dca_gentps_3arm/ (run.log, resp_dca{on,off}.json)
• Coherence/regression gates: solidpc:backup_models/scripts/dca_gate623_mc_coherence.sh, dca_gate_gemma_mc_regress.sh
• NIAH ladder: solidpc:backup_models/scripts/dca_niah_ladder_618.sh → bs2:/srv/ml/dca/dca_niah_ladder_618/
• Patch + RCA: vendors/patches/llamafile/0078-dca.patch; backup_models/.wolf/{buglog.json,cerebrum.md,memory.md}
• DSO under test: solidpc df4c46bca8a4b092 (build #4, qwen fix); bs2 30ed439580de858c (Gemma-validated; byte-identical on the Gemma DKQ=512 path by construction).

8. MTP × DCA composition at 256k (#431/#439)

bs2 GPU1, 2026-06-29/30. Speculative MTP (both engines) composes logit-equivalently with inter-chunk DCA at 256k — the decisive long-context composition cell (full detail in mtp.md §4c/§5):

⚠ The Gemma (#431) row was bug-2118-contaminated — RE-CONFIRMED on the fixed binary (2026-07-10). When this cell was first logged, bug-2118 made --dca on a silent no-op on Gemma, so its real_frac=0 measured DCA-off. Re-run with genuine DCA on the shipped WS2 binary (6468ae37/cb8f942d): real_frac=0.0, frac_full_agree=1.0 → still PASS (§12 Tranche 4c). The Qwen NextN (#439) row is a full-attention path and was never bug-2118-affected.

engine	model	recipe	verdict
Qwen NextN self-spec	Qwen3.6-27B-Omnimerge Q6_K	-c 262144 --dca on --dca-chunk-size 32768 (8 chunks), ref=plain / cand=--spec-type draft-mtp both DCA-on	PASS real_frac=0 (#439)
Gemma assistant-MTP	Gemma-4 A4B-it Q6_K + adapted gemma4-assistant drafter	same recipe, cand=--spec-type draft-assistant -ngld 99	PASS real_frac=0 (#431)

So MTP verify is bit-equivalent to plain decode over DCA-laid-out KV at 256k (the mean_tv_advisory non-zero is the known empty-top-logprobs artifact for spec-accepted tokens; real_frac is the gate). Separately, rolling-KV POSITION_WINDOW × MTP stands on logit-equivalence only — its 256k engagement is architecturally N/A on the current binary (Gemma iSWA window-mode ineligibility, bug-797; mtp.md §5).

9. All-KV-types DCA breadth matrix + q6_0 fix (#444–447)

2026-06-24, bs2 GPU0. Goal: prove DCA works with every K/V type and asymmetric combo, not just the per-type C-series logit-equiv gates. Harness .opencoti/dca/dca-allkv-matrix.sh (Qwen3.6-35B-A3B-redl, IQ3_XXS, 128 head_dim; DCA-on --dca-chunk-size 512 so cross-chunk attention engages; count-up coherence, longest monotone +1 run, FLOOR 250 / 1500 n_predict). Control dca-allkv-control.sh re-runs the non-passing cells DCA-off to separate DCA-path bugs from intrinsic tier quality.

Matrix verdict (26 combos): 18/26 COHERENT. Findings:

class	combos	verdict
scalar sym	f16, bf16, q8_0, q5_1, q5_0, q4_0	✓ COHERENT
scalar asym	q8_0/q5_1, q5_0/q4_0, f16/q4_0, bf16/q8_0	✓ COHERENT
turbo	turbo3, turbo4, turbo8, turbo3_tcq (sym); turbo3/turbo2, q8_0/turbo3 (asym)	✓ COHERENT
q6_0 (any position)	q6_0/q6_0, q8_0/q6_0, q6_0/q4_0	🔴 CRASH → FIXED
2-bit / marginal	turbo2, turbo2_tcq (sym); turbo3/q8_0, turbo3_tcq/turbo2_tcq (asym)	⚠ intrinsic quality, not DCA

q6_0 was the only real DCA defect (bug-694). dca_lift_to_f16() in dca.cpp had a scalar_quant whitelist missing q6_0 (added later, #512), so q6_0 took a direct ggml_cast(q6_0→f16) with no CUDA cpy kernel → CPU dup_from_q abort at ggml-cpu/ops.cpp:594. Fix: add GGML_TYPE_Q6_0 to the whitelist (lifts q6_0→f32→f16 on GPU like q5_0). Host-only rebuild (c081ea54→7c1d4ca8, DSO untouched), captured into 0081 (re-capture proven byte-identical). Re-test on bs2 gpu0: all 5 q6_0 DCA-on combos COHERENT 318 (q6_0/q6_0, q8_0/q6_0, q6_0/q4_0, f16/q6_0, q6_0/f16).

The ⚠ collapses are NOT DCA bugs — the DCA-off control proves it: turbo2 (84 off / 140 on), turbo2_tcq (113 / 68), turbo3/q8_0 (188 / 159) all collapse equally or worse with DCA off → intrinsic 2-bit-tier / marginal-mix lossiness on the count-up task. turbo3_tcq/turbo2_tcq is the one DCA-sensitive case (318 off → 173 on) — a weak 2-bit V pushed over by DCA's harder positional task, a quality limit not a correctness bug.

Net: DCA is correct across the full scalar ladder (f16/bf16/q8_0/q6_0/q5_1/q5_0/q4_0) + turbo3/4/8 + turbo3_tcq, symmetric and asymmetric. The only non-passing cells are 2-bit tiers at their intrinsic quality floor. C-series "DCA all-KV" (#444–447) is empirically complete on a real model.

10. Sparse-V — attention-gated V-skip (#546/#549/#552/#565/#566/#567)

Decode-time optimization: skip the V-contribution of low-attention keys (quantized-V only; f16/bf16 V is a no-op by kernel construction). Two threshold knobs: a static TURBO_SPARSE_V_TAU (skip keys whose softmax weight < τ) and an adaptive mass-budget TURBO_SPARSE_V_EPS (#565, skip while total dropped softmax mass < ε). Headline verdict: the win is iSWA-static-τ=0.05 (concentrated global V-cache → a fat skippable low-weight band); it is shipped self-configuring in the binary (#567 auto-policy, §10e). On full attention the picture splits by model class (§10c): a static τ is a quality cliff on native-1M diffuse-attention models, but a large lossless win on 256k-class merges; adaptive ε is always lossless but yields no full-attn speedup (§10d).

10a. Directional probe — Qwen2.5-14B-1M @24k (3090, 2026-06-28) — not a verdict

Block-skip held at 50% coverage (proven lossless); swept TURBO_SPARSE_V_TAU. RULER niah_single_1, 12 samples, fixed seed 42. tps = max-over-niah-eval-lines (UNRELIABLE).

cell	τ	niah	tps*
dense	—	100	17.97
block-only	—	100	21.50
+V τ=0.1	0.1	0	24.96
+V τ=0.05	0.05	91.67	25.35
+V τ=0.02	0.02	75.0	33.23
+V τ=0.01	0.01	91.67	36.55

*tps non-monotonic (rises as τ falls) → max-over-bursts metric is noise, not skip-tracking. No clean lossless static τ on q8/14B-1M @24k (best niah 91.67): at 24k uniform per-position weight ≈ 4e-5, so the whole 0.01–0.1 τ range sits above the thin-weight bulk and skips nearly all of it (all ≈ equally aggressive, all lossy). Motivates adaptive τ (#565). KEEPER: block-skip alone = lossless q8 win (niah=100, ~1.20× within-boot). Authoritative determination deferred to bs2 production models (§10b/c).

10b. Production iSWA validation — the lossless win (#566, bs2, 2026-06-29)

Binary C-series 511d5305 / 671f6f15, -ctk q8_0 -ctv q8_0 (or -ctv q8_0 over Q6_K weights). niah_single_1 base seed 42; tps = MEDIAN per-sample decode at true ctx depth. Three production iSWA models, all niah 100 at every τ:

Gemma-4-A4B-62e Q6_K — static τ is a lossless decaying win:

ctx	dense tps	τ=0.05 (niah/tps/Δ)	τ=0.10 (niah/tps/Δ)
64k	112.97	100 / 124.56 / +10.3%	100 / 124.76 / +10.4%
128k	91.10	100 / 97.85 / +7.4%	96 / 96.61 / +6.0%
256k	64.08	100 / 67.18 / +4.8%	100 / 67.94 / +6.0%

Official Gemma-4-31B-it Q6_K (not a merge) — confirms the win at production scale:

ctx	dense tps	τ=0.05 (niah/tps/Δ)	τ=0.10 (niah/tps/Δ)
64k	32.47	100 / 33.34 / +2.7%	100 / 33.71 / +3.8%
128k	25.41	100 / 25.79 / +1.5%	100 / 25.74 / +1.3%
256k	17.57	100 / 17.80 / +1.3%	100 / 18.12 / +3.1%

Gemma-4-A4B-98e-v7-coder IQ4_XS (the iSWA PRIMARY target) — lossless but FLAT on this 3090 vehicle:

ctx	KV	V-skip	niah	decode tps	vs dense
128k	f16	off	100	62.37	—
128k	f16	τ=0.05	100	61.97	-0.6%
128k	f16	τ=0.10	100	62.42	+0.1%
256k	q8_0	off	100	25.39	—
256k	q8_0	τ=0.05	100	25.61	+0.9%
256k	f16	off	100	75.85	— (probe at ~60k depth)
256k	f16	τ=0.05	100	75.59	-0.3%
256k	f16	τ=0.10	100	75.41	-0.6%

VERDICT (iSWA): static τ=0.05 is the safe always-on lossless setting. On the bigger weight-bound models (A4B-62e +5–10%, 31B +1–4%) it is a real decode win that decays with ctx (global V-cache fills → smaller relative skip); the win is gentler on heavier per-token compute (31B < A4B-62e). On the f16-V v7-coder the skip is a no-op (f16 V un-skipped by construction) so it is lossless-but-flat — composes safely, no regression. The lone quality nick is A4B-62e 128k τ=0.10 (niah 96) → τ=0.05 is the production knob. (v7-coder probe caveat: the fixed ~60k-token prompt measures decode at ~60k KV depth, not 256k; niah is at true 256k.)

10c. Full-attention behaviour — static-τ cliff vs 256k-class lossless (#566, bs2, 2026-06-29/30)

Full attention splits by model class. Native-1M Qwen2.5-14B-1M Q8_0 — static τ is a quality CLIFF:

ctx	dense	τ=0.05 (niah/tps)	τ=0.10 (niah/tps)
64k	28.45	60 / 41.46 (+46%)	0 / 41.56 (+46%)
128k	18.26	8 / 27.52 (+51%)	0 / 28.0 (+53%)
256k	10.01	0 / 18.85 (+88%)	0 / 16.19 (+62%)

256k-class Qwen3.6-27B-Omnimerge-v4 Q6_K (a sub-1M-native merge) — lossless AND large win:

ctx	dense tps	τ=0.05 (niah/tps/Δ)	τ=0.10 (niah/tps/Δ)
64k	37.48	100 / 45.03 / +20%	100 / 45.01 / +20%
128k	28.17	100 / 37.68 / +34%	100 / 37.56 / +33%
256k	18.08	100 / 26.62 / +47%	100 / 26.54 / +47%

VERDICT: the static-τ cliff is a property of native-1M diffuse-attention models specifically, NOT of full attention in general (a sharper finding than "no fixed τ is lossless on full attention"). The 14B-1M is trained diffuse to the extreme → flat long-ctx weights → the skippable low-weight band includes the needle's contribution, so any τ drops it (and harvests a big +46–88% speed in the process). The omnimerge is a 256k-class merge with sharper long-ctx peaks → the needle's V stays above the τ floor while the diffuse filler below it is safely skipped → +20–47% lossless. Serving rule: 1M-native full-attn → use adaptive ε (#565, lossless-but-flat, §10d); 256k-class full-attn → static τ=0.05–0.10 is a real lossless win. (14B-1M 128k-τ0.10 + the full 256k row filled 2026-06-30 from sparse-v-566/qwen14b1m.tsv.)

10d. Adaptive-τ mass-budget (#565) — lossless but no full-attn speedup (3090)

TURBO_SPARSE_V_EPS: skip key k iff its normalized contribution < ε/n_kv for every query column ⇒ total dropped softmax mass < ε by construction (ε=0 ≡ byte-identical static). Two kernel defects were found and fixed in fattn-vec.cuh (DSO 4326ca7807fe, both paths byte-identical; knobs=0 ⇒ dense output-identical): (1) partial-L inversion — old threshold divided the PARTIAL running softmax-denominator by the TOTAL n_kv, so the per-key bar was too low and skip VANISHED as ctx grew; FIX = divide by keys-actually-processed (true running mean); (2) sink cliff — added TURBO_SPARSE_V_SINK / _RECENT always-keep guards (protect attention-sink + recent window incl. MTP draft tokens; run sink=64 recent=256).

Qwen2.5-14B-1M full attention, q8_0 KV, RULER niah_single_1, MEDIAN decode tps:

ctx	dense	ε0.05+g	ε0.1+g	ε0.2+g	quality
24576	14.36	14.02	13.28	13.65	all niah 100
49152	9.25	9.12	9.16	9.22	all niah 100
65536	7.56	7.29	7.29	7.36	all niah 100

(64k single-ctx control: dense 7.83, static τ=0.05 → niah 33 / 11.91 (+52%, cliff repro); ε0.005–0.10 all niah 100 / ~7.8–7.9 = flat.) FINDING: the combined fix DECISIVELY solves QUALITY (niah 100 in every cell, cliff gone, inversion gone) but full attention yields NO decode win at any ε up to 64k (every cell 0.95–1.00× dense, flat across the whole ε range). ROOT CAUSE: a mean-relative threshold can only skip keys BELOW the running mean; diffuse full-attention weights concentrate NEAR the mean, so the skippable population is tiny regardless of ε — there is no fat sub-mean band to harvest. This is WHY the founding #546 "scales with ctx / more on full attn" assumption inverted: the win needs CONCENTRATED attention (iSWA's small global V-cache), which full attention by construction lacks. NET: adaptive ε is the SAFE full-attention default (never slower within noise, never lossy) but not a win there; the iSWA static-τ=0.05 win stands (§10b). Full-attention long-ctx speed is a different-mechanism problem (§11), not a V-skip threshold problem.

10e. In-binary auto-policy (#567, patch 0088, host-only, 2026-06-29)

The §10b sparse-V win is now self-configuring so a STANDALONE llamafile --server gets it with no flags (intelligence in the binary, NOT the opencoti TS adapter — a standalone CLI user bypasses the adapter). At llama_context construction: if the user has not set TURBO_SPARSE_V_TAU AND the model is iSWA (hparams.swa_type != NONE) AND V is quantized (type_v not f16/bf16/f32), setenv(TURBO_SPARSE_V_TAU=0.05, overwrite=0). The kernel reads that env (ggml-cuda is a runtime DSO → env is the cross-DSO channel) → host-only, DSO byte-unchanged. Full-attention models leave it unset → dense byte-identical. Announced once at WARN (model-load INFO is suppressed). GATE (3090, binary 5bf9fac6): decision table 4/4 PASS — A4B q8_0 auto=ENGAGED; A4B q8_0 +TURBO_SPARSE_V_TAU=0 override=off; A4B -ctv f16=off (kernel guard inert); Qwen-14B-1M q8_0 full-attn=off (no iSWA). A/B @ 24k (A4B iSWA q8_0, 24 samples): dense niah 100 / 65.48 tps vs auto niah 100 / 65.30 tps → LOSSLESS + non-slower (the iSWA win is tiny at 24k, grows with ctx — the 50-sample production runs in §10b carry the measurable win). The ε/sink/recent knobs remain available as explicit overrides for full-attention experiments (no auto-default there — no lossless win).

10f. MTP composition (#549/#552) — parallel, turbo3_tcq, Qwen-NextN

Sparse-V composes with speculative MTP across all tested KV tiers and engines; the old "disable sparse-V under MTP (kills draft acceptance)" guard is RETIRED (a pre-combined-kernel artifact).

A4B iSWA q8_0 × assistant-MTP at --parallel ≥2 (3090, DSO 4326ca78, draft acceptance = server-log real_frac). Single-prompt per-slot: p1_off 0.805/114tps, p1_sv(τ0.05) 0.770/110.7; p2_off 0.832+0.796 / 52.6+51.8tps, p2_sv 0.847+0.837 / 53.3+52.8, p2_svguard(sink64 recent256) 0.847+0.819 / 54.3+54.6. All 6 per-slot samples in a 0.80–0.85 band (noise); sparse-V tps if anything slightly HIGHER. Multi-prompt (4 diverse coding prompts, parallel 2, mean accept): off 0.787, sv 0.819, svguard 0.819 → sv/svguard ≥ off, per-prompt variance 0.74–0.89 dwarfs any sparse-V effect. (Harness gotchas, both in buglog: Gemma assistant-MTP at --parallel>1 hard-requires --kv-unified; wait $curlpid only, not bare wait.)

The two untested cells (#552, closed) — combined kernel × MTP, greedy chat, single-stream, 3090. CELL A (turbo3_tcq KV tier): A4B-v6 iSWA -ctk/-ctv turbo3_tcq + 26B-assistant MTP — A_off 0.7587 vs A_sv (auto-engaged: WARN confirms "iSWA + quantized V (turbo3_tcq) → 0.05") 0.7587 = BYTE-IDENTICAL acceptance; turbo3_tcq booted clean (no bug-631 garbage, 0.76 ≈ q8_0's 0.79). CELL B (Qwen NextN engine): Qwen3.5-9B --spec-type draft-mtp q8_0 — B_off 0.6287 vs B_svF (TURBO_SPARSE_V_TAU=0.05 FORCED; auto leaves full-attn OFF) 0.6287 = BYTE-IDENTICAL (at 8k the full-attn skip is near-inert, but no mechanical breakage of the NextN verify path). VERDICT: sparse-V composes with MTP across all KV tiers/engines — q8_0×assistant (#549), turbo3_tcq×assistant (A), Qwen-NextN-self-spec (B). Byte-identical acceptance = lossless-at-greedy. Production: auto-policy enables sparse-V only on iSWA+quant-V (incl. turbo3_tcq), correctly OFF on Qwen full-attn — MTP and sparse-V are orthogonal there.

11. Sparse attention — block-selector / vertical-slash (#551, CLOSED) + #548 scout

The open problem this line chased: lossless-AND-faster decode at 256k/1M on full-attention models (where §10 sparse-V has no win). Verdict (2026-06-29): the block-selector / vertical-slash line is DEAD across all five mechanisms tried. The shipped long-ctx win remains §10 sparse-V on iSWA.

11a. Quest block-skip @256k — fails the win bar (bs2, 2026-06-29, PIVOTAL NEGATIVE)

Front A validated the proven 24k Quest block-skip win at real serving ctx (Qwen2.5-14B-1M Q8_0, q8_0 KV, RULER niah_single_1 @262144, bs2 GPU1, binary 671f6f15 + sm120 DSO):

cell @256k	niah	decode tps	note
dense	100.0	13.43	baseline, n=4
quest_65	75.0	13.04	topk 2662/4096, recent 2048, sink 1 → LOSSY + NO win

The 24k bug-747 result (q8 niah 100 lossless @65%, 1.34×@50%) does NOT extrapolate to 256k: (1) niah collapses 100→75 at the SAME 65% coverage — Quest's geometric min/max kbounds score is an extreme-value statistic, so as n_block grows 24k→256k (384→4096 blocks) the needle's lone constant-signal key is buried by high-spread distractor blocks (SNR ~ 1/n_block); (2) no decode win (13.04 < 13.43, ~3% slower) — at 256k the per-step selector overhead (kbounds fill + top-k over 4096 blocks EVERY decode step) cancels the block-skip KV-read savings even though decode IS KV-bandwidth-bound there. Quest is a 24k-LOCAL win, fails "must be a win even at 256k" on both axes.

11b. Decode-probe correction + bug-761 refresh trap

The flat ~13 tps above was a per-step SELECT-KERNEL artifact, not a property of block-skip. Decode-tps probe (same model, ~231k prompt, 256-tok decode, bs2 GPU1): dense 14.05 (1.00×); q65_r1 (refresh 1, per-step select) 12.84 (0.91× — select tax > skip saving); q65_r256 (refresh 256, amortized select) 26.87 (1.91×). So block-skip at ~50–65% coverage nearly DOUBLES 256k decode throughput once the kbounds-fill+top-k-select over 4096 blocks is amortized off the per-step path — the speed half of the bar IS clearable. BUT bug-761: --sparse-attn-refresh 256 freezes the first decode token's pick (which usually doesn't attend the needle) → niah 0. Mechanism (code read): Quest (mode 0) selects at DECODE-time per token; refresh N trades correctness for speed. The correct per-step config (q65_r1) is 12.84 < dense → Quest cannot be both fast and lossless at 256k. The remaining gap is SELECTION QUALITY: refresh changes WHEN select runs, not WHAT it picks.

11c. Line closed — five mechanisms exhausted + #548 scout-negative

The revival thesis (vslash content-selection with a wider question-window would find the needle where Quest's geometric kbounds fails) was REFUTED on retest: restored q_win 8→64 (llama-graph.cpp:2871+3250), host-rebuilt (sha 32460c35), bs2 → vslash@24k f16 niah STILL 0 (zero movement). q_win is not the lever; the real wall (buglog #563) is that the prefill question-window is a WORSE needle probe than the decode answer-query (the decode query generates the needle; the prefill question doesn't reliably rank the needle block) + recent-blocks frozen at prefill. A free MMA side-output would make prefill scoring free but inherits the bad prefill-probe ranking → cannot fix selection correctness. Five mechanisms now dead (Quest-geometric / max-logit / softmax-decode-q1 / prefill-window / q_win-restored); the only ever-lossless variant (softmax-decode q_win=1) is 2.3× SLOWER + bimodal@40k. No lossless-AND-faster block-selector exists. Tasks #568–572 (S2–S5 side-output) SHELVED.

#548 importance-tiered bits SCOUT (theTom turboquant_plus layer-aware-v-compression.md): LA-V7 = V-only layer-aware (boundary L0,1,N-2,N-1 → q8_0-V; middle → turbo2-V; K=q8_0 all layers). theTom's OWN numbers: advantage over uniform turbo2-V ≈ ZERO by 16k (-0.006 PPL @16k phi-4), "no speed penalty" → NOT a 256k win; dilutes with context, same per-token needle-ranking wall. theTom's central finding "ALL quality loss is from K, not V" implies the only win-shaped salvage is layer-aware on K (unexplored bet) — DEFERRED by user choice. OPEN gap: lossless+faster 256k decode on full-attention 1M-native models has no proven lever; sparse-attention exploration is CLOSED.

12. #626 redo — Gemma×DCA evals on the FIXED binary (post bug-2118/2119/2120/2121)

Why (2026-07-06): bug-2118 made --dca on a silent no-op on ALL Gemma-4 archs, so every historical Gemma×DCA number in this file (and mtp.md §#431/#441, the June MRCR A4B on/off pair, the 256k/512k 3-model anchors) actually measured DCA-OFF. Fixing 2118 surfaced two criticals: bug-2120 (dual-ctx drafter accept=0 with any quant target KV — attention-rotation not inherited by the shared draft cache) and bug-2121 (deep-spill streaming-FA prefill corruption — CUDA-pool staging scratch aliased across streams). All four are fixed and gated; patches 0099–0102. Binary = llamafile.dualctx sha 91a017c8, DSO sha 8b23afbf. Historical Gemma×DCA rows above are STALE for DCA-on claims; this chapter is the authoritative replacement as cells complete.

Tranche 1 — 256k 3-model anchors (bs2 GPU0, 2026-07-06, first VALID Gemma DCA+MTP anchors)

212,327-token prompt, ctx 262144, q8_0-K/q4_0-V, window residency (all fully resident), greedy needle + tps probe (bs2-3model-anchors.sh). Gemma cells run --dca on --dca-chunk-size 32768

• dual-ctx assistant drafter; Qwen cell is native NextN (no DCA).

cell	prefill t/s	decode t/s	draft accept	needle
A4B Q4_K_M + assistant-Q8_0, DCA on	5200.2	73.5	0.700 (42/60)	PRESENT
31B Q6_K + assistant-Q8_0, DCA on	1148.9	26.9	0.456 (36/79)	PRESENT
Qwen3.6-35B UD-Q6_K NextN (native, no DCA)	2811.8	109.7	0.606 (40/66)	PRESENT

Decode-tps re-run — 2026-07-12, patch 0119 (P0 DCA-on scalar in-register), bs2 gate binary 7abd6840. The 2026-07-06 decode column above used the pre-0119 whole-cache to_fp16_nc f16-lift and is stale-LOW; on the current in-register DCA-on decode path the same cells (warmup + 256-tok cache-reused measure, all needle=PRESENT) are: A4B 73.5→114.75 (+56%), 31B 26.9→33.34 (+24%), Qwen-35B NextN 109.7→151.53 (+38%). Accept/needle unchanged (lossless — those columns stand). The Qwen measured call reports prompt_n=516 (a self-spec draft-mtp cache-reeval quirk); needle=PRESENT confirms the full 256k context is resident.

Reading: the previously-impossible composition (Gemma + DCA-actually-on + quant KV + dual-ctx drafter) now works end-to-end at 256k — accept 0.700 on A4B where the pre-fix binary scored 0.000 (bug-2120) on a config whose DCA flag did nothing (bug-2118). 31B accept 0.456 matches its historically weaker adapted drafter, not a defect. 512k anchors + §5b niah matrix Gemma cells + MRCR A4B on/off queued next.

Losslessness gate (bug-2120 fix, 2026-07-06, bs2 A4B + assistant, 4k, n_max=1): f16 — plain vs dual-ctx spec token-IDENTICAL (40/40), spec deterministic. k8_v4 — spec deterministic (specA==specB 40/40), accept 0.90, plain-vs-spec diverges at token 32: benign quant-KV batch-vs-sequential greedy numerics (same class as the bug-858 upstream profile; the f16 control run the same day is clean, so the spec path itself is lossless).

Tranche 2 — 512k 3-model anchors (bs2 GPU0, 2026-07-06)

Same harness at ctx 524288, 460k-token prompt (~424,631 tokens), q8_0-K/q4_0-V, DCA --dca on --dca-chunk-size 32768, dual-ctx assistant drafter. (The anchors script's inline curl truncated the A4B response on port-reap → false ABSENT; a clean re-probe with -m 3000 returned http 200, 3.0 MB body, needle PRESENT — recorded below.)

cell	prefill t/s	decode t/s	draft accept	needle
A4B Q4_K_M + assistant-Q8_0, DCA on	3211.4	42.3	0.651 (41/63)	PRESENT (re-probe)
31B Q6_K + assistant-Q8_0, DCA on	643.8	14.3	0.326 (31/95)	PRESENT

Reading: A4B holds needle + 0.65 accept at 512k (2× native ctx via DCA), decode 42 t/s. 31B accept softens to 0.326 at 512k (its adapted drafter degrades with chunked-DCA depth) but retrieval is intact. Both prove the fixed binary composes DCA + dual-ctx MTP + quant KV correctly out to 512k — the regime that was 100% broken pre-fix (bug-2120 accept 0.000 or bug-2121 corruption).

Tranche 3 — §5b DCA niah matrix redo (bs2, in progress) + a NEW crash it surfaced

Re-running the #575 Gemma-4-A4B-v6-coder niah_single_1 matrix (KV tiers × {256k,512k}, --dca on --dca-chunk-size 32768, ns=5) on the fixed binary. This is the first time these cells have genuinely run DCA — pre-fix (bug-2118) the flag was a silent no-op, so the historical "all 100.0" (§5b, #575) was DCA-off. Turning DCA genuinely on splits the tiers into a clean-pass group and a crash cluster:

tier (ctk/ctv)	ctx	niah	note
f16 / f16	256k	100.0	scalar tier — passes true DCA-on
q8_0 / q8_0	256k	100.0	scalar
q4_0 / q4_0	256k	100.0	scalar
q8_0 / q6_0 (q8q6asym)	256k	100.0	asym scalar (q6_0 only as V)
q8_0 / q4_0 (q8q4asym)	256k	100.0	production tier — matches the anchors
q6_0 / q6_0	256k	CRASH	bug-2125
turbo3_tcq / turbo3_tcq	256k	100.0	trellis-TCQ passes true DCA-on
turbo2_tcq / turbo2_tcq	256k	CRASH	bug-2125
turbo3_tcq K / turbo2_tcq V	256k	CRASH	bug-2125 (the turbo2_tcq-V)
f16 / f16	512k	100.0	clean re-run — the earlier 40 was contamination; no DCA-depth cost at 4× native
q8_0 / q8_0	512k	100.0	scalar — matches 256k
q4_0 / q4_0	512k	100.0	scalar — matches 256k
q8_0 / q6_0 (q8q6asym)	512k	100.0	asym scalar — matches 256k
q8_0 / q4_0 (q8q4asym)	512k	100.0	production tier — holds at 4× native
turbo3_tcq / turbo3_tcq	512k	CRASH	bug-2125 — crashes at 512k though it PASSES at 256k (chunk-count dependent)
q6_0, turbo2_tcq, t3kt2vasym	512k	CRASH	bug-2125 — same crashers as 256k

Method note: the first pass was contaminated by two runners colliding on the shared server port (duplicate rows, one spurious q6_0=100). The table above is from the clean single-runner authoritative sweep (s5b-clean.sh, one runner, liveness-checked boot, port reaped between cells) — COMPLETE (all 9 512k cells landed 2026-07-07): the crashes are confirmed real (not port-collision), f16-512k is 100.0, and every scalar/asym tier (f16, q8_0, q4_0, q8q6asym, q8q4asym) holds 100.0 at 512k while the q6_0/turbo2_tcq/turbo3_tcq crash cluster is bug-2125.

bug-2125 (NEW, exposed by the bug-2118 fix) — DCA + q6_0/turbo2_tcq KV-init segfault (task #627, being fixed — NOT deferred). Under real DCA, a subset of quantized tiers segfault during context/KV-cache init (past model load + sparse-v auto-policy, at the window-tail host-buffer setup), on llamafile.dualctx 91a017c8/8b23afbf, both -fit on and -fit off, at both 256k and 512k: q6_0-symmetric, turbo2_tcq-symmetric, and the turbo3_tcq-K / turbo2_tcq-V asym pairing all crash. The discriminator is neither axis nor bit-count: turbo3_tcq-symmetric passes 100.0 (3.25 bpv trellis on both K and V), and the crashing t3kt2vasym carries a safe turbo3_tcq-K — so its crash is the turbo2_tcq-V, proving V can trip it too. Nor is it "low bit": q6_0 is a 6-bit scalar, hardly low-precision, yet it crashes while the 3.25-bpv turbo3_tcq passes. The clean split is by specific type: q6_0 and turbo2_tcq crash (on either K or V); q6_0/turbo2_tcq-free tiers pass 100.0 at 256k. That q6_0 — a high-precision scalar — segfaults marks this as a genuine code defect, not a precision floor. Crucially, the crash set GROWS with DCA chunk count: turbo3_tcq passes 100.0 at 256k (8 chunks) but CRASHES at 512k (16 chunks), while f16 stays clean 100.0 at both. So the trigger is not a fixed per-type property — it scales with the number of chunks, strongly implicating a window-tail host-buffer whose size is mis-computed as a function of n_chunks (a block-size / row-alignment / dequant-on-lift assumption in the DCA chunked-rope window-tail setup that q6_0 and turbo2_tcq block layouts violate immediately, and that turbo3_tcq only overruns once the chunk count is high enough). None of these ever ran before (the bug-2118 contamination hid the path). It does not affect the production tier (q8_0-K/q4_0-V — clean at 512k) or any shipped anchor, but it is a real latent segfault being root-caused + fixed under task #627 (gdb/ASAN for the exact fault site). Historical §9/§5b turbo×DCA and q6_0×DCA cells are invalid for the same bug-2118 reason and are superseded by this chapter.

The f16-512k number is resolved: 100.0 (clean single-runner). The earlier 40.0 was the two-runner contamination artifact — there is no DCA-depth retrieval cost for f16 at 4× native (16 chunks). The remaining 512k scalar cells landed 2026-07-07 all at 100.0 (q8_0, q4_0, q8q6asym, q8q4asym) — the production tier q8_0-K/q4_0-V holds at 4× native. The stale note below is retained only for history:

⚠ (superseded) The f16-512k = 40.0 reading (vs 100 at 256k) is from the possibly-contaminated first pass and is being re-measured clean; if it holds, it is a genuine DCA-depth retrieval cost on this model at 4× native context (16 chunks), not a bug — but treat it as provisional until the clean sweep confirms.

Tranche 3b — bug-2125 crash cluster RESOLVED + one NEW asym bug (bug-2141), re-measured on the WS2-shipped binary (2026-07-10)

Re-ran the Tranche-3 crash cluster on the current shipped WS2 binary — commit 6468ae37 / dualctx cd65e45f / DSO cb8f942d (adds the bug-2125 fix cbe3cf5b5 #627 + WS1 bug-2127 DCA per-tensor defer + WS2 in-register turbo-V DEFAULT-ON on top of the Tranche-3 B1 binary 91a017c8/8b23afbf). Single-runner s626-crashcells.sh, GPU0, --dca on --dca-chunk-size 32768, Gemma-4-A4B-98e-v6-coder Q4_K_M, niah_single_1 ns=5. The entire bug-2125 crash cluster is CLOSED — every previously-crashing symmetric/scalar tier now boots and retrieves 100.0:

tier (ctk/ctv)	ctx	B1 (8b23afbf)	B3 (cb8f942d)	note
q6_0 / q6_0	256k	CRASH	100.0	bug-2125 FIXED
turbo2_tcq / turbo2_tcq	256k	CRASH	100.0	bug-2125 FIXED
turbo3_tcq / turbo3_tcq	256k	100.0	100.0	lossless-invariant (sanity)
turbo3_tcq K / turbo2_tcq V (asym)	256k	CRASH	20.0 ⚠	now RUNS but broken forward — bug-2141
q6_0 / q6_0	512k	CRASH	100.0	bug-2125 FIXED
turbo2_tcq / turbo2_tcq	512k	CRASH	100.0	bug-2125 FIXED
turbo3_tcq / turbo3_tcq	512k	CRASH	100.0	bug-2125 FIXED (the chunk-count-dependent crash is gone)
turbo3_tcq K / turbo2_tcq V (asym)	512k	CRASH	100.0	passes at the 16-chunk geometry (see bug-2141)

(The scalar / asym-scalar tiers f16, q8_0, q4_0, q8q6asym, q8q4asym scored 100.0 on B1 and are lossless-invariant under bug-2125/2127/WS2 — not re-run; they stand at 100.0 at both ctx. All symmetric turbo/TCQ tiers now hold retrieval under genuine DCA out to 4× native.)

Method note — re-measured on the WS2-shipped binary 6468ae37 / DSO cb8f942d. Correctness via RULER niah (never greedy); DSO deployed byte-identical at both bs2 side-load paths.

NEW: bug-2141 — asymmetric turbo3_tcq-K / turbo2_tcq-V is broken-forward at 256k (niah 20, 4/5 EMPTY generation) but passes at 512k (100). This is NOT tier quality: symmetric turbo2_tcq (2-bit K+V) = 100 and symmetric turbo3_tcq = 100 at both ctx, so every component of the asym cell already appears in a passing symmetric config — its K is strictly higher-bit than the passing sym-t2 K and its V is identical to the passing sym-t2 V. The only new variable is the K≠V mismatch, and the failure mode is empty generation (broken forward / NaN logits), not wrong-answer weak retrieval. Root cause CONFIRMED (static trace + GPU discriminator): under DCA-on decode dca_fused hardcodes type_K=F16 (K is f16-lifted) and dispatches the fused instance on V-type, so the asym cell is the only config pairing a turbo3-derived K_f16 with the turbo2_tcq-V read in-register (WS2 default-on) in the D512 fused kernel — a composition no symmetric control exercises. A DCA band-boundary phase whose mask is all--inf yields a zero exp-sum → 0/0 in the online-softmax normalization → NaN, which wht_o's 128-pt butterfly de-rotation (dca.cpp:396) smears across all 512 output lanes → empty generation. It is geometry-dependent (256k/8-chunk trips the empty-band phase; 512k/16-chunk does not) and WS2-specific (V f16-lift scored 60, functional; the WS2 in-register flip regressed it to 20). Decisive discriminator (bs2 GPU0, same binary): asym @256k DCA-on with WS2_DCA_TURBO_V=0 → niah 100.0 / 0 empty (vs WS2 default-on 20.0 / 4 empty) — forcing V back to the f16-lift path (same K-lift) fully recovers, proving the fault is the WS2 in-register-V + wht_o composition, not the K-lift or any codebook. (All per-tensor type/codebook selection was verified correct by static trace; the stringly-typed cache_k_/cache_v_ split gates only a 4% norm-alpha and cannot NaN.) Escape hatch that works today: WS2_DCA_TURBO_V=0 for asym turbo_tcq K≠V. Fix directions: (1) NaN-safe the online-softmax normalization for all--inf DCA band phases before wht_o (principled; keeps the WS2 win), or (2) gate WS2 in-register-V off for asym turbo_tcq K≠V. Quarantine both asym-turbo_tcq directions pending the fix; the production tier q8_0-K/q4_0-V is unaffected (clean 100.0 at 256k and 512k). Full RCA in .wolf/buglog.json bug-2141.

Direction-specific (not symmetric). The reverse asym turbo2_tcq-K / turbo3_tcq-V @256k scored 100.0 / 0-empty — clean. So the break is confined to the turbo3_tcq-K + turbo2_tcq-V ordering at the 256k/8-chunk geometry: it needs the higher-precision turbo3-K_f16 (sharper KQ logits, which is what drives a band phase to the all--inf boundary) paired with the turbo2-V-in-register instance. The reverse ordering (turbo2-K_f16 + turbo3-V-in-register) does not produce the boundary condition, and both symmetric configs are clean — consistent with the uniquely-uncovered-composition thesis. Confirmed cells (256k, DCA-on, ns=5): t3K/t2V = 20 (4/5 empty), t3K/t2V WS2_DCA_TURBO_V=0 = 100, t2K/t3V = 100, sym-t2 = 100, sym-t3 = 100. Recommendation: prefer the escape hatch or avoid the t3K/t2V pairing under DCA until the NaN-safe fix lands; production tier q8_0-K/q4_0-V is unaffected.

bug-2121 formal correctness gate (14B-1M q8K/q4V, window-mode host tail)

Beyond the greedy-needle deep-spill gate (tails 1600/3200/6400 all PRESENT, zero crashes), a semantic niah_single_1 gate (ns=5) on Qwen2.5-14B-1M q8_0-K/q4_0-V compares window mode with a forced multi-tile host tail (≥2 tail tiles) against the fully-resident control:

config	niah	note
resident (control)	100.0	reference — all 5 samples finish=stop, correct needle
window, deep tail (256/80128 resident ≈ 2.44 tail tiles), max_tokens=24	100.0	clean pass — genuine ≥2-tile spill, needle retrieved every sample

Verdict: the clean ns=5 gate PASSES — #625 CLOSED. The earlier "tail 3200 = 40.0" reading was a harness artifact, not a defect. The original gate used max_tokens=120, so some tail-region needles rambled to the token cap (finish=length) instead of retrieving and stopping — 2/5 correct, 2/5 ramble, 1/5 truncate = 40 — a scoring confound, not the pre-fix corruption signature (no GPFs, no garbage, no crash; decode was normal deep-spill ~0.6–1.6 tps, and an earlier "159–400 s/token" reading was just a misread of gen_secs, which includes the ~1307 s prefill). Re-running the same deep-spill config with max_tokens=24 (so a correct-but-verbose sample can't be mis-scored) and a forced 2.44-tile host tail — deeper than the original — the window path scores 100.0, identical to the fully-resident control.

So the bug-2121 window path is clean at genuine multi-tile deep spill, corroborating the single-forward basis that already validated the fix: the greedy-needle deep-spill gate (tails 1600/3200/6400 all PRESENT, zero crashes/GPFs), teacher-forced logit-equiv real_frac=0, and the 35B NextN tail-collateral recovery (0.449→0.625, below). The prefill-corruption defect bug-2121 targeted is gone by every measure — single-forward and now a clean multi-request niah.

Collateral: 35B NextN tail=200 accept recovery (confirms bug-2121 reach)

The pre-fix 35B-A3B NextN spill-decode anomaly — tail=200 accept dropping to 0.449 while tail=0 held 0.606 — recovered to 0.625 on the fixed DSO (8b23afbf) with the tail=0 control unchanged at 0.606 (same calibration). So the cross-stream pool aliasing (bug-2121) was also silently corrupting that draft path; the fix repairs it. (buglog bug-2121 collateral.)

Tranche 4 — anchor + MTP×DCA re-validation on the WS2-shipped binary (2026-07-10)

Re-ran the Tranche-1/2 anchors and the §8 Gemma MTP×DCA logit-equivalence gate on the current shipped binary — commit 6468ae37 / dualctx cd65e45f / DSO cb8f942d (bug-2125 fix

• WS1 bug-2127 + WS2 default-on), GPU0 (tier23-master.sh), --dca on --dca-chunk-size 32768.

(a) Correctness re-validated — accept + needle are byte-stable B1 → B3. Every anchor's draft acceptance and needle match the Tranche-1/2 (B1 8b23afbf) values exactly, on the genuinely-DCA-on WS2 binary:

cell	ctx	accept B1 → B3	needle	prefill tok/s B1 → B3
A4B Q4_K_M + assistant-Q8_0, DCA on	256k	0.700 → 0.700 (42/60)	PRESENT	5200 → 5188 (parity)
31B Q6_K + assistant-Q8_0, DCA on	256k	0.456 → 0.4557 (36/79)	PRESENT	1149 → 1147 (parity)
Qwen3.6-35B UD-Q6_K NextN (no DCA)	256k	0.606 → 0.606 (40/66)	PRESENT	2812 → 2428 ⚠ (contention)
A4B Q4_K_M + assistant-Q8_0, DCA on	512k	0.651 → 0.651 (41/63)	PRESENT	3211 → 3208 (parity)
31B Q6_K + assistant-Q8_0, DCA on	512k	0.326 → 0.326 (31/95)	PRESENT	644 → 642 (parity)

So MTP×DCA + dual-ctx drafters compose correctly and byte-stably on the shipped WS2 binary; the A4B/31B prefill tok/s are at parity with B1 (no prefill regression).

(b) Decode tok/s — DEFERRED, cross-GPU contention (not recorded as corrected values). These anchors ran while GPU1 was committed to task #631's spill sweep, and the cerebrum rule holds: tps cannot be trusted with the sibling GPU busy (PCIe/host-BW contention). The observed B3 decode numbers came in uniformly ~15–30 % below B1 across all cells — including the no-DCA Qwen NextN control (a change in a non-DCA path that no WS2/bug-2127 edit touches), the tell-tale signature of contention, not a code change. (Prefill, being compute-bound, held parity; decode, the PCIe-sensitive phase, is the one that sagged.) The manifest's "bug-2127 ~2× decode understatement" premise does not apply to these cells anyway — bug-2127's ~2× win is the asym scalar-K + turbo-V decode path; these anchors are q8_0-K/q4_0-V (scalar V), so neither bug-2127 nor WS2-in-register-turbo-V engages. A clean DCA-on decode-tps re-measure is deferred to when GPU1 (#631) is idle and should be run serially with the sibling GPU idle. (The DCA-on turbo/TCQ-V decode win that WS2 does deliver is already the §5x matrix.)

(c) §8 Gemma assistant-MTP × DCA @256k real_frac — RE-CONFIRMED (bug-2118 fix). The #431 logit-equivalence gate (ref = plain DCA-on, cand = --spec-type draft-assistant DCA-on, both --dca-chunk-size 32768, google 26B-A4B-128e Q4_K_M + drafters-hf assistant Q8_0, teacher-forced top-20) now runs with genuine DCA (bug-2118 made it a no-op when §8 was first logged): real_frac = 0.0, real_disagreements = 0, frac_full_agree = 1.0 → PASS. So Gemma assistant-MTP verify is bit-equivalent to plain decode over DCA-laid-out KV at 256k on the shipped binary — the §8 claim stands, now on real DCA. (The runner's mean_tv_advisory ≈ 0.44 "VERDICT: FAIL" is the documented empty-top-logprobs artifact for spec-accepted tokens — real_frac is the gate, and the runner emitted W4_GEMMA_DCA256K_PASS.) Harness tier3-mtp-dca-b3.sh.

GPU0 CLEAR after this tranche (servers reaped by port); decode-tps re-measure pending GPU1 idle.

Tranche 5 — #674: DCA-OFF scalar-quant MMA decode generalized to Gemma D256/D512 (bs2 GPU1, 2026-07-14)

Folds the #674 result into this campaign. This is not a DCA cell — it's the DCA-off high-VRAM serving path (scalar-quant KV over the tensor-core MMA decode kernel, patch 0127, chain→73), the D256(SWA-local)/D512(global) generalization of the #620 D128 lever that §3b (Tier-1) folded for Qwen. Gemma-4 A4B native (google 26B-A4B-128e Q4_K_M), DCA-off, decode-tps + logit-equiv vs f16 (Gemma PPL is garbage → correctness is top-token identity, real_frac):

ctx	nkv	f16	q4q4	q8q8	q8q4	q8q4 vs f16	env=0 VEC	real_frac
64k	42.5k	162.13	170.17	173.86	174.94	+7.9%	122.17 (MMA +43% over VEC)	0.0
128k	112k	152.15	152.95	—	154.46	+1.5%	(§3c had q8q4-VEC −35% here)	0.0

Verdict: scalar-quant KV now beats f16 and is top-token-lossless at both depths on Gemma's D256+D512 — so with enough VRAM it is the preferred serving path over turbo (turbo is slower AND lower quality). The §3c/§3d kernel-bound quant tax (−28…−35% at long ctx) is erased; the win narrows deep (MoE-FFN + per-step launch dilute the KV-bandwidth delta) but stays ≥ f16. env OPENCOTI_QUANT_MMA_DECODE=0 reverts each pair to its native FA-VEC route at every head-dim (graceful). Build note: the 20 D512-heavy instances push ggml-cuda.so to 4.43 GiB → bug-2178 (streamed sha256OfFile). Tax-erasure detail in hal_kv_compression.md §3c. 256k cell intentionally skipped — the curve is monotone toward the f16 floor (parity), not a cliff, so it adds no serving-guidance signal.

Tranche 6 — Tier-1 spill + Tier-2 short cells on the fixed binary (bs2, 2026-07-12)

The remaining #654 re-run cells (binary 7abd684039a3 / DSO 1d8fe395c53a, greedy, cache_prompt=false, n_predict 256–300, --kv-residency-mode window --vram-target 90000). All decode-tps single-GPU (no sibling contention this batch). Closes the campaign apart from two harness slips noted below.

Tier-1 — Gemma/Qwen spill decode @256k/512k (q8_0-K/q4_0-V, needle probe):

model	ctx	decode tps	needle
A4B (google 26B-128e)	256k	114.75	PRESENT
A4B	512k	68.00	PRESENT
Gemma-31B	256k	33.34	PRESENT
Gemma-31B	512k	23.24	PRESENT
Qwen3.6-35B-A3B NextN	256k	151.53	PRESENT

Tier-2 short cells:

• §3.0a — A4B MTP×KV-tier decode (turbo/TCQ), f16 anchor (decode tps, off → on / accept): f16 200.4 → 365.8 (0.984); turbo3 153.1 → 263.8 (0.984); turbo3_tcq 104.7 → 214.7 (0.990). MTP self-spec ~1.4–1.8× decode across every KV tier; turbo/TCQ stay accept-lossless (≥0.98).
• §2a — 3-arm DCA on/off prefill×gen (A4B / Gemma-31B, prefill → gen tps): A4B off 6412 → 156.7, on 7708 → 142.6; Gemma-31B off 2017 → 36.5, on 2121 → 35.7. DCA prefill even edges up on A4B (chunked attn), decode within noise — no DCA tax.
• §4c — NIAH q8_0/q8_0 16k/32k/64k/128k all 100.0 (n=5). real_frac gate (assistant-MTP vs plain, DCA-off): real_frac=0.0, frac_full_agree=1.0 → PASS (the runner "FAIL" is the documented empty-top artifact; mean_tv_advisory is advisory only).

Two harness slips (not engine regressions):

1. short4 fused-nextn ours BOOT-FAIL → FIXED + re-run (2026-07-14). The run() boot line omitted --server, so ours (llamafile) came up in CLI chat mode and never answered /health; b9859 (upstream llama-server) is a server by default so it booted (bug-2179). Fixed (conditional --server for ours) and re-run on GPU1: ours base 208.3 (≈ b9859 212.0, parity), n1 226.4/acc 0.892, n2 230.4/0.739, n3 240.1/0.638 — NextN decode trails b9859 by ~12–15 % (the unchanged bug-858 verify/CUDA-graph gap, accept at/above upstream at n1). Full table in fused_nextn_mtp.md.
2. tier1-gpuB t2 @512k Qwen-NextN parse-fail — server booted fine (KV fully resident) but the warmup prompt (435,574 tok) exceeded the available per-request context (262,144) → send_error, which the harness' JSON parse choked on. This is per-sequence context halving under the dual-context draft-mtp path at 512k (ctx_tgt+ctx_dft split), not a crash; the t1 @256k cell is clean (151.53, PRESENT). The Tranche-2 512k Qwen-NextN cell is optional — re-run needs a prompt sized to n_ctx/2, or -c 1M.