# MTP / speculative decoding — evaluation

> Status (2026-07-14): both flavours shipped, characterized, and **at throughput parity-or-better
> vs upstream b9859**. Gemma assistant-MTP runs upstream's dual-context execution (**§3e** —
> bug-858's two root causes found+fixed). #609 depth-3 CLOSED (bug-2103 TinyBLAS f32 mmvf fix).
> **#675 (§3h, patch `0128`) closed the last per-cycle host gap** (bug-2180 `common_sampler_clone`
> ~1.8 MiB deep-copy under cosmo dlmalloc = 1.07 ms/cycle + mmvf f16/bf16 verify lift): Qwen 35B
> NextN −12% → **−1.7%..+3%**, and **A4B assistant now BEATS b9859 at every depth** (9-prompt
> decode mean +6.6/+12.9/+7.6% at n1/n2/n3), accept bit-identical (pure-latency fix). Residuals:
> accept deficit at depth ≥2 (#495-class chained-draft numerics, −4.4pp aggregate at n3) and
> 31B/27B −2..−6% decode. Recommend n2–n3 on A4B, n3 on Qwen NextN. Current cross-engine numbers
> incl. the 9-prompt per-prompt table: [`three-way-tps.md`](three-way-tps.md) §2026-07-14.

This document is the rigorous record for **Multi-Token Prediction (MTP) speculative decoding** in
the opencoti llamafile fork, across **both** flavours and **all** supported architectures. It is the
MTP-specific companion to [context.md](context.md) (RULER retrieval / DCA long-context) — where
context.md proves *quality at length*, this proves *speed without quality loss* from speculation.

Throughput (gen tok/s), draft acceptance, and the correctness gate are reported per model. The
single invariant across everything here:

> **MTP speculative decoding is lossless by construction.** The target model's verify pass decides
> every emitted token; the draft head only proposes. So the *only* correctness question is whether
> our wiring preserves that invariant — proven per model by **logit-equivalence** (MTP-on vs the
> plain target), never by greedy token-match (FP-unstable; bug-270/356).

---

## 1. The two MTP flavours

Post-0.10.3 the two flavours are **distinct `--spec-type` strings** (the old unified `--spec-type mtp`
token is RETIRED) with distinct enums/graph types, dispatched in `common/speculative.cpp` /
`server-context.cpp`: the Gemma external assistant is `--spec-type draft-assistant`
(`COMMON_SPECULATIVE_TYPE_MTP`, our patch `0079`), the Qwen NextN self-spec is `--spec-type draft-mtp`
(`COMMON_SPECULATIVE_TYPE_DRAFT_MTP`, upstream-native). They differ in where the draft head lives:

| Flavour | Models | Draft head | Cache | Rollback |
|---|---|---|---|---|
| **External assistant** (M6-S4) | Gemma-4 A4B/12B/27B/E2B/E4B | a separate `gemma4_assistant` GGUF loaded INTO the target (`--mtp-head`) | iSWA (attention, seq_rm-able) | FULL — plain `seq_rm` of the rejected tail |
| **NextN self-spec** (M6-S5) | Qwen3.5-9B (qwen35), Qwen3.6-27B-Omnimerge (qwen35), Qwen3.6-35B-A3B (qwen35moe) | baked into the target GGUF (last `nextn_predict_layers` layer) — no second model | **hybrid** (gated delta-net recurrent backbone + full-attention layers) | **CKPT** — recurrent state checkpoint + re-decode |

Selection is explicit per engine: `--spec-type draft-assistant --mtp-head <gemma4_assistant.gguf>` for
the Gemma external assistant; `--spec-type draft-mtp` (no `--mtp-head`; `nextn_predict_layers>0` baked
into the target) for Qwen self-speculation. The TS adapter emits the right string from
`specType:"draft-assistant"` (+`mtpHead`) or `specType:"draft-mtp"` respectively (`config.ts`
`resolveEnvSpecType`; the retired `"mtp"` value is no longer accepted).

### 1a. Why qwen NextN needs a different rollback (checkpoint; upstream-native on 0.10.3 — W3b was opencoti's pre-bump port)

The qwen NextN models are **hybrid**: most backbone layers are gated delta-net (a *recurrent*
linear-attention whose state has no per-token history), and only the NextN/full-attention layers
keep a seq_rm-able KV cache. On a **partial** draft acceptance, the recurrent backbone has already
advanced through the rejected draft tokens and **cannot** erase its tail — `llama_memory_seq_rm`
returns false, and the upstream gate would simply *disable* speculation for these models.

We do not disable it (locked user directive). Instead we port upstream PR#19493's checkpoint idea,
adapted for self-spec (the NextN draft head is *stateless/regenerating*, so upstream's
discard-and-retry would re-emit the identical draft and livelock):

1. **Compat probe** is three-way now (`common_speculative_is_compat` → NO / FULL / **CKPT**). A
   target that decodes but can't partial-erase its tail returns CKPT — spec stays *enabled*.
2. **At draft time** (CKPT mode) the server snapshots the committed-prefix recurrent state
   (`llama_state_seq_get_data_ext`, `PARTIAL_ONLY` → only the recurrent half of the hybrid cache).
3. **On a partial accept** the server restores that checkpoint, trims the attention KV to the
   prefix, then **re-decodes the accepted tokens** to re-advance both halves to exactly the accepted
   sequence (the last token carries logits to seed the next draft's `h_prev`).

Cost: in principle one extra small decode per partial-accept step + one `PARTIAL_ONLY` snapshot per draft
step (the per-token ring buffer of PR#22673 would make both O(1) — tracked as **#457**). **PROFILED
2026-06-18 (§4b): on the 0.10.3 binary this path is DORMANT and the cost is MEASURED = 0** — upstream's
hybrid context supports *bounded partial sequence removal* (`common_context_can_seq_rm`), so for the
default `n_max=3` draft the recurrent rollback (`n_rollback ≤ 3`) is erased natively by `seq_rm` and the
checkpoint-restore/re-decode branch (`use_ckpt_*`) is never entered. The "`seq_rm` returns false" premise
above was the **pre-bump W3b** situation; the upstream-native 0.10.3 path no longer needs the re-decode at
shipped draft widths.

On the post-0.10.3 binary the shipped `draft-mtp` path is the **upstream-native**
`common_speculative_impl_draft_mtp` (NextN self-attention drafter + the checkpoint/recurrent-rollback above;
server `spec_ckpt`, `server-context.cpp`), and it is **`n_seq`-aware** — its samplers / backend-chains /
`pending_h` are all sized per seq, so it runs **per-slot under `--parallel`** (proven §5a/#485). This
**supersedes the `--parallel 1` limit** of opencoti's *retired* pre-bump W2/W3b CKPT self-spec. There are
**no** `// opencoti F5 M6-S5` source markers in the vendored tree: the NextN drafter and its rollback are
upstream-native; opencoti's only NextN-side vendored change is the W1 loader fix (#436 — load the NextN head
+ skip DeltaNet on the NextN layer). See the W2/W3b design record under `.opencoti/reconcile/phase0-20260615/`.

---

## 2. Hosts & methodology

| Host | GPU | Role |
|---|---|---|
| **solidpc** | 1× RTX 3090 (24 GB) | Throughput (off/on gen-tps) + correctness gates (logit-equiv). Q4_K_M / Q8_0 / IQ3 quantized models. |
| **bs2** | RTX 6000-class | 256k-context MTP + DCA-on cells (large KV). |

- **Throughput**: same fixed decode prompt, `--temp 0`, gen tok/s from the server completion
  `timings.predicted_per_second`. off = no `--spec-type`; on = self-spec / `--mtp-head`. Gate:
  `.opencoti/w6-qwen-mtp-tps.sh`.
- **Acceptance**: from the MTP NDJSON trace (`LLAMA_MTP_ACC_TRACE`) — accepted/drafted draft tokens
  and accept events (`statistics mtp` in the server log corroborates).
- **Correctness**: logit-equivalence — teacher-force shared prompts, compare per-position top-20
  logprobs MTP-on vs plain; **PRIMARY = `real_frac`** (real, non-benign-tie argmax disagreements;
  target 0). `mean_tv` is advisory only (it false-fails on spec-accepted tokens whose server returns
  empty top-logprobs — see the gemma precedent below). Gate: `.opencoti/w4-qwen-nextn-logit-equiv.sh`.

---

## 3. Gemma external-assistant MTP (M6-S4) — shipped

### 3.0 Recommended config (the adapter productizes this)

MTP is the largest single decode-throughput lever for the turbo / turbo_tcq KV tiers (it amortizes the
bandwidth-bound expert-weight reads across the draft window — see `docs/evaluations/decode-levers.md` L1/L2),
and it is **logit-equivalent** (§3a). The engine needs three llamafile flags acting together —
`--mtp-head <gguf> --spec-type draft-assistant -ngld 99` — but as of #533 the opencoti adapter
(`packages/opencoti-llamafile/src/config.ts`, `buildConfig`) **derives the companions so you set just one knob**:

| You set | Adapter derives | Result |
|---|---|---|
| `mtpHead: "<assistant.gguf>"` | `specType="draft-assistant"`, `draftNGpuLayers=99` | full gemma assistant-MTP, GPU-offloaded draft head |
| `specType: "draft-mtp"` (no mtpHead) | *(nothing)* | qwen NextN self-spec (rides the target's own offload) |
| *(neither)* | *(nothing)* | plain decode |

Explicit values always win (e.g. set `draftNGpuLayers: 40` to override the `99` default). Env equivalents:
`OPENCOTI_LLAMAFILE_MTP_HEAD`, `OPENCOTI_LLAMAFILE_SPEC_TYPE`, `OPENCOTI_LLAMAFILE_DRAFT_NGL`. The `-ngld 99`
default matters: a CPU-resident draft head stalls the GPU and erases the win. Recommended pairing for turbo
tiers: `ctypeK/ctypeV = turbo3_tcq` + `mtpHead = <A4B assistant>` → turbo3_tcq decode ≈ 89 tok/s on RTX 3090
(graphs+MTP, +69% over the 52.9 plain baseline). Covered by `test/mtp-defaults.test.ts`.

### 3.0a. Per-KV-tier MTP gain — re-confirmed on coherent output (#553, A4B v6-coder Q4_K_M, solidpc, 2026-06-24)

The §3.0 claim holds across the KV ladder. A4B chat coding prompt (in-domain, `/v1/chat/completions`,
`--temp 0`, 256 gen tok, graphs default, `coherent-mtp.sh`):

| KV tier | MTP-off tok/s | MTP-on tok/s | MTP gain | draft acceptance |
|---|--:|--:|--:|--:|
| f16 | 88.95 | 112.37 | **+26%** | 0.787 |
| turbo3 | 69.66 | 94.44 | **+36%** | 0.827 |
| **turbo3_tcq** | 56.63 | 75.63 | **+34%** | 0.652 |

**Re-confirmed on the fresh 0121 binary (#654, A4B v7-coder Q4_K_M, solidPC RTX 3090, ctx 8192,
`sp-mtp30a.sh`, 2026-07-12):** the per-tier win holds and is *larger* on the current engine —

| KV tier | MTP-off tok/s | MTP-on tok/s | MTP gain | draft acceptance |
|---|--:|--:|--:|--:|
| f16 | 76.58 | 137.03 | **+79%** | 0.912 |
| turbo3 | 65.67 | 97.10 | **+48%** | 0.864 |
| **turbo3_tcq** | 53.18 | 74.03 | **+39%** | 0.663 |

Same ordering (biggest on quantized tiers), same acceptance profile (turbo3_tcq's lower single-prompt
0.66 is the documented prompt-noise, not a deficit) — a clean reproduction on the shipping binary.

MTP is a real decode-tps win on **every** tier, biggest on the quantized-KV tiers (the §3.0 lever), and the
two graph layers compose (graphs survive turbo3_tcq + sparse-V: +13.4% / +13.1% on today's DSO, matching the
decode-levers L6 reference +13.1%). The single-prompt turbo3_tcq acceptance (0.65, BST prompt) is **prompt
noise, not a deficit**: averaged over 3 diverse coherent coding prompts (`coherent-mtp-multi.sh`) the means are
f16 **0.745** / turbo3 **0.757** / turbo3_tcq **0.703** — a ~0.05 spread within each tier's own prompt-to-prompt
variance, and on one prompt turbo3_tcq (0.766) *beat* turbo3 (0.721) and f16 (0.729). Consistent with
turbo3_tcq being the *more* accurate tier (§3a is logit-equivalent; 27B KLD 0.0346 < turbo3 0.047): no spec bug.

> ⚠️ **Acceptance/tps MUST be measured on coherent output.** Feeding an instruction-less prompt (e.g. a raw
> RULER/WikiText haystack to `/completion`) collapses greedy decode into a degenerate repetition loop where
> logits are near-tied; draft acceptance then becomes meaningless noise and even **f16 shows greedy MTP-off ≠
> MTP-on** (impossible on real tasks — f16 is proven `real_frac=0`, §3a). An earlier sweep on that prompt
> mis-measured turbo3_tcq acceptance at 0.25 and read "MTP hurts turbo" — a pure harness artifact (bug-710),
> not a kernel/codebook defect. Use `/v1/chat/completions` with a real instruction. **Gemma-4 thinking-coder
> writes its answer to `choices[0].message.reasoning_content`, not `.content`** (`.content` is empty with
> `finish_reason=length`) — parse `reasoning_content`. Latent note: the `TURBO_TCQ_CB` runtime codebook
> override reaches only the FA-VEC TU's `__constant__`, not the MMA-lift TU (`d_turbo3_tcq_cb_cpy`); the two
> baked tables are byte-identical so default decode is consistent, but an A′ codebook sweep would diverge
> draft (VEC) vs verify (MMA-lift) — fix before relying on `TURBO_TCQ_CB` for spec/MTP runs.

### 3a. Correctness — logit-equivalence (A4B v6-coder Q4_K_M, solidpc)

`s4-p3-logit-equiv` (MTP verify path vs plain target, 8 teacher-forced prompts):

| metric | value | meaning |
|---|---|---|
| **real_frac** | **0.000** | no real argmax divergence → logit-equivalent |
| real_disagreements | 0 | — |
| frac_full_agree | 1.000 | every position's argmax matched over the shared prefix |
| mean_top1_mae | 0.000 | top-1 logprob identical |
| mean_tv (advisory) | 0.444 | trips the catastrophe guard but is the empty-top artifact, NOT a real fail |

**Verdict: lossless.** (The compare script's literal `VERDICT` printed FAIL purely on the advisory
`mean_tv` guard — this is the documented spec-accepted-token artifact; the gate reads `real_frac`.)
Acceptance on these prompts: **1.00** (e.g. `statistics mtp … #acc drafts = N / #gen drafts = N`).

### 3b. Throughput — MTP off vs on (all 5 Gemma targets) + ours-vs-upstream (#441/#442)

Two regimes. The **canonical** figure is the A4B **RULER@32k** milestone (long-context, the representative
win). The **series sweep** (`.opencoti/gemma-mtp-series-sweep.sh`) fills all five targets at a fast,
consistent **4k-ctx / n_predict-256 / temp-0 / 2-prompt** regime, and — answering #441's "also include the
llama.cpp PR benchmark" — is run a **second time against upstream llama.cpp's *native* Gemma assistant-MTP**
(`b9660`, PR #23398/#24282) at the **same regime on the same RTX 3090** (`.opencoti/upstream-mtp-tps.sh`), so
ours-vs-upstream is apples-to-apples. `accept` = accepted/generated drafts (identical log metric both sides;
upstream reads it from the completion JSON `draft_n_accepted/draft_n`, ours from the server `draft acceptance`
line — same AtomicBot-lineage definition). Measured 2026-06-18 on solidpc RTX 3090.

**Canonical (A4B, RULER@32k):** off **56.6** → on **88.8** tok/s = **1.57×** (#484, post-bump), accept 0.87.
Absolute tps is lower than the 4k sweep below (long KV); the ~1.5–2× *speedup* is the regime-robust signal.

**Series sweep — ours** (llamafile 0.10.3 + #491 host-guard; single-context `--mtp-head` / `--spec-type
draft-assistant`; FA on throughout):

| model | arch | off tok/s | on tok/s | speedup× | acceptance |
|---|---|---|---|---|---|
| Gemma-4 A4B-26B | gemma4 (iSWA)    | 79.14  | 109.01 | 1.38× | 0.631 |
| Gemma-4 E2B     | gemma4 (elastic) | 132.84 | 195.23 | 1.47× | 0.533 |
| Gemma-4 E4B     | gemma4 (elastic) | 86.79  | 145.03 | **1.67×** | 0.448 |
| Gemma-4 12B     | gemma4           | 48.57  | 100.44 | 2.07× | 0.792 |
| Gemma-4 31B     | gemma4           | 19.64  | 39.13  | 1.99× | 0.662 |

**Series sweep — upstream b9660 native** (`ctx_other` engine; `-md … --spec-type draft-mtp
--spec-draft-n-max 3 -fa on`; same HW + regime):

| model | off tok/s | on tok/s | speedup× | acceptance | note |
|---|---|---|---|---|---|
| A4B | 92.92 | 147.97 | 1.59× | 0.839 | |
| E2B | 157.81 | 264.43 | 1.68× | 0.628 | |
| E4B | 99.28 | **CRASH** | — | — | `-fa on` aborts (D512/gqa=2 FA-kernel gap, bug-567); upstream's own documented workaround is global **`-fa off`** → 98.47 → 182.07 (1.85×, 0.542) |
| 12B | 51.70 | 137.59 | 2.66× | 0.928 | |
| 31B | 20.45 | 51.09 | 2.50× | 0.788 | |

**Upstream PR-published references** (PR #23398/#24282, mixed HW — *not* directly comparable; cited as the
"PR benchmark" the request asked for): 31B Q4 **1.55×** / Q8 **1.97×** (RTX PRO 6000); 31B **1.74×**
(2×RTX 3090, the closest HW match); DGX Spark wall 290.0 s → 120.7 s (**≈2.4×**), accept 0.588; "dense
**>2×** on average, MoE none"; E4B accept **0.479** (125/261) — within noise of our 0.448. Repro:
`python3 scripts/mtp-bench.py` against `llama-server … --spec-type draft-mtp --spec-draft-n-max 4`.
Upstream's own caveats, both load-bearing for the §3d decision: **quantized-KV → 0% acceptance** (later
fixed for Q only, still composition-fragile) and **`-fa on` crashes E4B** (PR #24282, verbatim).

**Three honest readings.**
1. **The off-baseline gap is a base-version artifact, not an engine gap.** Upstream out-decodes us even
   with MTP *off* (A4B 92.9 vs 79.1; 12B 51.7 vs 48.6) — that's newer `b9660` kernels vs our `0.10.3` pin,
   the price of carrying the opencoti KV moat upstream's base cannot. The MTP *speedup ratio* is the
   engine-comparable number; there upstream's separate-context engine is modestly ahead (1.6–2.7× vs our
   1.4–2.1×).
2. **E4B is a capability divergence, not a 4k-speed win.** Upstream **crashes** with FA on and must disable
   FA *globally*; opencoti's surgical **#491** host non-FA guard disables FA only for the one uncovered op
   (D512/gqa≤2) and keeps it everywhere else → E4B runs at 1.67× with FA on. At 4k, upstream's FA-off
   workaround is actually *faster* in raw tps (182 vs our 145; newer base + no-FA penalty is negligible at
   short KV) — so the #491 advantage is **latent at 4k**; it pays off at long context (FA-off is O(n²)) and
   for FA-dependent composition, not in this short-ctx cell. Plainly: ours keeps a capability upstream
   sacrifices; upstream is faster on the trivial case.
3. **The moat is composition, not raw speed.** Upstream's `ctx_other` engine cannot compose with PolyKV /
   rolling-KV / M2 split / TurboQuant / DCA and collapses under quantized KV — see §3d. That, plus
   E4B-FA-on, is why opencoti keeps its single-context engine despite conceding the f16/short-ctx tps race.

> Regime caveat: the sweep rows are **4k/n256** (fast, same-regime), **not** RULER@32k — absolute tps is
> higher at 4k (short KV). Compare *speedups* and the *ours/upstream deltas*, not absolutes, against the
> canonical A4B row; A4B appears in both as the cross-regime anchor.

### 3b′. am17an PR benchmark — A4B, single-harness ours-vs-upstream (consistent acceptance, 2026-06-18)

The §3b series tables read ours-acceptance from the per-request `draft acceptance` server line but
upstream-acceptance from the cumulative completion JSON — an **asymmetric aggregation**, fine for the
speedup column but not a like-for-like acceptance read. To settle ours-vs-upstream on **one metric from
one script**, the canonical am17an PR benchmark (`mtp-bench.py`, gist 228edfb84 / PR #23398 — 9 fixed
prompts, `max_tokens=192`, `seed=42`) is run **verbatim against every engine**; it reads
`timings.draft_n` / `draft_n_accepted` identically for ours and upstream. A4B v6-coder Q4_K_M, ctx 8192,
draft depth 2 (ours' measured optimum: `--draft-block-size 3` ↔ upstream `--spec-draft-n-max 2`),
`--temp 0`, FA on, RTX 3090 (`.opencoti/am17an/run-gemma-a4b.sh`):

| engine | spec path | accept | drafted | accepted | mean tok/s | ×ours-off |
|---|---|---|---|---|---|---|
| ours-off   | base, no spec                                 | —      | 0    | 0    | 78.04  | 1.000 |
| ours-seq   | `--mtp-head` draft-assistant, per-step sync    | 0.6628 | 1474 | 977  | 109.66 | 1.405 |
| ours-fused | `--mtp-head` draft-assistant, fused (shipped)  | 0.6635 | 1474 | 978  | 110.07 | 1.410 |
| up-off     | b9660 base, no spec                            | —      | 0    | 0    | 82.88  | 1.062 |
| up-mtp     | b9660 `-md … --spec-type draft-mtp` (ctx_other)| 0.7738 | 1344 | 1040 | 123.00 | 1.576 |

Upstream MTP-on is **1.118×** ours MTP-on (123.00/110.07), which factors cleanly into **two independent
levers, ≈half each**:

1. **Base decode — 1.062× (base-version-bound).** ours-off 78.04 vs up-off 82.88. CUDA graphs are reused
   on both sides (`graphs reused = 190…1700+` in our server log), so per-token cost is **GPU-kernel-bound,
   not launch overhead** — it is purely newer kernels in b9660 (`7dad2f1a1`) vs our pinned base
   `dbe9c0c8c` (a confirmed direct ancestor; 20 CUDA commits between them). The A4B-relevant ones:
   **#24087 enroll `mul_mat_vec_q_moe` into PDL** — A4B is MoE, and that GEMV is its single hottest decode
   kernel — and **#23729 MMVQ Turing parameter table** (Q4_K_M decode matmuls); the broader PDL
   (Programmatic Dependent Launch) cluster trims dependent-launch stalls even inside captured graphs. This
   6% is the **cost of the KV-moat pin**: we hold `dbe9c0c8c` so the 24-patch stack applies, foregoing
   these June kernel PRs. Recoverable only by a base bump (re-triggering the rebase friction).
2. **MTP speedup ratio — 1.053× (engine/acceptance-bound).** ours speedup-over-its-own-base 1.410× vs
   upstream 1.484× (123.00/82.88), driven by acceptance 0.664 vs 0.774 on the *identical* drafter GGUF.
   Note ours **drafts more** (1474) yet **accepts fewer** than upstream (1344 drafted, 1040 accepted): our
   single-context cross-read over-proposes lower-quality 2nd-step drafts where upstream's `ctx_other`
   recompute does not. Base-independent; a genuine future opportunity inside our control-plane (the
   same-draft-position hypothesis **#492** was tested and proved **inert** — prompt-1 152→151 accepted, a
   1-token regression; the lever is elsewhere, the 2nd-token hidden-state source). Not chased here.

Neither lever is the E4B FA crash — A4B is dense-MoE with gqa>2 and never hits bug-567; E4B's FA-on
capability is the §3b row. This A4B block is the authoritative **consistent-acceptance** ours-vs-upstream
comparison; the §3b series tables remain the 5-model coverage (with the noted aggregation caveat on their
A4B acceptance cell).

### 3b″. E4B D512/gqa≤2 — real flash-attn, #491 non-FA fallback retired (#493, patch `0080`, 2026-06-18)

The §3b E4B row ran on the **#491 host stop-gap**: a `build_attn_mha` guard that routed the drafter's
single GLOBAL layer (head4/kv2 → gqa 2 at head_dim 512 — the bug-567 kernel hole) to the non-FA
explicit-softmax branch, so `-fa on` no longer aborts but that one layer is "fa off". **#493 (patch
`0080`) replaces that with a real D512 f16 flash-attn** — a new `FATTN_VEC_CASE(512, F16, F16)` VEC
instance + selector route + VEC dispatch case, and the `#491` guard narrowed to fire only for the
residual no-kernel case (quantized/bf16 D512/gqa≤2, which the assistant drafter never produces). The
VEC kernel tiles `Q->ne[1]` (cols_per_block 1 decode / 2 prefill+verify), so it is a real FA at any n_q.

| E4B `-fa on`, assistant-MTP | decode tps | acceptance | verified output |
|---|---|---|---|
| OLD (#491 non-FA soft_max on the global layer) | 150.97 | 0.67910 | reference |
| NEW (#493 D512 f16 VEC flash-attn) | 147.86 | **0.67910** | **byte-identical** |

Correctness is decisive: assistant-MTP is exact spec-decode (the target re-validates every drafted
token), so a wrong drafter FA cannot corrupt the verified greedy output but **would** collapse
acceptance — equal acceptance (Δ=0) **and** a byte-identical verified prefix together prove the VEC FA
numerically correct. Decode tps is within boot-to-boot noise (the global layer is one of many; −2% is
not a regression). The two head-size switches in `fattn.cu` (selector **and** VEC dispatch) both need
the D512 case or the selector picks VEC while the dispatch aborts at decode time (`fattn.cu:363`).
A4B is **gqa 8** (head16/kv2 on its D512 globals) → untouched MMA path, no #493 code; its raw-`/completion`
"garbage" is a thinking-coder prompt-template artifact (bug-573), coherent via `/v1/chat/completions`.
Gate: `.opencoti/m493-fa-d512-gate.sh`. DSO rebuilt byte-identical both paths (sha256 `a8ddcd82`).

### 3b‴. #654 re-run on the fresh 0121 binary — both hosts (2026-07-12)

Refreshes §3b on the **current shipping engine** (patch chain through `0121`; the §3b "ours" table
above was the retired **#491 host-guard** stopgap, now superseded by the **#493** proper D512/gqa≤2 FA).
Two hosts, both **single-GPU pinned** (no cross-GPU host contention), robust decode-tps method (8-tok
`cache_prompt` warmup → 256-tok cache-reused measured completion, `timings.predicted_per_second`);
`accept = draft_n_accepted/draft_n`. bs2 = §3b family (ctx 32768); solidPC = sp-mtp-family (ctx 8192).

**bs2 (RTX 6000, `mtp3b-family.sh`, ctx 32768):**

| model | arch | off tps | on tps | speedup× | accept |
|---|---|---|---|---|---|
| Gemma-4 A4B     | assistant | 188.90 | 320.04 | 1.69× | 0.903 |
| Gemma-4 E2B     | assistant | 270.38 | 452.02 | 1.67× | 0.985 |
| Gemma-4 E4B     | assistant | 185.25 | 365.72 | **1.97×** | **1.000** |
| Gemma-4 12B     | assistant | 127.61 | 266.17 | 2.09× | 0.907 |
| Gemma-4 31B     | assistant | 42.61  | 86.55  | 2.03× | 0.563 |
| Qwen3.6-27B-Omni | nextn    | 52.55  | 99.55  | 1.89× | 0.637 |
| Qwen3.6-35B-A3B | nextn     | 195.26 | 275.79 | 1.41× | 0.868 |

The **E4B `-fa on`** cell is the direct #493/bug-567 verification: 1.97× at accept **1.000**, no abort —
confirming the D512/gqa≤2 FA gap is fixed on ours (the "`-fa off`" caveat in §3b belongs to **upstream
b9660**, not opencoti).

**solidPC (RTX 3090, `sp-mtp-family.sh`, ctx 8192):**

| model | off tps | on tps | speedup× | accept |
|---|---|---|---|---|
| Gemma-4 E2B         | 151.28 | 328.82 | **2.17×** | 0.959 |
| Gemma-4 E4B         | 92.26  | 233.57 | **2.53×** | 0.964 |
| Gemma-4 12B         | 48.43  | 117.84 | 2.43× | 0.851 |
| Gemma-4 A4B (iq4xs) | 80.34  | 156.15 | 1.94× | 0.990 |
| Qwen3.6-27B-Omni    | 54.37  | 85.49  | 1.57× | 0.629 |
| Qwen3.5-9B NextN    | 45.29  | 67.62  | 1.49× | 0.527 |
| Gemma-4 31B (resident, `--vram-target 23000`) | 18.93 | 31.88 | **1.68×** | 0.616 |
| Gemma-4 31B (spilled, `--vram-target 22000`)  | 11.12 |  8.42 | 0.76× (loss) | 0.605 |

The 3090's smaller absolute base tps leaves *more* headroom to hide draft cost → higher multipliers than
bs2 (E4B 2.53×, 12B 2.43×). The **Gemma-4 31B** is a **1.68× win when the KV stays resident** — the two
rows above are the SAME model/drafter/prompt/accept (~0.61); the only difference is VRAM headroom, and
it is decisive. This closes two earlier confusions:

1. **Coherent-output fix (harness):** the 31B is a *thinking* model (`<|turn>` framing, writes
   `reasoning_content`); the original bench fed it a raw RULER haystack via `/completion` with
   `ignore_eos:true` + greedy + no instruction (the §3.0a/bug-710 "measure on coherent output" violation),
   collapsing decode into a `</u>`×256 loop and a meaningless flat 11.14→11.28 / accept 0.926. Fixed by
   routing `sp-mtp-family` through `/v1/chat/completions` (chat template applied; chat response still
   carries the full `timings` incl. `draft_n`/`draft_n_accepted`).
2. **"MTP slower on 3090" (config, NOT a bug):** at the harness default `--vram-target 22000` the drafter
   (0.51 GB) + its dual-ctx shared-KV path squeeze the resident-KV window to **0 MiB budget**, so the
   whole KV streams from host over PCIe Gen3 (~6.5 GB/s) — and with `n_max=3` the drafter re-reads that
   spilled KV per draft step, so each cycle pays ~4 host-KV streams to produce ~1.8 accepted tokens
   (~2× the autoregressive KV traffic) → net loss. Lift the cap 1 GB to **23000** and the KV fits
   resident (host tail 62 MiB, negligible): base decode 11→**18.93** and MTP flips to **31.88 tps
   (+68%)**. So the 22000 default is simply too conservative for the 31B on a 24 GB card.

The GGUF/quant/template were never at fault (byte-identical to canonical
`bartowski/google_gemma-4-31B-it-GGUF`; `<|turn>` is the genuine Gemma-4 turn format — vocab has no
`<start_of_turn>`). On **bs2** (RTX 6000, roomy VRAM, ctx 32768) the same pair is **2.03×**; the 3090's
1.68× is the same win on a tighter card. bug-1116.

**§4c niah ladder re-run** (bs2, A4B-128e Q4_K_M native, DCA-on chunk 32768, q8_0/q8_0 KV,
`short3-niah.sh`): **niah_single_1 = 100.0** at **16k / 32k / 64k / 128k** — clean retrieval at every
length on the fresh binary.

### 3c. 0.10.3 restore (#423) — `--spec-type draft-assistant`, patch `0079`

The assistant drafter is restored onto the llamafile 0.10.3 base as the first-class
`--spec-type draft-assistant` (patch `0079-gemma-assistant-mtp`), coexisting with upstream-native
NextN (`--spec-type draft-mtp`) — distinct enums (`COMMON_SPECULATIVE_TYPE_MTP` vs `…_DRAFT_MTP`)
and graph types (`LLM_GRAPH_TYPE_MTP` vs `…_DECODER_MTP`). Gemma-4 is iSWA with no NextN head, so
the assistant is its ONLY MTP path — which is why the bump restores rather than drops it. Live
boot-smoke on the rebuilt CUDA host binary (A4B v6-coder Q4_K_M target + 26B-A4B assistant Q8_0,
solidpc RTX 3090, `.opencoti/0079-boot-smoke.sh`):

- server loads the assistant INTO the target (`MTP assistant loaded into target`) and the spec
  engine selects `draft-assistant`, **not** the `draft-simple` fallback that segfaulted (bug-517);
- decodes coherently and **prefix-stable** vs spec-off: 405-char byte-identical greedy prefix, then
  the expected benign FP-tie paraphrase split (verify-batch matmul reduction order ≠ single-token
  decode — the bug-263/270 artifact; free-running greedy is **never** the gate, §3a logit-equiv is);
- assistant actively drafts: 238/545 accepted over a 512-tok run (0.44; 0.83 over the first 96 tok).

Throughput here is NOT the §3b figure — at 512 tokens the per-step draft overhead is unamortized
(on 47 vs off 78 tok/s); the representative win is RULER@32k (#414 1.46×; **re-measured 1.57× / +57%**
after the #484 `tok_embd`→OUTPUT-buft re-claim — off 56.6 → on 88.8 tok/s, 87% accept; the 0.10.3 bump
had dropped the head to the CPU buft, bug-540, restored via the additive `create_tensor_output_head`
helper). The §5 composition cells (turbo / DCA / PolyKV / POSITION_WINDOW × assistant-MTP) remain the
locked logit-equiv gate (#461), now unblocked by this restore.

### 3d. Upstream-alignment (#476) — adopt `#23398`/`#24282` data-plane, keep our control-plane

Upstream merged Gemma-4 MTP (PR **#23398** = b9549, + E-series **#24282**). To make a future llamafile
bump onto a #23398 base a *mechanical* reconcile (clean delete of the identity layer), `0079` now carries
upstream's **data-plane identity** while keeping the opencoti **control-plane engine**. Decision +
rejection rationale: `docs/protocols/UPSTREAM_SYNC.md` (the `0079` block) and `LLAMAFILE_UPGRADE.md` G34.

- **Adopted (data-plane):** arch `gemma4-assistant` (hyphen); tensors `nextn.pre_projection` /
  `nextn.post_projection` + E-series `masked_embd_centroids` / `masked_embd_ordering` (bare, no `.weight`);
  KV `%s.embedding_length_out` + `%s.nextn_predict_layers`; matching internal C++ symbols
  (`LLM_TENSOR_NEXTN_PROJ_*`, `LLM_TENSOR_MASKED_EMBD_*`, hparams `n_embd_out_impl`). Drafter GGUFs
  re-formatted in place (Q8_0 bytes verbatim) via `.opencoti/gguf-upstream-rename.py`.
- **Rejected — `ctx_other`:** upstream's assistant runs a **second `llama_context`** aliasing the target KV
  (assumes vanilla single-owner/contiguous/unquantized KV; 0% accept under quantized KV upstream). It would
  break **all five** gate features (PolyKV / rolling-KV / M2 split / turbo / DCA), so we keep our
  single-context `build_attn_mtp` cross-read + ordered head + `tok_embd`→OUTPUT buft + fused N-step +
  `--spec-type draft-assistant`. **⚠ SUPERSEDED by §3e (2026-07):** the single-context facade never
  reached upstream accept parity (bug-858), so the `ctx_other` execution model WAS ported after all
  (dual-ctx, `OPENCOTI_MTP_DUAL_CTX=1`) — with the read-only shared-cache guards extended to
  opencoti's own KV machinery (bug-2102).

**Inert-rename proof (A4B v6-coder Q4_K_M, solidpc, canonical `gemma4-assistant` drafter):** **`real_frac = 0`**
(`s4-p3-logit-equiv.sh`) — same as §3a, the rename changed no logits; boot-smoke routes `draft-assistant`,
44% accept, coherent. The `0079` patch is **surgically confined**: 16/23 file-stanzas byte-identical to the
pre-rename `0079`; only the 7-file data-plane surface changed. **Chain proof (G33/G35):** full `0001..0079`
strict-applies from pristine (0 rejects), final tree == live byte-identical across **2941 source files**,
reverse round-trips. **All five drafters re-formatted** (E2B/E4B ordered head, 12B/31B/A4B dense) to the
canonical `gemma4-assistant` identity and verified byte-faithful (per-tensor sha256 source==output, arch
flipped, `embedding_length_out` + `nextn_predict_layers` present, no residual `mtp.*`); the transitional
`.upstream.gguf` artifacts were renamed to the canonical `<name>.gguf` and the old underscore-arch sources
removed. The full KV-composition matrix on the renamed engine is FOLLOW-ON (#461).

### 3e. bug-858 dual-ctx port + the two root causes — FIXED (2026-07-05, bs2, native A4B pair)

The #607 wholesale port runs the assistant as upstream does — a real second `llama_context`
(`ctx_dft`, gated by `OPENCOTI_MTP_DUAL_CTX=1`) sharing the target's KV cells — reversing the §3d
`ctx_other` rejection. Getting it to parity surfaced **two independent defects, both ours**:

1. **bug-2101 — `llm_graph_input_attn_kv_iswa::can_reuse` base drift** (`llama-graph.cpp`). The
   llamafile-0.10.3 base nested the kq-mask width check under `if (self_k_idxs && …->buffer)`; the
   KV-less drafter's k/v_idxs are dead nodes → check skipped → the boot **reserve graph**
   (full-cache-width mask) was silently reused for every draft decode; `set_input` filled only
   `n_kv` columns, the rest read scheduler-pool garbage. Signature: accept collapsed with prompt
   length (0.83 @114 tok → 0.00 @1314; upstream flat). Fix = upstream-verbatim independent
   if-blocks (b9859 restructured exactly when KV-less graphs were introduced).
2. **bug-2102 — opencoti lazy-clear on the aliased shared cache** (`llama-kv-cache.cpp`). The
   F4-M3 residency calls `ensure_cleared()`/`shrink_if_idle()` in `init_batch` ran UNGUARDED on the
   draft cache, whose layers **alias the target's k/v tensors** (`layer_share`) with a zero
   `n_cells_cleared` watermark — the drafter's first decode **memset the head of the target's live
   KV**. Signature: deterministic dual-vs-plain trajectory flips at 4–17-logit margins, persisting
   with p_min=1.01 (zero drafts); isolated by a 2-factor probe (tap-on + drafter-decodes-skipped ==
   plain byte-identical). Fix = `if (other) return;` read-only guards in both (upstream has neither
   function). Rule extracted: after adding cache sharing, EVERY mutating member needs the guard —
   including fork-specific ones upstream doesn't have.

(Also reverted an earlier `fattn.cu` `Q->ne[1]<=2` VEC experiment back to upstream `==1` — its
apparent win had been bug-2102's masked effect.)

**Post-fix gates (native pair: google 26B-A4B Q4_K_M + HF-canonical assistant Q8_0, bs2, vs b9859):**

- **Accept (9-prompt compare, greedy 200-tok):** n1 **0.862 vs 0.876** (parity; 8/9 prompts within
  noise), n2 **0.800 vs 0.792** (above), n3 0.679 vs 0.736 (#609-class target numerics —
  characterized and closed below; recommend n2 for A4B).
- **Accept vs prompt length (filler×k):** ours **flat at/above upstream at every length**
  (0.85/0.83/0.92/0.88/0.92/0.92 vs 0.87/0.83/0.91/0.88/0.87/0.87 at 34→2594 tok) — the bug-2101
  cliff is gone.
- **Losslessness (dual==plain, temp 0):** now matches upstream's own flip profile — n1 fib/fr
  byte-identical end-to-end; the two residual flip prompts (clamp@43, forces@126) are positions
  where ours-plain also differs from up-plain (pre-existing #495 base numerics; upstream itself
  flips clamp@96). n3: 3/4 == upstream's 3/4.
- **Throughput (mtp-matrix, 400-tok):** base **204 vs 190** (ours +7%); MTP-on n2 **314.7 vs
  319.4** (−1.4%), n1 261 vs 288. Full matrix incl. 31B / Qwen-35B / Qwen-27B in
  [`three-way-tps.md`](three-way-tps.md) §2026-07-05.

**#609 depth-3 rung: CLOSED (bug-2103, 2026-07-05).** The Qwen-35B n3 decode collapse was
TinyBLAS's `mmvf ne11<=3` cap dropping the 4-token verify batch's f32 GEMVs into a 1-block
GemmEx; post-fix n3 is monotone (225→237→247) and ours' best Qwen rung. A4B n3 accept recovered
to 0.784 cell / 0.688 9-prompt aggregate vs upstream 0.736 — the residual −4.8pp is spread
thinly across prompts (#495-class target numerics, not depth machinery). **2026-07-05 final
#614-binary re-verification (all four pairs — A4B + 31B assistant, Qwen 35B + 27B NextN):**
decode·accept unchanged within noise (A4B n1/n2/n3 257·0.975 / 302·0.845 / 329·0.784; 31B
100·0.849 / 119·0.791; q35 225·0.858 / 237·0.752 / 247·0.662; q27 99·0.752 / 105·0.626), and
the 9-prompt aggregates hold at n1 0.856 vs 0.876, n2 0.784 vs 0.792, n3 0.688 vs 0.736
(per-prompt table in three-way-tps.md). Recommended depth: **n2 on A4B/31B assistant, n3 on
Qwen NextN**. Historic diagnostic ladder: buglog bug-2101/2102/2103; probes at
`bs2:/srv/ml/opencoti-c1/gate{4,5,10,13,14,15,16,18}.sh`, matrix at `mtp614-out/`.
**⚠ The decode-tps columns above are SUPERSEDED by §3h (#675, 2026-07-14)** — every ours spec
cell gained +4–20% from the spec-clone-cheap fix; accepts are unchanged (bit-identical).

---

### 3f. #611 (2026-07-05): single-context facade REMOVED — dual-ctx is the only assistant path

The retired single-context in-target engine (`llama_decode_mtp` facade, `build_attn_mtp`
cross-read, `llm_build_gemma4_mtp`, `common_speculative_impl_draft_assistant`, async worker) is
deleted (patch `0096`, ~1600 lines); the `OPENCOTI_MTP_DUAL_CTX` env gate is gone and
`--spec-type draft-assistant` now ALWAYS runs the §3e dual context (server creates `ctx_dft`
with `ctx_other=target`; CLI surface unchanged). Gate on the #611 binary (bs2, GPU0, no env):
A4B n2 **305.4 tps / 0.849 accept** (== §3e reference), q35 NextN n3 **248.8 / 0.662**.

**bug-2110 — RESOLVED-NOT-A-BUG (harness artifact, same day).** The tps-gate harness showed
accept 0.849→0.635 at `--parallel 2 --kv-unified`, but its prompt (160× repeated filler, temp 0,
`ignore_eos`) drives the model into degenerate repetition loops; par2 merely shifts greedy decode
onto a *different* garbage trajectory (#495-class knife-edge), and accept-on-garbage swings wildly
between trajectories (upstream b9859 flips the same way on that harness). On the REAL 9-prompt
mtp-bench (`/srv/ml/opencoti-c1/2110-real/`): **ours par1 0.785 vs par2 0.783 aggregate** (max
per-prompt delta 2pp), upstream 0.793 vs 0.792 — the dual-ctx assistant HOLDS accept at
`--parallel 2 --kv-unified`, within 1pp of upstream at both. PolyKV × assistant-MTP composes
cleanly. Lesson (cerebrum Do-Not-Repeat): never read accept off the repetitive-filler tps harness.

### 3g. WS2 turbo-V in-register × MTP-on — the ship-gate closer (#629, A4B, bs2)

WS2 (#629) reads a rotated turboN V-cache **in-register inside the multichunk DCA fused flash-attn
kernel** (killing the per-forward whole-cache `cpy_turboN_f16` cast; `n_q≤16` gate; **DEFAULT-ON**;
full decode-tps matrix in [context.md §5x](context.md)). The remaining question for MTP was whether
the in-register path stays lossless **through the draft-assistant verify batch** (n_q > 1). It does —
airtight, by measurement not inference:

- **Determinism floor (MTP-on, `WS2=0` twice): byte-IDENTICAL.** The WS2=0 reference path is
  deterministic, so any WS2=0-vs-1 diff is real signal, not sampling noise.
- **n_q=1 teacher-forced logit-equiv (no MTP):** `real_frac 0.0`, `mean_tv 0.00041`, `ffa 1.0` →
  **logit-lossless** on the single-token decode.
- **n_q>1 teacher-forced logit-equiv (MTP draft-assistant verify batch, n_q = draft+1):**
  `real_frac 0.0`, `mean_tv 2e-05`, `ffa 1.0` → **logit-lossless in the multi-token verify batch.**
  This is the airtight closer — the verify batch is exactly where an `n_q`-gated in-register path
  could have diverged, and it does not (measurement, not inference).
- **Free-running divergence (MTP-on `WS2=0` vs `WS2=1`):** forks at char ~82 (and the n_q=1 no-MTP
  arm at char ~486), both coherent / greedy / same GGUF → a **pre-existing #495-class base FP-tie
  autoregressive fork**, NOT a WS2 defect. **Accept parity: 90/90.**
- **Compose:** WS2 default-on in-register (**+89% decode**, §5x A4B turbo2_0) composes with the MTP
  draft-assistant path; **both are lossless and stack.**

**Ship-gate:** WS2 × MTP is closed — in-register turbo-V is logit-lossless through the verify batch,
accept holds at parity, and the only free-running divergence is the known base FP-tie fork. Cross-ref:
[context.md §5x](context.md) (WS2 decode-tps matrix + caveats).

### 3h. #675 (2026-07-14): the last per-cycle host gap — bug-2180 spec-clone under cosmo dlmalloc (patch `0128`)

After #609/#614, Qwen 35B NextN still trailed b9859 ~12% at every spec depth with base decode at
parity — a constant per-cycle cost, invisible to nsys/GPU profiling. Per-phase wall-clock timers
(`OPENCOTI_SPEC_PROF`, temporary) in BOTH servers produced a phase-parity table: draft, verify,
and process each at parity; the whole gap sat in the **process→sample** seam. A direct timer
nailed it: **`common_sampler_clone` cost 1.071 ms/cycle on ours vs ~0.07 ms on b9859** — from
IDENTICAL source. The clone deep-copies the n_vocab-sized `cur` scratch (~1.8 MiB); under the
cosmo **dlmalloc** runtime every ≥256 KiB allocation is a fresh mmap + page-fault-on-write +
munmap, while b9859's glibc reuses the freed block. A runtime-allocator asymmetry, not an
algorithmic one (bug-2180).

**Fix (spec-clone-cheap, `common/sampling.cpp`):** clone with empty `cur`/`cur_p`. Safe because
`cur` is write-only scratch that `set_logits()` repopulates before any read, and the copied
`cur_p.data` pointed into the SOURCE sampler's buffer anyway (latent upstream aliasing — no
consumer can rely on the clone's copy). Restore via move-assign unaffected. Clone 1.071 →
**0.005 ms/cycle**. Companion fix in the same patch: the mmvf **f16/bf16** ampere_mma TinyBLAS
lift (`ne11≤8`, `src0_small`-guarded) — the bf16 shared-expert gate GEMVs
(`ffn_gate_inp_shexp`) in the 2–4-token verify batch were the dominant residual tinyblasGE
cliff after #609's f32 fix. Both fixes are output-identical: accept bit-identical pre/post at
every depth.

**Gate (bs2 GPU1, binary `05240e8e57c7191e`, all diag stripped):** q35 base 212.6 = parity
(unregressed); n3 **282.6 vs b9859 287.4 (−1.7%**, was −12%); full same-day matrix — A4B
**307.9/353.8/378.3 vs 289.1/321.8/369.1** (ours WINS all depths), q35 265.2/270.0/278.8 vs
261.3/277.7/270.7, 9-prompt decode mean **317.7/361.6/365.7 vs 298.1/320.2/339.8** — tables in
[`three-way-tps.md`](three-way-tps.md) §2026-07-14. Characterized non-blockers: ~0.15–0.2
ms/cycle pre-draft glue; #495-class accept deficit at depth ≥2 (chained-draft numerics), worth
its own investigation only if the 31B/27B −6% matters. Diagnostic lesson (cerebrum): a constant
ms/cycle host gap with GPU parity on identical source ⇒ suspect the RUNTIME (allocator mmap
threshold), and measure with host wall timers, not nsys windows. Build gotcha hit during
cleanup: bug-2181 (ccache+cosmocc fat-object stale `.aarch64` sidecar — delete both `.o` twins
+ `CCACHE_DISABLE=1`, verify the APE with `strings`).

## 4. Qwen NextN self-speculation (M6-S5) — all 3 models GATED (correctness + tps)

> **Status (2026-06-18).** All 3 Qwen models gated on BOTH axes. **Correctness** (`real_frac=0`, §4a): 9B
> Q8_0 (K=48, full KV-composition sweep), 27B-Omnimerge Q4_K_M (K=256), 35B-A3B IQ3 MoE (K=256) — the spec
> path is lossless on every model (recall matches plain on all prompts). **Throughput** (§4b): dense 9B 1.92×
> / 27B 1.89× (NextN head survives the Omnimerge merge), MoE 35B-A3B 1.18× (MoE wins less). Fused draft
> (#454) ≡ sequential (no win — fusing draft-token graph steps buys nothing; the recurrent-snapshot cost
> #457 targets is a *separate*, unmeasured residual — not "moot"). **CoT-budget finding (bug-560):** the
> needle harness default K=48 spuriously inflates `real_frac` on reasoning models (plain decode cut off
> mid-`<think>`); K≥256 → `real_frac=0`. Both 256k inter-chunk-DCA cells now **PASS** `real_frac=0` (§4c):
**Qwen NextN** (27B-MTP Q6_K, 2026-06-29) and **Gemma** assistant-MTP (A4B-it + adapted drafter, 2026-06-30).

Models: `Qwen3.5-9B-Q8_0` (qwen35, nextn@32) · `Qwen3.6-27B-Omnimerge-v4-Q4_K_M` (qwen35, nextn@64)
· `Qwen3.6-35B-A3B-UD-IQ3_XXS` (qwen35moe, nextn@40). Loader fix #622 (W1); drafter graph W2;
self-spec + CKPT rollback W3/W3b.

### 4a. Correctness — logit-equivalence (self-spec vs plain), solidpc

**All 3 models DONE — `real_frac=0`** (9B at K=48; 27B/35B at K=256 — see CoT-budget caveat below).
`w4-qwen-nextn-logit-equiv.sh` (a wrong recurrent rollback would surface here as large `real_frac`; it does
not — the spec path is lossless on every model, recall matches plain on all prompts). The 9B cell also
carries the full KV-composition sweep (the Qwen column of §5): base + asymmetric + DCA + q4/q8 +
turbo2/3/4/8, **all `real_frac=0`**.
VERDICT prints `FAIL` from the `mean_tv` advisory only — the **known empty-top artifact** (spec-accepted
tokens return `top:[]`); the decisive signal is `real_frac`/`real_disagreements` (bug-263/270), both 0.

| model | real_frac | frac_full_agree | draft acceptance | verdict |
|---|---|---|---|---|
| Qwen3.5-9B (Q8_0) | **0.0** (K=48) | 1.0 | 0.79–0.94 | **PASS** (real_frac; TV-`FAIL`=empty-top artifact) |
| Qwen3.6-27B-Omnimerge (Q4_K_M) | **0.0** (K=256) | 0.875 | 0.685 (engaged, §4b) | **PASS** — K=48 showed 0.125–0.167, a CoT-budget artifact (bug-560), not a spec defect |
| Qwen3.6-35B-A3B (IQ3) | **0.0** (K=256) | 0.75 | 0.705 (engaged, §4b) | **PASS** — both recall 8/8; 2 benign divs (1 near-tie pos-7, 1 post-answer tail pos-193) |

> **CoT-budget caveat (bug-560).** The needle harness default `K`=`n-predict`=48 is **too short for a
> reasoning/CoT model**. The 27B-Omnimerge emits `<think>…</think>` before answering; at K=48 a benign
> pos-3 near-tie (`tok 198→271` = "keep thinking" vs "close think+answer", margin 0.02–0.25 nats <<
> `tie_margin` 0.7, `top1_mae=0`) tips plain decode toward verbose CoT, which then runs out of budget
> **before** emitting the needle → spurious `ref_recall=False` while the spec path answered correctly
> (`cand_recall=True`). That scored as `real_frac` 0.125 (n=8) / 0.167 (n=24) — but **every** divergence
> was *cand-better, never cand-worse*. At **K=256 both paths recall 8/8 → `real_frac=0.0`**: the spec path
> is lossless (the flip is the documented batch-vs-sequential FP micro-bias, surfacing on Q4_K_M where the
> Q8_0 9B had none). Run reasoning-model cells with `K≥256`.

### 4b. Throughput — off vs on, solidpc (ctx 8192, fixed decode prompt, `--temp 0`, 2026-06-18)

**All 3 Qwen models DONE** — self-spec is a decode win on every one; `qwen-nextn-tps-confirm.sh` (tps
from server `timings.predicted_per_second`, bug-470). The win is large on the **dense** models (~1.9×) and
small on the **MoE** (1.18×) — see the MoE note below.

| model | arch | off gen tok/s | on gen tok/s | speedup× | draft acceptance |
|---|---|---|---|---|---|
| Qwen3.5-9B (Q8_0) | dense | 44.2 | 84.7 | **1.92×** | 0.76–0.80 |
| Qwen3.6-27B-Omnimerge (Q4_K_M) | dense | 22.8 | 43.0 | **1.89×** | 0.685 |
| Qwen3.6-35B-A3B (IQ3_XXS) | MoE (A3B) | 76.8 | 90.9 | **1.18×** | 0.705 |

9B is two fixed prompts (off 44.23/44.09 → on 82.80/86.5 tok/s, 1.87×/1.96× @ accept 0.757/0.795); the
table reports their mean. 27B confirms the win holds on a Q4_K_M merge whose NextN head survived the
Omnimerge (draft engaged at 0.685).

**MoE self-spec wins less.** The 35B-A3B is a MoE with only ~3B active params, so its baseline decode is
already fast (76.8 tok/s off, vs 44/22.8 for the dense 9B/27B) and **memory-bandwidth-light per token**.
Speculation trades a fixed draft+verify overhead for skipped target steps; when each target step is already
cheap (few active params) that overhead amortizes far less, so the net is 1.18× rather than ~1.9× — despite
comparable acceptance (0.705). Self-spec still helps MoE, just modestly; the headline win is a dense-model
effect.

**Fused draft (#454) ≡ sequential — no win.** `OPENCOTI_MTP_FUSED=1` (one fused N-step graph launch) is
byte-identical to the per-step sequential path and no faster (9B: seq 82.80 vs fused 83.15 tok/s = 1.004×;
identical acceptance 0.75652 and identical per-iteration drafted-token NDJSON, `w6b-…-equiv.sh`). Fusing the
**draft-token graph steps** buys nothing — the native drafter's per-step graph work is already minimal — so
#454 lands as "sequential is optimal".

**#454 ≠ #457 (don't conflate the two costs).** #454 fused the *draft-token* graph steps (measured: no
win). #457 targets a **different** cost: the hybrid backbone is recurrent (gated delta-net), so on a
*partial* draft accept the server would restore a `PARTIAL_ONLY` recurrent-state checkpoint and re-decode
the accepted tokens to re-advance past the rejected AR draft (server `spec_ckpt`, `server-context.cpp`
~3578-3606). PR#22673's ring-buffer rewind would make that snapshot/rollback O(1).

**#457 PROFILED (2026-06-18, Qwen3.5-9B Q8_0, solidpc, `.opencoti/m457-ckpt-profile.sh`) → re-decode cost
MEASURED = 0; PR#22673 port NOT justified at shipped config.** Driving the binary's own `LLAMA_TRACE=1`
instrumentation over a 512-token decode (71.2 tps):

| metric | value |
|---|---|
| spec iterations | 183 |
| partial accepts (`accepted < drafted`) | **108 (59%)** — dist: 40×`0/3`, 31×`1/3`, 37×`2/3`, 74×`3/3` |
| **`(restore checkpoint)` re-decode events** | **0** |
| draft acceptance | 0.60 (328/547) |
| `#457` speedup ceiling `(1+P)` | **1.000×** |

The decisive log line is `common_context_can_seq_rm: the context supports bounded partial sequence
removal`: upstream 0.10.3's hybrid context natively partial-rewinds the recurrent state up to a bound that
covers the default `n_max=3` draft, so all 108 partial rollbacks were handled by `seq_rm` and the
checkpoint-restore/re-decode branch (`use_ckpt_*`, gated on `ctx_*_seq_rm_type == FULL` or rollback >
`llama_n_rs_seq`) was **never entered**. The snapshot is also skipped (same gate), and even if taken it is
an `ON_DEVICE` D2D copy of recurrent state (~µs). So the ~1.9× win has **no re-decode residual to absorb** —
the rollback is already effectively O(1). **Caveat (the only path that would revive #457):** a draft width
large enough that `n_rollback > llama_n_rs_seq` (e.g. raising `--draft-max` well past the bound) would start
entering the checkpoint path; at that point re-profile before porting. Not a #462 blocker; **closed**.

The earlier "0.75× W6 regression" was a stale-token artifact (retired `--spec-type mtp` on the older binary
→ 32.46 tok/s); `draft-mtp` on the current binary gives the wins above (bug-557).

**R-series closure — the "3-3.5×" headline was architecture-capped at the measured ~1.9×; #452/#453 are
subsumed (2026-06-30).** The Qwen NextN self-spec line was scoped (#451) with an aspirational *3-3.5×*
target; the delivered, dual-axis-gated ceiling is **1.92× (9B) / 1.89× (27B) / 1.18× (MoE 35B)** — §4b
above, post-#457. The gap is structural, not a missing optimization: the hybrid gated-delta-net backbone's
verify step bounds the win, the fused-graph path is byte-identical-and-no-faster (#454), and the
checkpoint/re-decode cost #457 would have absorbed is **already 0** (native `seq_rm` partial-rewind covers
the default draft width — re-decode events measured = 0, `(1+P)` ceiling = 1.000×). #452's nominal
deliverable (DCA composing in the MTP block) is **done** — Qwen NextN 256k inter-chunk DCA-on PASSES
`real_frac=0` (§4c). #453's generalize/build/eval/ship is **done** — all three archs (qwen35 / qwen35moe)
gated in §4b and shipped via `0079`. No remaining work pushes 1.9× → 3.5× on this design; chasing it would
need a fundamentally different (non-recurrent) drafter, not a continuation of this line. **Closed as
delivered-at-1.9×.**

### 4c. MTP + DCA-on interaction (256k) — Qwen NextN **PASS** + Gemma assistant **PASS** (bs2, 2026-06-29/30)

**Qwen NextN self-spec — PASS** (`real_frac=0`, bs2 GPU1, 2026-06-29). Model:
`Qwen3.6-27B-Omnimerge-v4-Q6_K` from the **dedicated MTP-quant repo**
(`ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF`, 22.4 GB, carries the 4 `blk.64.nextn.*` tensors — the
plain `…-v4-GGUF` Q6_K has **no** NextN head and fails MTP load). Harness `w4-dca256k-bs2.sh`: `-c
262144`, `--dca on --dca-chunk-size 32768` (8 chunks → genuine **inter-chunk** DCA on this native-256k
model), ref = plain target, cand = `--spec-type draft-mtp`, **both DCA-on** so the comparison isolates
MTP composition over DCA-laid-out KV + the W3b CKPT rollback. Verdict: `real_frac=0.0`,
`real_disagreements=0`, `frac_full_agree=1.0` → NextN self-spec is **logit-equivalent to plain** under
inter-chunk DCA at 256k. (`mean_tv_advisory=0.458` is the known empty-top-logprobs artifact for
spec-accepted tokens — advisory only; `real_frac` is the gate, per [[mtp-s4-p3]].) **Caveat:** 1 of 2
~228k prompts overflowed `n_ctx_train`; the compare ran over the 1 clean prompt × 64 decode positions
(clean `real_frac=0`, sufficient for the composition gate which §4a + the 9B sweep already established
mechanistically).

**Gemma assistant-MTP — PASS** (`real_frac=0`, bs2 GPU1, 2026-06-30, 2/2 non-empty). Target = Gemma-4
A4B-it (`gemma-4-A4B-62e-…-it-Q6_K`, arch gemma4/iSWA, native 262144); drafter = the **adapted**
`gemma4-assistant` (regenerated on bs2 via `gguf-upstream-rename.py` from the stale underscore
ollama_staging copy — the binary rejects the old `gemma4_assistant` underscore arch). cand adds
`--mtp-head <drafter> --spec-type draft-assistant -ngld 99`; both boots `--dca on --dca-chunk-size
32768`. Verdict `real_frac=0.0`, `real_disagreements=0`, `frac_full_agree=1.0` → assistant-MTP verify
composes logit-equivalently with DCA-laid-out KV at 256k. (`real_frac` is robust to drafter/target
quality — verify always uses target logits — so it isolates the MTP+DCA composition, not acceptance.)

---

## 5. Composition matrix — every KV development × MTP (the locked two-engine gate, #461)

MTP is TWO engines (§1) and **both** must compose with every other KV feature; where an engine is
limited we extend it (never "N/A" — user directive 2026-06-17). Rows = features; cols = the engine
that carries each model. The assistant-drafter restore (#423) is **DONE** — the engine boots, loads
into the target, and drafts on 0.10.3 (§3c). The gemma gate scripts' `mtp`→`draft-assistant` refresh is
done (#473); **9 of the Gemma cells PASS** `real_frac=0` (§5b, 2026-06-17), DCA composes, **PolyKV
composes under `--kv-unified`** (#485, corrects the earlier bug-550 misdiagnosis). The last
Gemma-side item — POSITION_WINDOW-at-overflow at long ctx — is **architecturally N/A**, not open:
`--kv-residency-mode window` is ineligible for Gemma's iSWA caches (see the POSITION_WINDOW row of
the composition matrix below); the cell stands on logit-equivalence.

| feature | NextN-MTP (Qwen, `--spec-type draft-mtp`) | assistant-MTP (Gemma, `--spec-type draft-assistant`, 0079) |
|---|---|---|
| scalar f16 / q8_0 / q4_0 | **PASS** real_frac=0 (GAP-1) | **PASS** real_frac=0 — f16 (#474) + q8_0/q4_0 (`gemma-mtp-compose.sh`, A4B v6-coder, 2026-06-17) |
| asymmetric q8_0/q4_0 | **PASS** real_frac=0 (GAP-2) | **PASS** real_frac=0 (2026-06-17; V dequant-on-lift under the draft ctx) |
| turbo2/3/4/8 (fused FA-VEC) | **PASS** real_frac=0 (GAP-3) | **PASS** real_frac=0 all 4 tiers (2026-06-17; InnerQ `tier-family` scale, hd-map 2048:256,1024:512) |
| DCA-on INTRA (ctx ≤ n_ctx_train) | **PASS** real_frac=0 | **composes** real_frac=0 (`--dca on --dca-chunk-size 1024`, 4 chunks @4096; no boot banner — genuine inter-chunk = 256k bs2) |
| **PolyKV** (SharedKVPool, `--parallel ≥2`) | **PASS** — composes (§5a) | **PASS** — composes under `--kv-unified` (which PolyKV forces); boot guard still rejects non-unified multi-slot. Re-proven on the #611 dual-ctx path with the real 9-prompt bench: accept **0.783 @ `--parallel 2 --kv-unified`** vs 0.785 single-slot (upstream 0.792/0.793) — bug-2110 investigation, §3f |
| rolling-KV **POSITION_WINDOW** (iSWA) | n/a — Qwen non-iSWA | **PASS on correctness** real_frac=0 (logit-inert, §5b) + forward path proven by `rolling-kv-equiv-gate` + #409. **256k engagement is architecturally N/A on the post-bump binary** (bs2, 2026-06-30): `--kv-residency-mode window` is *ineligible* for Gemma's iSWA caches — the only windowable cache is the SWA one (`swa_type=1`, bounded 1536 cells → never overflows), and window-mode is disqualified by `swa_type=1`/`v_trans=0`/`offload=1` (`llama_kv_cache: … ineligible … falling back to head/resident`). The 2-KV-head global cache spills on the **head-axis** path (M2/#485), not position-window. So this cell stands on logit-equivalence; "engage POSITION_WINDOW at real 256k overflow" is not exercisable for Gemma without a binary-level eligibility change (not pursued — composition correctness already established) |
| 256k stream / inter-chunk DCA | **PASS** real_frac=0 (27B-MTP Q6_K, chunk 32768/8 chunks @256k, #439, 2026-06-29 §4c) | **PASS** real_frac=0 (A4B-it + adapted assistant drafter, chunk 32768 @256k, #431, 2026-06-30 §4c) |

**§5b — Gemma assistant-MTP composition (solidpc, A4B v6-coder Q4_K_M, 2026-06-17).** Eight cells
**PASS** (`real_frac=0`, `gemma-mtp-compose.sh` → `s4-p3-logit-equiv.sh` with KVARGS/DRAFT_KVARGS hooks,
#473): scalar f16/q8_0/q4_0, asymmetric q8_0/q4_0, turbo2/turbo3/turbo4/turbo8. ref and cand BOTH carry
the feature flags so the feature's own accuracy cancels and `real_frac` isolates the draft/verify path;
the renamed binary loads the **canonical** `gemma-4-…-assistant-Q8_0.gguf` drafter (post-reformat) under
each KV type and routes `draft-assistant`. DCA-on (chunk 1024, 4 chunks) composes (`real_frac=0`) but has
no boot banner and is positionally inert at ctx ≤ `n_ctx_train` (262144) — the decisive inter-chunk DCA
test is 256k/bs2. POSITION_WINDOW is logit-inert (`real_frac=0`) yet the residency planner keeps the tiny
2-KV-head global iSWA cache fully resident at 4096 even under `--vram-target 1`/`window` (the historical
`--vram-target 1100`+auto that windowed 256/4096 no longer forces overflow on the post-bump binary) — so
its real regime is long-context overflow (256k). An **engagement guard** in the driver (`MARK[]` greps the
cand server.log for the feature banner) prevents a silently-disengaged feature from masquerading as a
vacuous f16 pass.

**PolyKV × assistant-MTP — COMPOSES under `--kv-unified` (#485; corrects the earlier bug-550 misdiagnosis).**
`gemma-polykv-probe.sh` boots the full PolyKV combo (`--parallel 2 --kv-unified --no-cache-idle-slots
-ctk/-ctv q8_0 --mtp-head … --spec-type draft-assistant`) → **boots, routes `draft-assistant`,
`real_frac=0` + ~93% draft acceptance** across the shared-prefix slots. PolyKV (SharedKVPool) forces
`--kv-unified`, so the PolyKV config lands on the supported path by construction.

*What the earlier session got wrong (bug-550).* The first probe reported "BOOT FAIL / `--parallel 1` only".
That was a **probe false-positive**: the readiness loop grepped a bare `failed to`, which matched the
*benign, caught* warning `[spec] failed to measure draft model memory` (the spec-fit pre-measure tries to
build a standalone draft `llama_context`; `gemma4-assistant` correctly refuses to be a primary `-m` model,
the warning is logged, and the server **continues**). The server was in fact booting fine. Two fixes folded
into `0079`: (a) skip the impossible standalone fit-measure for the assistant engine
(`spec_assistant_fit`, `server-context.cpp:809`); (b) tighten the probe grep to genuine fatal signatures.

*The one real limit — non-unified multi-slot — is now a clean boot guard, not a crash.* `--parallel >1`
**without** `--kv-unified` selects the per-stream-split KV layout (patch 0031). The single-context
assistant draft cross-reads the target KV via the shared `build_attn_mha` (`build_attn_mtp` →
`get_k`/`get_v`), and `build_attn_mha` infers the batch's stream count from the **target** K tensor's 4th
axis — `const auto n_stream = k->ne[3]` (`llama-graph.cpp:2115`) — then splits the *query's* token axis by
it (`:2117`). Under the per-stream split `get_k` concats one view per covered stream on dim 3, so
`k->ne[3] = n_streams > 1`; but the assistant draft query is single-stream, single-token
(`q->ne[2] = 1`, `gemma4_mtp_build_one_step` builds `[.,1]` step tensors, `ctx_dft==nullptr`). Partitioning
a 1-token query by `n_streams` (`1/n_streams → 0`) corrupts the stream/token geometry → ggml reshape
nelements assert (`ggml.c:3754`) at the first draft decode. The boot guard (`server-context.cpp:940`)
requires `--kv-unified` for `--parallel >1` and fails clean with an actionable message instead of the
mid-request SIGABRT. Under `--kv-unified` the cache is **not** per-stream-partitioned (`n_stream=1`,
`get_k` returns `ne[3]=1`), the line-2117 split is the identity, and the cross-read matches the draft
query exactly — the byte-identical path the engine is proven on. Making the draft itself `n_seq`-aware
(one drafted token per active stream, like Qwen NextN §5a) would lift the non-unified limit too, but is
**not** needed for PolyKV (which is unified) — tracked under the #469 extend-native-MTP family.

**Head-axis KV residency × assistant-MTP — COMPOSES on a real 31B (Gemma-4-31B-it Q6_K, bs2 GPU1,
2026-06-30).** This is the first 31B-specific MTP-composition cell (the §5b matrix above was A4B-only) and
the first over a *genuinely spilled* head-split KV. `headaxis-mtp-31b-bs2.sh` boots the official
`google_gemma-4-31B-it-Q6_K.gguf` at 128k with **f16 KV** and `--kv-residency-mode auto --vram-target 7000`,
forcing the dense global cache to split GPU/host on the **head axis** (M2/PolyKV residency — the path the
ineligible `window` mode falls back to on Gemma's iSWA caches, cf. §5 POSITION_WINDOW). Engagement is real,
not vacuous: GPU1 sits at **33.4 GB** (ref) / **33.9 GB** (cand) vs ~40 GB for full f16-resident, i.e.
~6 GB of KV is held in pinned host RAM and lifted on read. ref-plain (head-split) vs cand-MTP (head-split +
`gemma-4-31B-it-assistant.Q8_0` drafter, `--spec-type draft-assistant -ngld 99`) → **`real_frac=0`,
`real_disagreements=0`, `frac_full_agree=1.0`** (PASS): the assistant draft/verify path is logit-identical
even when half the global KV is dequant-on-lifted from host. (`mean_tv_advisory=0.44` is the known
empty-top-logprobs artifact for spec-accepted tokens, not a divergence — `real_frac` is the gate.) Closes
the open "did we validate 31B on the head axis?" question: previously 31B was proven with MTP decode/accept
(§1/§3) and sparse-V long-ctx (#566), but the head-axis residency × MTP cell had only been run on A4B.

### 5a. PolyKV × NextN-MTP — COMPOSES (Qwen3.5-9B Q8_0, solidpc, 2026-06-17)

Overturns the plan's GAP-5 "architecturally exclusive (parallel-1)". Native MTP is `n_seq`-aware by
construction (`common_speculative_impl_draft_mtp` does `smpls.resize(n_seq)` +
`backend_chains.assign(n_seq)`; `server-context.cpp:1066` inits spec with `n_parallel`), so it runs
per-slot under `--parallel`. The "--parallel 1" limit was a property of our *retired* W2/W3b CKPT
self-spec, not the native path. Probe `.opencoti/polykv-mtp-probe.sh`:
`--parallel 3 --kv-unified --no-cache-idle-slots --spec-type draft-mtp -ctk/-ctv/-ctkd/-ctvd q8_0`.

- boot OK; spec engaged (MTP draft ctx 519 MiB; checkpoints max=32, min-spacing=256).
- 2 agents share a read-only prefix (P=873) via SharedKVPool **with draft-mtp on**: both recall the
  needle, isolation holds (A=Alice¬Bob / B=Bob¬Alice), counted-once (each `cache_n=873≈P`,
  `prompt_n=36≈suffix` → prefix entered KV once, seq-shared across 3 slots), hook fired ≥2.
- MTP actively drafting during the shared decodes: slots 1 & 2 `draft acceptance = 0.19 (4/21)`.
- **Re-verified 2026-06-18 on the post-#493/#485 binary** (DSO `a8ddcd82`, host 13:11): byte-for-byte
  identical PASS — boot OK, spec engaged (MTP draft ctx 519 MiB), all Phase-2 asserts green (P=873,
  both recall needle, isolation holds, counted-once, hook ≥2), MTP drafting+accepting (slots 1 & 2
  4/21) during the shared decodes. Confirms #485's two *shared* `server-context.cpp` spec hooks are
  inert for native NextN: `spec_assistant_fit` skip is `gemma4-assistant`-specific, and the
  `--kv-unified`-when-`--parallel>1` boot guard is satisfied by construction (SharedKVPool forces
  `--kv-unified`). #493's fattn D512-VEC case is unreachable here (Qwen is head_dim 128 ≠ 512). So the
  #469 composition scope is closed on the current binary; the only NextN remainder is the bs2-blocked
  256k-stream / genuine inter-chunk DCA cell (#431/#439).

Flag note: the 0.10.1→0.10.3 bump renamed `--no-clear-idle` → `--no-cache-idle-slots` (bug-500;
runbook G29). The PolyKV mechanism (0072) is unchanged — only the residency boot flag moved.

---

## 6. Provenance

- Source markers: `// opencoti F5 M6-S4 mtp` (gemma), `// opencoti F5 M6-S5 nextn-mtp` (qwen).
- Patches: `0079-gemma-assistant-mtp` (gemma assistant, #423 — restored onto the 0.10.3 base; the
  pre-bump `0074`/`0076`/`0077` captures are RETIRED, #464). Qwen NextN is upstream-native
  (`--spec-type draft-mtp`) — no source capture beyond the W1 loader fix. Snapshot-diff (23 files, 0
  index lines), reverse+forward byte-identical, registered in `docs/protocols/UPSTREAM_SYNC.md`.
- Build integrity (bug-460): every server boot aborted in `common_params_parser_init`
  (`arg.cpp:2200`, `ggml_type_name(params.cache_type_v)`) because the incremental cosmocc make did not
  track the `common.h` header dep, leaving a `common_params` layout split across TUs after W3b added
  `use_checkpoints` — the consumer read an uninitialized `ggml_type` ≥ `GGML_TYPE_COUNT`. Fixed by a
  clean host rebuild (`rm -rf o/` + `OPENCOTI_NO_CCACHE=1 build:llamafile:make`). The DSO was never at
  fault; the q1_0-string proxy was a red herring (Q1_0 has no CUDA kernel, so its name is absent from a
  correct DSO too). Both `/root/ggml-cuda.so` paths remain byte-identical (`67e3171b…`).
- Upstream convergence: gemma assistant MTP restored onto the 0.10.3 bump as `0079` (#423, DONE);
  qwen CKPT rollback ports PR#19493 (enum names match upstream for clean re-sync).