Add measured MTP draft-acceptance metrics (77.7% @128k , 84.9% @262k prod, 93-96% code) + acceptance chart; update manifest

MTP_QUANT_MANIFEST.json

@@ -10,5 +10,15 @@
     "n_dump_files": 17701,
     "damp_fraction": 0.01
   },
-  "simplification_note": "MTP forward replay skipped attention (treated post-attn as identity-residual input); RMSNorms, h_proj/e_proj, gate, and per-expert MLP path are real."
+  "simplification_note": "MTP forward replay skipped attention (treated post-attn as identity-residual input); RMSNorms, h_proj/e_proj, gate, and per-expert MLP path are real.",
+  "measured_draft_acceptance": {
+    "note": "Weighted draft acceptance from live vLLM speculative-decoding metrics (num_speculative_tokens=1). Output is verifier-exact regardless of acceptance; acceptance affects decode speed only.",
+    "num_speculative_tokens": 1,
+    "profiles": {
+      "general_chat_128k": {"accepted": 2354, "drafted": 3030, "weighted_acceptance": 0.777, "mean_accepted_length": 1.78, "windows": 5},
+      "production_262k": {"accepted": 15274, "drafted": 17991, "weighted_acceptance": 0.849, "mean_accepted_length": 1.85, "windows": 25},
+      "long_context_262k_research_variant": {"accepted": 319, "drafted": 357, "weighted_acceptance": 0.894, "mean_accepted_length": 1.89, "windows": 3}
+    },
+    "code_heavy_ext10_gate_adapter_assisted": {"weighted_acceptance_range": [0.93, 0.96]}
+  }
 }

MTP_QUANT_MANIFEST.md

@@ -11,3 +11,13 @@
 
 ## Note
 - MTP forward replay skipped attention (treated post-attn as identity-residual input); RMSNorms, h_proj/e_proj, gate, and per-expert MLP path are real.
+
+## Measured draft acceptance (post-release, num_speculative_tokens=1)
+Weighted draft acceptance from live vLLM spec-decode metrics. Output is verifier-exact regardless — acceptance affects decode speed only.
+
+| Profile | Accepted / drafted | Weighted acceptance | Mean accepted length |
+|---|---:|---:|---:|
+| General chat, 128k | 2354/3030 | 77.7% | 1.78 |
+| Production, 262k | 15274/17991 | 84.9% | 1.85 |
+| Long-context research variant, 262k | 319/357 | 89.4% | 1.89 |
+| Code-heavy (ext10 gate, adapter-assisted) | — | 93–96% | — |

README.md

@@ -42,7 +42,27 @@ Hardware: 2× **NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition** (sm_12
 | **This model (v2 GPTQ), 524k** | **85.52** | 155 ms | **+62% (1.62×)** |
 | **This model (v2 GPTQ), 128k single-stream** | **~111** | ~310 ms | **+110% (2.10×)** |
 
-(v1 RTN release was 83.97 tok/s @ 524k; v2 GPTQ adds ~+2% on top of that. Most of the speedup over base comes from MTP self-speculation; the GPTQ vs RTN delta is the MTP draft acceptance-rate improvement from better expert weights.)
+(v1 RTN release was 83.97 tok/s @ 524k; v2 GPTQ adds ~+2% on top of that. Most of the speedup over base comes from MTP self-speculation; the GPTQ vs RTN delta is the MTP draft acceptance-rate improvement from better expert weights — now **measured**, see below.)
+
+### MTP draft acceptance — measured
+
+The decode speedup above is entirely MTP self-speculation: each model forward drafts one extra token that the main model then verifies in the same step. The **draft acceptance rate** — the fraction of drafted tokens the verifier keeps — determines how much of the theoretical 2× is realized. These numbers were **measured post-release** from live vLLM speculative-decoding metrics (`vllm:spec_decode_num_accepted_tokens` / `_num_draft_tokens`, extracted by `scripts/summarize_spec_decode_metrics.py`), not estimated.
+
+![MTP draft acceptance by workload](charts/chart6_mtp_acceptance.png)
+
+| Serving profile | Windows | Accepted / drafted | **Weighted acceptance** | Mean accepted length |
+|---|--:|--:|--:|--:|
+| General chat, 128k | 5 | 2,354 / 3,030 | **77.7%** | 1.78 |
+| General chat, 262k (production) | 25 | 15,274 / 17,991 | **84.9%** | 1.85 |
+| Long-context research variant, 262k¹ | 3 | 319 / 357 | **89.4%** | 1.89 |
+| Code-heavy workload (strict ext10 gate)² | — | — | **93–96%** | — |
+
+- **`num_speculative_tokens = 1`**, so the ceiling for mean accepted length is 2.0; the production profile realizes **1.85 (92.5% of ceiling)**.
+- Acceptance tracks next-token predictability: free-form chat sits at **~78–85%**, structured/code-heavy generation climbs into the **low-to-mid 90s**.
+- **Acceptance affects speed only, never output quality.** Every accepted draft is verified by the full model, so emitted text is bit-identical to plain greedy decoding without MTP.
+
+¹ A shared-FP8 research variant (different shared-expert quant, same MTP head); small sample, included for range.
+² From the extended-coding reliability gate (candidate v5/v8 runs) and **adapter-assisted** — reflects a code-specialized runtime configuration, not the bare checkpoint. Listed separately for that reason.
 
 ### Concurrency profiles — real measurements
 
@@ -260,7 +280,7 @@ The full pipeline is reproducible in ~30 minutes total: ~25 min calibration capt
 
 - **Hardware**: validated only on 2× RTX PRO 6000 Blackwell Max-Q (sm_120). Should also work on RTX PRO 6000 Server, DGX Spark / GB10, and 8× H200 — same code path as base. **Without `--disable-custom-all-reduce`, Max-Q deadlocks at post-graph eager warmup.**
 - **TP**: TP=2 only. TP=1 OOMs on a single 96 GB GPU; TP≥4 hits an upstream W4A16 MoE scale-sharding bug ([vllm-project/vllm#41511](https://github.com/vllm-project/vllm/issues/41511)).
-- **MTP quality**: GPTQ pipeline used a *simplified* MTP forward that skipped the attention layer in calibration (used identity-residual). This is a known approximation chosen to avoid re-implementing DSV4 hybrid attention (CSA+HCA) in pure PyTorch — it captures the post-FFN-norm input distribution to each expert, but the attention's contribution to that distribution is approximated. The output produced by the served model is unaffected (the main model verifies every accepted draft); only the MTP draft accept rate is impacted. Lift on this hardware vs. v1 RTN: ~+2% decode TPS on long-context profile.
+- **MTP quality**: GPTQ pipeline used a *simplified* MTP forward that skipped the attention layer in calibration (used identity-residual). This is a known approximation chosen to avoid re-implementing DSV4 hybrid attention (CSA+HCA) in pure PyTorch — it captures the post-FFN-norm input distribution to each expert, but the attention's contribution to that distribution is approximated. The output produced by the served model is unaffected (the main model verifies every accepted draft); only the MTP draft accept rate is impacted — now measured at **77.7% @128k / 84.9% @262k production** (see *MTP draft acceptance* above). Lift on this hardware vs. v1 RTN: ~+2% decode TPS on long-context profile.
 - **`num_speculative_tokens`**: capped at 1 because DSV4-Flash ships exactly one MTP head (`num_nextn_predict_layers=1`). Higher values would not produce more draft tokens.
 - **Reasoning parser**: with `--reasoning-parser deepseek_v4`, model output is split into `content` and `reasoning_content` — applications that read only `content` will see empty strings for "thinking" responses. Adjust accordingly.