docs: PolyKV cross-arch evals (#710) + bug-2203 hybrid gotcha + pool-prefill token-array recipe — sync from git 9969839543

USAGE.md

@@ -67,8 +67,8 @@ trusting aggressive KV tiers on anything else.
 
 opencoti-llamafile is **upstream llamafile 0.10.3 plus an additive
 patch series** (`patches/` in the HF repo,
-`vendors/patches/llamafile/` in the git repo — ~79 patches, numbered
-`0006`–`0133`). Three properties are contractual:
+`vendors/patches/llamafile/` in the git repo — ~81 patches, numbered
+`0006`–`0135`). Three properties are contractual:
 
 1. **Off means off.** Every opencoti feature is opt-in behind a flag,
    env var, or per-request JSON field. With no opencoti flags set, the
@@ -293,6 +293,65 @@ slot never decodes, so its prefix never slides — every re-attach is
 free. Note the capacity row is about per-session speed, not just
 aggregate: 12 pooled sessions each decode faster than 8 private ones.
 
+**Cross-architecture results** (RTX 6000 96GB, P≈5073, GEN=256,
+A/B/A naive/shared/naive): the pool is validated on all three
+attention architectures, and the memory axis is
+architecture-independent (~6.8–6.9× at N=8 — it counts cells, not
+attention math).
+
+| axis | Qwen2.5-14B-1M Q8_0 (pure full attention) | Qwen3.6-27B-Omnimerge-v4 Q4_K_M (hybrid GDN + NextN MTP n=3) |
+|---|---|---|
+| KV cells (N=8) | **6.78×** (~406 vs ~9 agents on a fixed 8192-cell buffer) | **6.89×** (~304 vs ~9 agents) |
+| batched decode N=8 | 358 → 403 tok/s (**+12.5%**) | 106 → 123 tok/s (**+15.6%**) |
+| batched decode N=24 | 412 → 742 tok/s (**+80%**; 17.2 → 30.9 tok/s per session) | 90 → 125 tok/s (**+39.7%**; 3.7 → 5.2 per session) |
+| shared-only sweep N=32/48/64 | 813 / 871 / 865 tok/s (plateau ~870 near N=48) | 126 / 124 / 124 tok/s (saturates by N≈24–32) |
+
+The shared-vs-naive decode gain **grows with N** on both. On
+hybrid/recurrent models (delta-net, mamba) the *absolute* aggregate
+saturates much earlier than on pure attention — the recurrent layers
+batch worse — so there the pool buys **concurrency capacity and
+memory**, not aggregate throughput past N≈24.
+
+**How to prefill the pool — use the common-prefix token array, not
+the document text.** Tokenizers merge across the document/suffix
+boundary (on the Qwen tokenizer the last prefix token fuses with the
+suffix start), so `tok(DOC)` can be one token longer than the common
+prefix the agents actually share — and a pool that is even one token
+longer than `shared_prefix_n_tokens` cannot be shared exactly. The
+correct client sequence:
+
+```jsonc
+// 1. tokenize the FULL agent prompts and compute
+//    P = min over agents of commonPrefixLen(tok(DOC), tok(DOC+suffix_i))
+// 2. prefill the pool slot with the token array itself (llama.cpp
+//    /completion accepts token arrays) — pool state == P on ANY tokenizer:
+{ "prompt": [/* tok(DOC+suffix_1)[:P] */], "id_slot": 0,
+  "cache_prompt": true, "n_predict": 1 }
+// 3. agents attach with prompts STRICTLY longer than P:
+{ "prompt": "<DOC + private suffix>", "id_slot": 1,
+  "shared_pool_slot": 0, "shared_prefix_n_tokens": P, … }
+```
+
+On attention-only models a text prefill happens to work (the ranged
+cell copy tolerates the extra token); on hybrid/recurrent targets it
+silently disables every share — see the gotcha below. The token-array
+prefill is correct everywhere.
+
+**Hybrid/recurrent gotcha (GDN / mamba / `qwen35moe`-class models).**
+A recurrent cache has one rolling state per sequence, not per-position
+cells, so a pool share is only possible as an **exact full-state**
+share. The server enforces this: the share engages only when
+`shared_prefix_n_tokens == pool state length` *and* the request prompt
+is strictly longer than the shared prefix; anything else logs
+`poly-kv-pool: hybrid/recurrent target needs exact full-state share …
+skipping share, full reprocess (bug-2203)` and falls back to a full
+(correct, slower) reprocess. If you see zero speedup on a hybrid
+model — or mass HTTP 500s at high N because N unshared full prompt
+copies overflow the unified KV — grep the server log for that WARN:
+it almost always means the pool was prefilled with text instead of
+the token array. (Older builds crashed outright here —
+`failed to remove sequence N with p0=…` — fixed by patch `0135`.)
+
 Sizing note: the pooled prefix pins P cells in **both** iSWA caches
 (global + SWA) for the pool's lifetime. Budget `-c` for pool prefix
 + N session windows + generation headroom, or long-running sessions
@@ -558,7 +617,7 @@ currently appear only in the server log.
 | **quant-KV** | — | K-only honors | ✅ (tiles dequant-on-lift) | ✅ | ✅ | ✅ (the win case) | ✅ (turbo+MTP is the top decode combo) | ✅ |
 | **auto-tier** | | — | ✅ (it *manages* spill) | ✅ | ✅ (probes in DCA state) | ✅ | ✅ | ✅ (sizing is eff-plan-aware) |
 | **rolling-KV** | | | — | ✅ | ✅ | ✅ | ✅ | ✅ (validated: window spill × RYS on hybrid) |
-| **PolyKV pool (SharedKVPool)** | | | | — | ✅ | ✅ | ✅ | ✅ (validated: 2-agent share gate × `--repeat-layers` on A4B) |
+| **PolyKV pool (SharedKVPool)** | | | | — | ✅ | ✅ | ✅ | ✅ (validated: 2-agent share gate × `--repeat-layers` on A4B; hybrid-GDN omnimerge × NextN MTP full gate, patch `0135`) |
 | **DCA** | | | | | — | ✅ | ✅ (dual-ctx) | ✅ (eff→src mapped) |
 | **sparse-attn** | | | | | | — | ✅ | ✅ |
 | **MTP** | | | | | | | — | ✅ (draft runs base stack; target keeps RYS) |