Document tested B70 wide-query prefill gain, setup and quality scope

README.md

@@ -15,24 +15,20 @@ pipeline_tag: image-text-to-text
 ---
 
 <!-- gemma26-public-summary:start -->
-## Gemma 4 26B-A4B at a glance — Wondernuttz's custom OpenVINO fork
+## Gemma 4 26B-A4B — custom OpenVINO runtime update
 
-- **Tested platform:** one **Intel Arc Pro B70, 32 GB VRAM, running Linux**. These are not Windows or 12 GB GPU benchmark results.
-- **Custom runtime:** [Wondernuttz's OpenVINO fork](https://github.com/Wondernuttz/openvino/tree/arc-xe2-gemma4-pa-2026.4), branch `arc-xe2-gemma4-pa-2026.4`, with matching OpenVINO GenAI. The fork combines custom Gemma/Arc optimizations with credited Intel upstream fixes.
-- **Model format:** OpenVINO INT4 weights, already compressed. Use OpenVINO GenAI, not direct Transformers model loading. Hugging Face Hub is used to download the files.
-- **Uncached prompt processing:** **7,435 tokens/s at 6,622 input tokens; 6,866 at 15,872; 4,767 at 24,576.** These are Heretic reference-model results, with prefix reuse OFF and compiled kernels warmed—not cached-prompt or cold-start speeds.
-- **Response generation:** about **111 tokens/s on short prompts** and **93 tokens/s after 24K input** in that reference test. The lookup optimization targets prefill; no meaningful decode gain is claimed.
-- **Memory:** roughly **26.5 GiB sampled peak card memory at 24K input** in the reference test, including an 8 GiB KV-cache allocation. Download size is not runtime VRAM usage; execution buffers and KV cache also need memory. A 12 GB GPU minimum is **not established**.
-- **Context:** 24K text checks are documented in the release report. Separate 32K reference probes are not a blanket per-model guarantee. A **131K RoPE lookup table is not proof of usable 131K context**. Reserve room for generated output when setting a total-token limit.
-- **Other devices:** CPU, Windows, other Arc cards, and native vision/audio performance are not validated by these B70 Linux text benchmarks.
+- **Tested hardware:** one **Intel Arc Pro B70, 32 GB VRAM, on Linux**. No B50, Windows, CPU, or offload performance claim.
+- **Latest uncached 24K prefill:** **4,841 → 5,653 tokens/s (+16.8%)** in a matched Heretic-reference comparison; the clean deployment build confirmed **5,667 tokens/s**. Prefix reuse was OFF, kernels were warmed, and PP means input tokens divided by time to first token. This is not a cache-hit benchmark or a 6K+ result at 24K.
+- **Shorter prompts / generation:** the matched test retained about **7.4K PP at 6,622 input tokens** and **112 output tokens/s on short prompts**. These are reference-model figures, not guaranteed speeds for this exact fine-tune or every RP request.
+- **Memory:** about **26.8 GiB sampled peak card usage at 24K** with an 8 GiB KV pool. Download size is not total VRAM. A 12 GB minimum has not been established.
+- **Custom fork required for this gain:** [Wondernuttz/OpenVINO, wide-query commit](https://github.com/Wondernuttz/openvino/commit/80c431dfd426bcff914886d6f9a9cef83b44b0c4), plus matching OpenVINO GenAI/tokenizers. This combines custom tuning with credited Intel/OpenVINO kernels and fixes; it is not an unmodified stock wheel.
+- **Inference:** use the complete OpenVINO export with OpenVINO GenAI, not direct Transformers loading. The tested 26B/B70 profile uses PA, DQ128, U4 KV, cache8 GiB, batch16384 and one sequence. Set `GEMMA_MIXED_512_TILE=wideq` before Python starts, alongside `MOE_USE_GROUPED_GEMM_PREFILL=1` and `MOE_GROUPED_BINARY_LOOKUP=1`. RP prefix caching may stay enabled. See [full instructions and rollback](https://github.com/Wondernuttz/openvino/blob/80c431dfd426bcff914886d6f9a9cef83b44b0c4/WIDEQ_24K_20260906.md).
+- **Quality / scope:** all 17 matched Heretic output strings were identical; Chimera passed separate 24K recall and prefix-update checks. The clean kernel passed 24 repeated U4 reference tests with bit-identical repeats. The old baseline showed U4 variability, and an extra OFF/ON comparison gate did not pass; that caveat remains documented. This is bounded validation, not a universal intelligence guarantee.
+- **Context and installation:** the live Heretic bots received the new kernel while keeping their existing 16K context and settings. Valid exports need **no recompression**. Rebuild custom runtime wheels/container to get the new kernel—`git pull` alone does not replace OpenVINO inside an image. No new prebuilt image is supplied. 131K RoPE LUT coverage is not validated 131K usable context; reserve output space.
 
-**This checkpoint:** passed the documented 24K text/recall/cache release gates, with per-model RP caveats. The headline speeds and memory above belong to the **Heretic reference model, not a throughput benchmark of this exact fine-tune**.
+**This checkpoint:** the headline result is the Heretic reference benchmark. This fine-tune has not received a fresh per-checkpoint wide-query qualification in this update; its existing conversion and model-specific profile are unchanged.
 
-**Getting the speedup:** use the matching patched runtime—not just `pip install -U openvino-genai`. Set `MOE_USE_GROUPED_GEMM_PREFILL=1` and `MOE_GROUPED_BINARY_LOOKUP=1` before pipeline creation, along with the documented scheduler settings. A stock wheel cannot gain this patch from environment variables alone. Existing valid exports do not need recompression.
-
-**[Setup and full benchmark](https://github.com/Wondernuttz/openvino/blob/7b27ac8eb88faec682d4c96dfba749b93df32285/WONDERNUTTZ_GEMMA4_PREFILL_20260906.md)** · **[Per-model quality checks and known limitations](https://github.com/Wondernuttz/openvino/blob/7b27ac8eb88faec682d4c96dfba749b93df32285/WONDERNUTTZ_GEMMA4_RELEASE_GATE_20260906.md)**
-
-These are bounded tests, not an exhaustive intelligence or stability certification; known GPU fault/teardown caveats remain documented. Original model/merge authors retain their credits below. Wondernuttz's work here is the OpenVINO conversion, custom runtime integration/optimization, and B70 testing.
+Earlier binary-lookup results (including 7,435 PP at 6,622 and 6,866 PP at 15,872) remain historical measurements with their original profiles. The wider-query result above is an additional cached/chunked-prefill improvement, not a replacement claim for every model or context size. Original model/merge credits, license and detailed historical notes below remain unchanged.
 <!-- gemma26-public-summary:end -->