Clarify 26B B70 Linux benchmarks and Wondernuttz custom OpenVINO fork

README.md

@@ -14,6 +14,31 @@ tags:
 pipeline_tag: image-text-to-text
 ---
 
+<!-- gemma26-public-summary:start -->
+## Gemma 4 26B-A4B at a glance — Wondernuttz's custom OpenVINO fork
+
+- **Tested platform:** one **Intel Arc Pro B70, 32 GB VRAM, running Linux**. These are not Windows or 12 GB GPU benchmark results.
+- **Custom runtime:** [Wondernuttz's OpenVINO fork](https://github.com/Wondernuttz/openvino/tree/arc-xe2-gemma4-pa-2026.4), branch `arc-xe2-gemma4-pa-2026.4`, with matching OpenVINO GenAI. The fork combines custom Gemma/Arc optimizations with credited Intel upstream fixes.
+- **Model format:** OpenVINO INT4 weights, already compressed. Use OpenVINO GenAI, not direct Transformers model loading. Hugging Face Hub is used to download the files.
+- **Uncached prompt processing:** **7,435 tokens/s at 6,622 input tokens; 6,866 at 15,872; 4,767 at 24,576.** These are Heretic reference-model results, with prefix reuse OFF and compiled kernels warmed—not cached-prompt or cold-start speeds.
+- **Response generation:** about **111 tokens/s on short prompts** and **93 tokens/s after 24K input** in that reference test. The lookup optimization targets prefill; no meaningful decode gain is claimed.
+- **Memory:** roughly **26.5 GiB sampled peak card memory at 24K input** in the reference test, including an 8 GiB KV-cache allocation. Download size is not runtime VRAM usage; execution buffers and KV cache also need memory. A 12 GB GPU minimum is **not established**.
+- **Context:** 24K text checks are documented in the release report. Separate 32K reference probes are not a blanket per-model guarantee. A **131K RoPE lookup table is not proof of usable 131K context**. Reserve room for generated output when setting a total-token limit.
+- **Other devices:** CPU, Windows, other Arc cards, and native vision/audio performance are not validated by these B70 Linux text benchmarks.
+
+**This checkpoint:** passed the documented 24K text/recall/cache release gates, with per-model RP caveats. The headline speeds and memory above belong to the **Heretic reference model, not a throughput benchmark of this exact fine-tune**.
+
+**Getting the speedup:** use the matching patched runtime—not just `pip install -U openvino-genai`. Set `MOE_USE_GROUPED_GEMM_PREFILL=1` and `MOE_GROUPED_BINARY_LOOKUP=1` before pipeline creation, along with the documented scheduler settings. A stock wheel cannot gain this patch from environment variables alone. Existing valid exports do not need recompression.
+
+**[Setup and full benchmark](https://github.com/Wondernuttz/openvino/blob/7b27ac8eb88faec682d4c96dfba749b93df32285/WONDERNUTTZ_GEMMA4_PREFILL_20260906.md)** · **[Per-model quality checks and known limitations](https://github.com/Wondernuttz/openvino/blob/7b27ac8eb88faec682d4c96dfba749b93df32285/WONDERNUTTZ_GEMMA4_RELEASE_GATE_20260906.md)**
+
+These are bounded tests, not an exhaustive intelligence or stability certification; known GPU fault/teardown caveats remain documented. Original model/merge authors retain their credits below. Wondernuttz's work here is the OpenVINO conversion, custom runtime integration/optimization, and B70 testing.
+<!-- gemma26-public-summary:end -->
+
+
+<details>
+<summary>Detailed September runtime notes, configuration, and validation history</summary>
+
 <!-- prefill-runtime-20260906:start -->
 ## September 6 runtime update: faster long-prompt processing
 
@@ -51,6 +76,9 @@ Correction to older tuning attribution: `MOE_MICRO_GEMM_N_HINT` is bypassed by t
 Original base-model, fine-tune, merge credits and licenses below are unchanged. Wondernuttz's contribution is the OpenVINO conversion/runtime work and testing.
 <!-- prefill-runtime-20260906:end -->
 
+</details>
+
+
 
 > **OpenVINO INT4 AWQ** - asymmetric 4-bit weight compression, verified group size 64.