docs: update templatefix test notes

README.md

@@ -3,11 +3,11 @@ license: gemma
 library_name: gguf
 base_model: google/gemma-4-26B-A4B-it
 base_model_relation: quantized
-model_name: Gemma-4-26B-A4B-it-Cerebellum-v6-GGUF
+model_name: Gemma-4-26B-A4B-it-Cerebellum-v6.1-templatefix-GGUF
 model_creator: google
 model_type: gemma4
 quantized_by: deucebucket
-pipeline_tag: image-text-to-text
+pipeline_tag: text-generation
 tags:
   - GGUF
   - gemma4
@@ -18,239 +18,149 @@ tags:
   - imatrix
   - moe
   - 3-bit
-  - conversational
-  - multimodal
-  - vision
+  - templatefix
 ---
 
-# Gemma 4 26B-A4B-it — Cerebellum v6 GGUF
+# Gemma 4 26B-A4B-it Cerebellum GGUF
 
-> **Numbers under audit (2026-05-08)** — an internal review found the v6 benchmark numbers below need to be re-measured against the same protocol used for v1–v4. A clean re-run with audited wrong-answers and per-question JSONLs is underway. The GGUF file itself is unchanged — this is a measurement issue, not a model issue. Treat the table as preliminary until corrected numbers replace it.
+This repository contains GGUF builds derived from
+`google/gemma-4-26B-A4B-it`.
 
-Cerebellum v6 is an ablation-guided mixed-precision GGUF quantization of [google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it).
+## 2026-05-22 Update
 
-This is a 26B-parameter MoE model with 4B active parameters per token, 128 experts per layer, and 30 layers. This release uses tensor-level precision overrides selected from 140+ ablation experiments across six internal iterations, including per-layer MoE router surgery.
+Added:
 
-**This model supports vision** when used with the included mmproj file. See [Vision Support](#vision-support) below.
+```text
+gemma-4-26B-A4B-it-cerebellum-v6.1-templatefix.gguf
+sha256: d24229facdef8360a7ffa8b37a50e1de636b9139a5eba0efe899828e45ae7989
 
-## At a Glance
-
-| | |
-|---|---|
-| **File** | `gemma-4-26B-A4B-it-cerebellum-v6.gguf` |
-| **mmproj** | `mmproj-google_gemma-4-26B-A4B-it-f16.gguf` |
-| **Size** | 11.7 GB (backbone) + 1.14 GB (mmproj) |
-| **Base model** | `google/gemma-4-26B-A4B-it` |
-| **Base quant** | Q3_K_M with [bartowski's imatrix](https://huggingface.co/bartowski/google_gemma-4-26B-A4B-it-GGUF) |
-| **Format** | GGUF, mixed precision |
-| **Test hardware** | RTX 3090, llama.cpp |
-
-## Benchmarks
-
-| Benchmark | Result |
-|-----------|:------:|
-| WikiText PPL | 12,054 |
-| HumanEval pass@1 | 72.0% |
-| ARC-Challenge | 95.6% |
-| HellaSwag | 84.7% |
-| MMLU-Redux | 71.2% |
-
-All results measured locally on an RTX 3090 with llama.cpp. PPL was measured on the WikiText-2 test set with 2048 context and 128 chunks.
-
-PPL is high in absolute terms for this model. This appears consistent across Gemma 4 26B quant levels tested locally and may reflect the model's MoE routing behavior on WikiText specifically.
-
-## What Changed: v1 Through v6
-
-Each version added a new layer of ablation data. The method is always the same: change one thing, measure PPL, keep it only if it helps.
-
-| Version | PPL | HumanEval | What Changed |
-|---------|-----|-----------|-------------|
-| v1 | 20,614 | 65.2% | Group-level ablation: 5 tensor groups tested at Q2_K |
-| v2 | 19,826 | 65.9% | + attn_q per-layer ablation (30 layers tested, 9 promoted to Q5_K) |
-| v3 | 19,826 | 67.1% | + PLE protection (norms/scales forced to F32) |
-| v4 | 12,614 | 69.5% | + ffn_up per-layer ablation + precision rebalance |
-| v5 | 12,356 | 71.3% | + attn_k reverse ablation (30 layers tested, 7 promoted to Q3_K) |
-| **v6** | **12,054** | **72.0%** | + MoE router surgery: layer 8 ffn_gate_inp F32→Q8_0 |
-
-## How Cerebellum Works
-
-Cerebellum assigns quantization precision per tensor based on measured impact. Each tensor group and individual layer is tested by changing its precision and measuring perplexity. Only changes that improve or maintain quality are kept.
-
-### Group Ablation
-
-Each tensor category was tested at Q2_K and measured by PPL impact:
-
-| Group | Tensors | PPL Delta | Action |
-|-------|---------|-----------|--------|
-| attn_q | 30 | +13.4% | Per-layer testing (9 layers need Q5_K) |
-| ffn_gate | 30 | -1.2% | Left at Q3_K |
-| expert_gate_up | 30 | -5.5% | Set to Q2_K |
-| attn_k | 30 | -12.1% | Per-layer testing (7 layers benefit from Q3_K) |
-| ffn_up | 30 | -18.2% | Set to Q2_K |
-
-Three of five tested groups had lower PPL at Q2_K — meaning Q3_K_M was using bits on tensors that don't need them.
-
-### Layer Ablation
-
-Groups with mixed results were tested per layer:
-
-- **attn_q**: All 30 layers tested individually at Q2_K. 9 layers exceeded the sensitivity threshold and stay at Q5_K. The other 21 tolerate Q2_K.
-- **attn_k**: All 30 layers tested individually. 7 layers showed PPL improvement when promoted from Q2_K to Q3_K (layer 23: -3.8%, layer 18: -2.8%). 4 layers (5, 11, 16, 29) were confirmed better at Q2_K.
-
-### MoE Router Surgery (New in v6)
-
-llama-quantize ignores `--tensor-type-file` overrides for `ffn_gate_inp.weight` (MoE router) tensors. We built [gguf_tensor_surgery.py](https://github.com/deucebucket/osmosis/blob/master/scripts/gguf_tensor_surgery.py) to recast individual tensors directly in the GGUF file.
-
-All 30 router layers were tested individually at Q8_0 (F32→Q8_0):
-
-| Layer | PPL | Delta | Category |
-|-------|------|-------|----------|
-| 8 | 12,054 | -2.4% | Best universal candidate |
-| 10 | 11,872 | -3.9% | Best PPL but regresses HumanEval (-9.7%) |
-| 6 | 11,988 | -3.0% | Win (not stacked — routing compensation) |
-| 9 | 12,044 | -2.5% | Win (not stacked) |
-| 12 | 12,041 | -2.5% | Win (not stacked) |
-| 23 | 12,052 | -2.5% | Win (not stacked) |
-| 0 | 12,974 | +5.0% | Sensitive |
-| 1 | 13,525 | +9.5% | Very sensitive |
-| 2 | 13,239 | +7.1% | Sensitive |
-| 4 | 13,047 | +5.6% | Sensitive |
-
-**Why layer 8 and not layer 10:** Layer 10 had the best PPL improvement (-3.9%), but full HumanEval testing showed it regresses code generation from 71.3% to 61.6%. Layer 10's router controls routing to code-relevant experts — degrading it hurts coding while helping general perplexity. Layer 8 improves PPL (-2.4%) AND HumanEval (+0.7%) with no regressions on any benchmark.
-
-**Router stacking doesn't work:** Combined demotion of even the top 3 layers worsens PPL vs baseline. The model compensates for one degraded router but not multiple simultaneously. This is a routing compensation effect specific to MoE architectures.
-
-**Precision curve for layer 8's router:**
-
-| Precision | PPL | Delta |
-|-----------|------|-------|
-| F32 (default) | 12,356 | — |
-| Q8_0 | 12,054 | -2.4% |
-| Q4_0 | 12,355 | ~0% |
-| Q6_K | 14,317 | +15.9% |
-| Q2_K | 14,482 | +17.2% |
-
-Q8_0 is the only precision that improves PPL. K-quant formats (Q6_K, Q2_K) use 256-element super-blocks with sub-block scales — this structure disrupts the router's fine-grained expert selection. Q8_0's simpler per-block rounding acts as beneficial regularization.
-
-### Final Precision Map (v6)
-
-| Tensor Type | Precision | Count | Rationale |
-|-------------|-----------|-------|-----------|
-| attn_q (9 sensitive layers) | Q5_K | 9 | Layer-validated critical |
-| attn_q (remaining) | Q2_K | 21 | Group-level demotable |
-| attn_k (7 promoted layers) | Q3_K | 7 | Reverse ablation: improve when promoted |
-| attn_k (remaining) | Q2_K | 23 | Group-level demotable |
-| ffn_up | Q2_K | 30 | Group PPL delta: -18.2% |
-| expert_gate_up | Q2_K | 30 | Group PPL delta: -5.5% |
-| ffn_gate | Q3_K | 30 | Tolerant (-1.2%) |
-| ffn_gate_inp layer 8 (router) | Q8_0 | 1 | Per-layer surgery: -2.4% PPL, +0.7% HumanEval |
-| ffn_gate_inp (router, other) | F32 | 29 | Group PPL delta: +30.7% when crushed |
-| Norms, scales | F32 | 392 | Structural — always full precision |
+gemma-4-26b-a4b-it.mmproj.gguf
+sha256: b762c43119ebdc3e3c36d929d958e827fac35b03278dda9203f87131aee1f185
+```
 
-91 tensor-level overrides + 1 surgical router recast on top of Q3_K_M base.
+The v6.1 file keeps the v6 tensor allocation and updates GGUF/runtime-facing
+metadata for Gemma 4 chat-template use. The update was tested with
+`llama-server --jinja --reasoning auto` and request-level no-thinking controls.
 
-## Vision Support
+Older files in this repository are retained for reproducibility.
 
-This model is multimodal — it can process images alongside text. Vision requires two files used together:
+## Tested Runtime
 
-- `gemma-4-26B-A4B-it-cerebellum-v6.gguf` — the text backbone (this file)
-- `mmproj-google_gemma-4-26B-A4B-it-f16.gguf` — the vision encoder + projector (1.14 GB)
+Runtime used for the 2026-05-22 templatefix checks:
 
-The image token `<|image|>` (token ID 258880) is already in the vocabulary. No metadata changes needed.
+```text
+llama.cpp fork: https://github.com/deucebucket/llama.cpp
+branch: cerebellum/gemma4-runtime-fixes
+fork commit: ded491334 fix: harden Gemma 4 server budgets
+base build: b8930-59fa0b455
+```
 
-### Usage with llama-server
+Server shape used locally:
 
 ```bash
 llama-server \
-  -m gemma-4-26B-A4B-it-cerebellum-v6.gguf \
-  --mmproj mmproj-google_gemma-4-26B-A4B-it-f16.gguf \
+  --model gemma-4-26B-A4B-it-cerebellum-v6.1-templatefix.gguf \
+  --mmproj gemma-4-26b-a4b-it.mmproj.gguf \
+  --n-gpu-layers 99 \
+  --ctx-size 65536 \
+  --parallel 1 \
+  --flash-attn on \
+  --cache-type-k q8_0 \
+  --cache-type-v q8_0 \
   --jinja \
-  --reasoning off \
-  --reasoning-budget 0 \
-  -ngl 99 \
-  -c 4096
+  --reasoning auto \
+  --media-path /tmp/
 ```
 
-**Required flags:**
-- `--mmproj` — loads the vision encoder. The mmproj filename starts with `mmproj-` so it also works with `--mmproj-auto` auto-download if placed in the same directory.
-- `--jinja` — enables the Gemma 4 chat template (embedded in the GGUF; required for correct formatting)
-- `--reasoning off --reasoning-budget 0` — disables thinking mode which can cause infinite loops without dedicated reasoning tokens
+Normal no-thinking requests used:
 
-### Usage with curl
-
-```bash
-curl http://localhost:8080/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "gemma4-cerebellum",
-    "messages": [
-      {
-        "role": "user",
-        "content": [
-          {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}},
-          {"type": "text", "text": "What is shown in this image?"}
-        ]
-      }
-    ]
-  }'
+```json
+{
+  "chat_template_kwargs": {"enable_thinking": false},
+  "thinking_budget_tokens": 0
+}
 ```
 
-### Usage with ollama
+Bounded-thinking smoke requests used `thinking_budget_tokens: 128`.
 
-Create a `Modelfile`:
+## 2026-05-22 Templatefix Test Artifacts
 
-```dockerfile
-FROM ./gemma-4-26B-A4B-it-cerebellum-v6.gguf
-FROM ./mmproj-google_gemma-4-26B-A4B-it-f16.gguf
-TEMPLATE {{ .Prompt }}
+Creative-writing smoke files:
+
+```text
+creative_eval_20260522/regular_v6_1_templatefix_creative_summary.json
+creative_eval_20260522/regular_v6_1_templatefix_creative_rerun_longcaps_summary.json
 ```
 
-Then:
-```bash
-ollama create gemma4-cerebellum -f Modelfile
-ollama run gemma4-cerebellum
+Non-coding tool-use files:
+
+```text
+agentic_eval_20260522/README.md
+agentic_eval_20260522/regular_v6_1_noncoding_agentic_tools_strict_summary.json
 ```
 
-### Technical notes
+Observed 2026-05-22 results from those artifacts:
 
-- The vision encoder has 27 layers with 1152 hidden dimension, projecting to 2816 (matching the text model's embedding dimension).
-- Image resolution: native input size 224×224, patches of 16×16. Dynamic resolution supported.
-- The mmproj was converted using llama.cpp's `convert_hf_to_gguf.py` from the original Google model and is redistributed under the Apache 2.0 license. Conversion credit: [bartowski](https://huggingface.co/bartowski).
-- Vision works out of the box — no special tokens, metadata edits, or re-quantization needed.
+| Area | Harness | Observed result |
+|---|---|---|
+| No-thinking output channel | six creative prompts | `reasoning_len=0` in recorded outputs |
+| Template leakage markers | six creative prompts | no `<think>` marker or template marker recorded by checker |
+| Creative long-cap rerun | four prompts rerun after initial length caps | four stop finishes in rerun summary |
+| Non-coding tool workflow | three strict OpenAI-style tool tasks | `schedule_strict`, `release_notes_strict`, `creative_brief_strict` listed in `pass_cases` |
 
-## Usage
+The non-coding tool harness used mock tools named `list_calendar`,
+`create_calendar_hold`, `search_notes`, `save_note`, and `add_task`. It did not
+test code editing.
 
-### llama.cpp
+## Historical Same-Repo Benchmark Artifacts
 
-```bash
-# Text only
-./llama-server -m gemma-4-26B-A4B-it-cerebellum-v6.gguf -ngl 99 -c 4096
+The following benchmark artifacts are from the earlier v6 line and the local
+Q3_K_M baseline. They are included as historical same-project measurements, not
+as new v6.1 measurements.
+
+| Artifact set | ARC-Challenge | HellaSwag | MMLU-Redux | HumanEval note |
+|---|---:|---:|---:|---|
+| `q3km_baseline_*` | 95.2218 | 86.5664 | 73.6667 | `q3km_baseline_humaneval_results.json`: 62.2 pass@1 |
+| `cerebellum_v6_*` | 95.5631 | 84.55 | 71.3333 | v6 HumanEval artifacts are retained but marked for audit in local notes |
 
-# With vision
-./llama-server -m gemma-4-26B-A4B-it-cerebellum-v6.gguf --mmproj mmproj-google_gemma-4-26B-A4B-it-f16.gguf --jinja --reasoning off --reasoning-budget 0 -ngl 99 -c 4096
+For Gemma 4 HumanEval/EvalPlus, the local protocol now uses chat completions,
+not raw completions:
+
+```text
+llama-server --jinja --reasoning auto
+chat_template_kwargs: {"enable_thinking": false}
+thinking_budget_tokens: 0
+BENCH_WORKERS=1
 ```
 
-### ollama
+## Files and Provenance
 
-```bash
-ollama create gemma4-cerebellum -f Modelfile
-ollama run gemma4-cerebellum
+Main v6.1 GGUF:
+
+```text
+source base: google/gemma-4-26B-A4B-it
+quantization family: mixed-precision GGUF
+recipe lineage: Cerebellum v6 tensor allocation
 ```
 
-Fits in 24 GB VRAM at full GPU offload with room for 4K context.
+Matching mmproj:
+
+```text
+gemma-4-26b-a4b-it.mmproj.gguf
+```
 
-## Technical Details
+## Notes
 
-- **Architecture**: Gemma 4 26B — 26B total params, 4B active per token, 128 experts/layer, 30 layers
-- **Base quant**: Q3_K_M with bartowski imatrix
-- **Ablation experiments**: 140+ across 6 iterations (including 30-layer router surgery)
-- **Quantizer**: llama.cpp `llama-quantize` with `--tensor-type-file` overrides + `gguf_tensor_surgery.py` for router recast
-- **Hardware**: RTX 3090 (24 GB VRAM)
-- **Vision**: See [Vision Support](#vision-support) for details
+- The 2026-05-22 tests were run on local `llama-server`.
+- The opencode coding-agent test is not used as a model-card result. In one
+  internal White and Black project run, the model connected through the harness
+  and ran a Godot test, then produced malformed edit-tool calls.
+- The creative-writing checks are smoke tests plus mechanical checks, not a
+  human preference benchmark.
+- The non-coding tool checks use mocked tools and fixed task definitions.
 
 ## Credits
 
-- **Base model**: [Google Gemma Team](https://huggingface.co/google/gemma-4-26B-A4B-it)
-- **Imatrix**: [bartowski](https://huggingface.co/bartowski/google_gemma-4-26B-A4B-it-GGUF)
-- **mmproj conversion**: [bartowski](https://huggingface.co/bartowski/google_gemma-4-26B-A4B-it-GGUF)
-- **Method & quantization**: [deucebucket/osmosis](https://github.com/deucebucket/osmosis) — Cerebellum pipeline
+- Base model: Google Gemma Team, `google/gemma-4-26B-A4B-it`
+- GGUF/runtime: llama.cpp
+- Quantization and local test artifacts: deucebucket Cerebellum workflow