---
license: apache-2.0
base_model:
- unsloth/gemma-4-26B-A4B-it-GGUF
library_name: llama.cpp
tags:
- gguf
- llama-cpp
- gemma4
- speculative-decoding
- draft-model
- mtp
- multi-token-prediction
- dflash
- vision
- tater
- nothink
- chat-template
- tater-pick
- tater-nothink
- llama.cpp
pipeline_tag: image-text-to-text
---

# gemma-4-26B-A4B-it-GGUF Tater NoThink

This is a Tater NoThink test build of `unsloth/gemma-4-26B-A4B-it-GGUF`, bundled with compatible Multi-Token Prediction (MTP) and DFlash draft models for llama.cpp speculative decoding.

The main model weights and projector are from the upstream Unsloth GGUF release. The only intended change to the main text model is its embedded GGUF `tokenizer.chat_template` metadata:

- `gemma-4-26B-A4B-it-UD-Q4_K_M.gguf`

The template is patched to force:

```jinja
{%- set enable_thinking = false -%}
```

This makes the model prefer the no-thinking chat-template path even when a client accidentally exposes or passes a thinking flag.

## Included Files

- `gemma-4-26B-A4B-it-UD-Q4_K_M.gguf`: Tater NoThink patched GGUF.
- `mmproj-F16.gguf`: unchanged upstream vision projector.
- `gemma-4-26B-A4B-it-MTP-Q8_0.gguf`: Q8_0 MTP draft model converted from Google's official Gemma 4 assistant checkpoint. This is a sidecar draft model, not a standalone chat model.
- `gemma-4-26B-A4B-it-DFlash-Q8_0.gguf`: Q8_0 DFlash draft model from the exact Gemma 4 26B-A4B-it DFlash checkpoint. This is also a sidecar draft model, not a standalone chat model.

## Speculative Decoding In Tater

Download the main model and one compatible draft model, then configure the llama.cpp provider in Tater.

### MTP

1. Select `gemma-4-26B-A4B-it-UD-Q4_K_M.gguf` as the main model.
2. Enable **Speculative Decoding**.
3. Set **Method** to **Multi-Token Prediction (MTP)**.
4. Select `gemma-4-26B-A4B-it-MTP-Q8_0.gguf` under **Draft Model (GGUF)**.
5. Leave **Maximum Draft Tokens** at the recommended value of `3`, then use **Save & Load**.

The draft must be paired with this Gemma 4 26B target family. It is not interchangeable with DFlash, DSpark, or drafts made for another target model.

### DFlash

1. Select `gemma-4-26B-A4B-it-UD-Q4_K_M.gguf` as the main model.
2. Enable **Speculative Decoding**.
3. Set **Method** to **DFlash**.
4. Select `gemma-4-26B-A4B-it-DFlash-Q8_0.gguf` under **Draft Model (GGUF)**.
5. Set **Maximum Draft Tokens** to `3`, then use **Save & Load**.

The DFlash checkpoint was trained with a block size of 16, but its published mixed-workload benchmark found `3` to be the best general starting value. Structured tasks such as code or JSON may benefit from `4` or `5`.

### llama.cpp command-line example

Use a recent llama.cpp build with Gemma 4 MTP support:

```bash
llama-server \
  --model gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
  --model-draft gemma-4-26B-A4B-it-MTP-Q8_0.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --jinja \
  --flash-attn on
```

For DFlash, replace the draft model and speculative type:

```bash
llama-server \
  --model gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
  --model-draft gemma-4-26B-A4B-it-DFlash-Q8_0.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 3 \
  --jinja \
  --flash-attn on
```

Actual speedup depends on prompt, hardware, context length, and draft acceptance rate.

## Source And Attribution

Upstream repository:

- https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF
- https://huggingface.co/google/gemma-4-26B-A4B-it-assistant
- https://huggingface.co/z-lab/gemma-4-26B-A4B-it-DFlash
- https://huggingface.co/williamliao/gemma-4-26B-A4B-it-DFlash-GGUF

License:

- Apache 2.0

This repo is meant as a practical Tater compatibility build, not a new model or fine tune.

### Draft model provenance

The MTP sidecar was converted from `google/gemma-4-26B-A4B-it-assistant` revision `6e5aaaf4c42b98394530b8fda2e95cadd65c151c` with llama.cpp revision `8e7f22b67ef4`, using Q8_0 output. During conversion, the source tokenizer configuration's empty `extra_special_tokens` list was treated as an empty mapping to satisfy the converter; model weights were not changed.

The DFlash sidecar is redistributed unchanged from `williamliao/gemma-4-26B-A4B-it-DFlash-GGUF` revision `d1800ac59f255542ae096018fa696f03918066a6`. That conversion is derived from the Apache-2.0-licensed `z-lab/gemma-4-26B-A4B-it-DFlash` checkpoint and was tested by its publisher with llama.cpp and the same Gemma 4 26B-A4B-it target family.

| File | Size | SHA-256 |
| --- | ---: | --- |
| `gemma-4-26B-A4B-it-MTP-Q8_0.gguf` | 461,766,592 bytes | `9764ab8276181017bf565c2be9e7c61cc59b6f74f877c3d1514ca36e01d040f1` |
| `gemma-4-26B-A4B-it-DFlash-Q8_0.gguf` | 472,432,704 bytes | `48ecabebc399e5424b89913197204f80cda23e42228c67b6725f1ab1cc8f5da3` |

## DSpark Status

A Gemma 4 26B-A4B-it DSpark GGUF exists, but its current SpecForge conversion requires llama.cpp PR `#26275` and the `dflash.bonus_anchor` runtime behavior. That support is not present in the llama.cpp revision bundled with the current Tater release, so the DSpark file is intentionally not mirrored here yet. It will be suitable to add after Tater moves to a compatible llama.cpp revision and passes a paired-model test.

## Tater Usage

Use this repo as a llama.cpp GGUF model in Tater. For vision, select `mmproj-F16.gguf` as the projector.

Recommended first file:

```text
gemma-4-26B-A4B-it-UD-Q4_K_M.gguf
```