---
license: gemma
base_model: Cactus-Compute/gemma-4-e2b-it-hybrid
tags:
  - gguf
  - gemma4
  - hybrid
  - handoff
---

# gemma-4-e2b-it-hybrid — GGUF

GGUF builds of [Cactus-Compute/gemma-4-e2b-it-hybrid](https://huggingface.co/Cactus-Compute/gemma-4-e2b-it-hybrid):
Gemma 4 E2B instruction-tuned with a built-in **handoff probe** — every
generation returns a `confidence` score so your app can answer locally when
the model is likely right and escalate to a bigger model when it isn't.

## Requirements

The `gemma-4-e2b-it-hybrid` architecture is not yet in upstream llama.cpp.
Run these files with a build that includes the Cactus patch series (see the
[cactus-hybrid-demo](https://github.com/cactus-compute/cactus-hybrid-demo)
repo, `integrations/llama.cpp/` — `apply.sh` builds it in ~3 minutes, prebuilt
binaries planned). On unpatched llama.cpp the files fail to load with
"unknown model architecture" by design.

```bash
llama-server -hf Cactus-Compute/gemma-4-e2b-it-hybrid-GGUF --jinja
```

Chat-completions responses (and the final SSE chunk when streaming) carry a
top-level `"confidence"` field. Default handoff threshold: **0.85**; see the
calibration note below for quantized builds.

## Files

| file | quant | size | notes |
|---|---|---|---|
| `gemma-4-e2b-it-hybrid-f16.gguf` | F16 | 9.31 GB | closest to the bf16 reference |
| `gemma-4-e2b-it-hybrid-Q4_K_M.gguf` | Q4_K_M | 3.43 GB | recommended for consumer hardware |

The probe head (11 `probe.*` tensors) is stored in F32 in **all** quants —
only the trunk is quantized.

## Calibration note

Quantized trunks shift the layer-28 activations the probe reads, moving
confidences downward relative to the bf16 reference (measured mean drift:
F16 −0.07, Q4_K_M −0.10; easy-vs-hard ordering fully preserved). If you use
aggressive thresholds, calibrate per quant; the 0.85 default remains
conservative (it hands off more, never less).

## License

Gemma is provided under and subject to the Gemma Terms of Use. This
derivative includes the Cactus handoff probe head.