---
library_name: gguf
license: apache-2.0
base_model: google/gemma-4-E4B-it
tags:
  - gemma-4
  - quantization
  - imatrix
  - awq
  - iq2_m
  - iq3_m
  - iq4_xs
  - gguf
  - llama.cpp
datasets:
  - pearsonkyle/llmtk-sft-corpus-v2
  - pearsonkyle/broad-domain-supplement
model-index:
  - name: gemma-4-E4B-it-IQ2_M-imatrix
    results: []
  - name: gemma-4-E4B-it-IQ3_M-imatrix
    results: []
  - name: gemma-4-E4B-it-IQ4_XS-imatrix
    results: []
  - name: gemma-4-E4B-it-IQ2_M-awq
    results: []
  - name: gemma-4-E4B-it-IQ3_M-awq
    results: []
  - name: gemma-4-E4B-it-IQ4_XS-awq
    results: []
  - name: gemma-4-E4B-it-qat-IQ2_M-imatrix
    results: []
  - name: gemma-4-E4B-it-qat-IQ3_M-imatrix
    results: []
  - name: gemma-4-E4B-it-qat-IQ4_XS-imatrix
    results: []
  - name: gemma-4-E4B-it-qat-IQ2_M-awq
    results: []
  - name: gemma-4-E4B-it-qat-IQ3_M-awq
    results: []
  - name: gemma-4-E4B-it-qat-IQ4_XS-awq
    results: []
---

# Gemma-4 E4B — Imatrix vs AWQ Quantization Study

A 12-quant matrix comparing **imatrix** and **AWQ** calibration methods across three quantization types (**IQ2_M**, **IQ3_M**, **IQ4_XS**) on two same-caliber models:

- `google/gemma-4-E4B-it` — base E4B (instruction-tuned)
- `google/gemma-4-E4B-it-qat-q4_0-unquantized` — QAT (quantization-aware trained)

All quants were calibrated on **15M tokens** of `pearsonkyle/llmtk-sft-corpus-v2` (32k context, seed-42 shuffle, 9,597 train sessions) and benchmarked on `pearsonkyle/broad-domain-supplement` (general-domain + instruct/tools splits).

## Results

| Model | Quant | Method | Size (GiB) | BPW | KLD (general) | KLD (instruct) | Top-P (general) | Top-P (instruct) | Decode (tok/s) |
|---|---|---|---|---|---|---|---|---|---|
| gemma-4-E4B-it | IQ2_M | imatrix | 3.550 | 4.056 | 1.461 | 0.978 | 50.0 | 61.8 | 50.3 |
| gemma-4-E4B-it | IQ2_M | AWQ | 3.527 | 4.030 | 12.266 | 13.830 | 0.1 | 0.1 | 69.9 |
| gemma-4-E4B-it | IQ3_M | imatrix | 4.391 | 5.017 | 0.342 | 0.222 | 74.9 | 81.1 | 68.8 |
| gemma-4-E4B-it | IQ3_M | AWQ | 4.365 | 4.988 | 16.393 | 17.650 | 0.3 | 0.3 | 63.5 |
| gemma-4-E4B-it | IQ4_XS | imatrix | 4.723 | 5.396 | 0.149 | 0.094 | 82.8 | 87.6 | 59.5 |
| gemma-4-E4B-it | IQ4_XS | AWQ | 4.691 | 5.360 | 16.443 | 17.562 | 0.03 | 0.02 | 40.8 |
| gemma-4-E4B-it-qat | IQ2_M | imatrix | 3.527 | 4.060 | 4.888 | 4.940 | 21.9 | 23.8 | 76.2 |
| gemma-4-E4B-it-qat | IQ2_M | AWQ | 3.527 | 4.060 | 18.363 | 20.090 | 0.1 | 0.0 | 65.1 |
| gemma-4-E4B-it-qat | IQ3_M | imatrix | 4.365 | 5.025 | 0.255 | 0.205 | 76.6 | 80.1 | 56.6 |
| gemma-4-E4B-it-qat | IQ3_M | AWQ | 4.365 | 5.025 | 15.942 | 17.118 | 0.2 | 0.1 | 59.5 |
| gemma-4-E4B-it-qat | IQ4_XS | imatrix | 4.691 | 5.400 | 0.057 | 0.049 | 88.0 | 89.8 | 86.1 |
| gemma-4-E4B-it-qat | IQ4_XS | AWQ | 4.691 | 5.400 | 14.067 | 15.494 | 0.9 | 0.7 | 68.4 |

### Key findings

1. **Imatrix dominates AWQ** for gemma-4 E4B across every quant type. KLD is 5–120× lower with imatrix; top-p agreement is 100× higher.
2. **IQ4_XS imatrix is the sweet spot**: near-FP16 KLD (0.057–0.149) at 4.7 GiB, 86–88% top-p agreement.
3. **IQ3_M imatrix is the best size/quality tradeoff**: KLD 0.22–0.34 at 4.4 GiB, 75–81% top-p.
4. **IQ2_M imatrix is viable** at 3.5 GiB (KLD 1.0–1.5, 50–62% top-p) but AWQ at 2-bit is unusable.
5. **QAT model benefits more from imatrix** at higher bit rates (IQ3_M, IQ4_XS) — the QAT IQ4_XS imatrix achieves the best KLD of the entire matrix (0.057).
6. **AWQ is unusable for this model** — logit drift is too large (F16 reference KLD > 12 on all AWQ quants), likely due to the model's GQA architecture (4G GQA, 24 KV-grouped layers) which makes the RMSNorm fold less effective.

## Files

| File | Type | Method | Size |
|---|---|---|---|
| `gemma-4-E4B-it-IQ2_M-imatrix.gguf` | IQ2_M | imatrix | 3.55 GiB |
| `gemma-4-E4B-it-IQ2_M-awq.gguf` | IQ2_M | AWQ | 3.53 GiB |
| `gemma-4-E4B-it-IQ3_M-imatrix.gguf` | IQ3_M | imatrix | 4.39 GiB |
| `gemma-4-E4B-it-IQ3_M-awq.gguf` | IQ3_M | AWQ | 4.37 GiB |
| `gemma-4-E4B-it-IQ4_XS-imatrix.gguf` | IQ4_XS | imatrix | 4.72 GiB |
| `gemma-4-E4B-it-IQ4_XS-awq.gguf` | IQ4_XS | AWQ | 4.69 GiB |
| `gemma-4-E4B-it-qat-IQ2_M-imatrix.gguf` | IQ2_M | imatrix | 3.53 GiB |
| `gemma-4-E4B-it-qat-IQ2_M-awq.gguf` | IQ2_M | AWQ | 3.53 GiB |
| `gemma-4-E4B-it-qat-IQ3_M-imatrix.gguf` | IQ3_M | imatrix | 4.37 GiB |
| `gemma-4-E4B-it-qat-IQ3_M-awq.gguf` | IQ3_M | AWQ | 4.37 GiB |
| `gemma-4-E4B-it-qat-IQ4_XS-imatrix.gguf` | IQ4_XS | imatrix | 4.69 GiB |
| `gemma-4-E4B-it-qat-IQ4_XS-awq.gguf` | IQ4_XS | AWQ | 4.69 GiB |

## Methodology

- **Calibration**: 15M tokens from `pearsonkyle/llmtk-sft-corpus-v2` 32k split (seed-42 shuffle, 9,597 sessions, 32k context). Imatrix collected via `llama-imatrix -c 32768 --parse-special`; hybrid_custom variant re-weights per-tensor. AWQ α-search via proxy quantizer, imatrix collected on folded F16.
- **GQA fix**: gemma-4 has 4G GQA (8 Q / 2 KV heads, 42 layers, 24 KV-grouped layers). `llama-imatrix` only collects K/V stats for layers 0–23; a `_backfill_missing_kv_layers` patch copies per-channel mean of collected K/V vectors onto the 18 missing layers.
- **Evaluation**: KLD + perplexity + top-p agreement via `llama-perplexity` on `pearsonkyle/broad-domain-supplement` (general-domain: 30,710 tokens; instruct/tools: 30,853 tokens). Speed via `llama-bench`.
- **Machine**: Apple Silicon (MPS), llama.cpp vendored build.