---
license: apache-2.0
base_model: ProCreations/grug-35b-v2
tags: [grug, gguf, qat, llama.cpp, reasoning, moe]
language: [en]
---

# grug-35b-qat-q4-gguf

35b MoE brother in QAT four-bit rock. brain feel rounding rock during
training so Q4 squish hurt less.

recipe: expert-freeze QAT on [grug-35b-v2](https://huggingface.co/ProCreations/grug-35b-v2) -
ALL text linear (expert include) fake-quant int4 g32 in forward, gradient only
flow to attention/DeltaNet/shared path (1.4B trainable; expert too heavy for
one cave GPU). ~1.7M grug token, Adafactor lr 2e-6. release = 25% QAT + 75%
original anchor (grug family standard). fresh bf16 export, ONE Q4_K_M squish.

## rocks

| file | what |
|---|---|
| grug-35b-qat-Q4_K_M.gguf | the QAT rock (~21 GB) |
| mmproj-grug-35b-v2-f16.gguf | eye rock (vision) |

## if rock act broken

single-token spam = context-shift corruption, not rock. recent llama.cpp +
`-c 16384`+ for agent frontends. see main gguf card for full troubleshoot.

## how run

```bash
llama-server -m grug-35b-qat-Q4_K_M.gguf --mmproj mmproj-grug-35b-v2-f16.gguf \
  -c 16384 --temp 0.6 --top-p 0.95 --top-k 20
```

ordinary rocks: [grug-35b-v2-gguf](https://huggingface.co/ProCreations/grug-35b-v2-gguf).
27b QAT brother: [grug-27b-qat-q4-gguf](https://huggingface.co/ProCreations/grug-27b-qat-q4-gguf).
grug made by ProCreations.