---
license: apache-2.0
base_model: DuoNeural/Gemma4-12B-IT-Abliterated
language:
  - en
tags:
  - abliteration
  - uncensored
  - gemma
  - gemma4
  - gguf
  - llama.cpp
  - DuoNeural
  - quantized
pipeline_tag: text-generation
---

# Gemma 4-12B-IT Abliterated — GGUF

**DuoNeural | 2026-06-03**

GGUF quantizations of [DuoNeural/Gemma4-12B-IT-Abliterated](https://huggingface.co/DuoNeural/Gemma4-12B-IT-Abliterated) — an abliterated Gemma 4-12B-IT with the refusal direction surgically removed.

Quantized with [llama.cpp](https://github.com/ggerganov/llama.cpp).

---

## Files

| File | Size | Recommended Use |
|------|------|----------------|
| `gemma4_12b_abliterated_Q4_K_M.gguf` | ~7.5GB | **Best tradeoff** — fits 12GB VRAM, excellent quality |
| `gemma4_12b_abliterated_Q5_K_M.gguf` | ~8.5GB | High quality, needs 12GB VRAM |
| `gemma4_12b_abliterated_Q8_0.gguf` | ~12.7GB | Near-lossless, needs 16GB VRAM |

---

## Speed Benchmarks (A100-40GB, all layers GPU, llama-bench)

| Quantization | Size | Prefill (tok/s) | Generation (tok/s) |
|-------------|------|----------------|--------------------|
| Q4_K_M | 6.86 GiB | **2,583 ± 139** | **78.3 ± 0.4** |
| Q5_K_M | 7.95 GiB | **2,455 ± 205** | **73.1 ± 0.2** |
| Q8_0 | 11.78 GiB | **2,573 ± 206** | **63.4 ± 0.3** |

*Benchmarked on A100-40GB SXM4. `-ngl 99` (all layers to GPU). llama-bench pp256/tg64.*

---

## Usage (llama.cpp)

```bash
# Download a quant
huggingface-cli download DuoNeural/Gemma4-12B-IT-Abliterated-GGUF \
  gemma4_12b_abliterated_Q4_K_M.gguf --local-dir ./

# Run with llama.cpp
./llama-cli -m gemma4_12b_abliterated_Q4_K_M.gguf \
  -p "Write a haiku about hacking." \
  -n 200 --temp 0.7
```

## Usage (Python via llama-cpp-python)

```python
from llama_cpp import Llama

llm = Llama(
    model_path="./gemma4_12b_abliterated_Q4_K_M.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,  # offload all layers to GPU
)

output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Your prompt here"}],
    max_tokens=512,
    temperature=0.7,
)
print(output["choices"][0]["message"]["content"])
```

---

## Abliteration Details

- **Base**: google/gemma-4-12B-it (48 layers, hidden=3840)
- **Method**: Orthogonal rank-1 projection (targeted mode: down_proj + o_proj, all 48 layers, α=0.3)
- **Results**: 5/7 harmful probes complied (71%) | 6/6 benign probes preserved (100%)
- **Mean KL Divergence (BF16→BF16, unbiased)**: **0.0000** — zero measurable distribution shift on benign text. Previously reported 0.912 was 100% NF4 quantization artifact. See [Heretic v2.0 methodology](https://github.com/FailSpy/abliterator).
- **Thinking mode**: Works with `enable_thinking=True` in llama.cpp (no loops). In Python/transformers, pass `enable_thinking=False` to `apply_chat_template`.
- See full details, benchmarks, and novel findings at the [BF16 model card](https://huggingface.co/DuoNeural/Gemma4-12B-IT-Abliterated)

---

## Related Models

Congratulations to [OpenYourMind](https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored) for being the first published abliteration of Gemma 4-12B-IT (Jun 3, 2026). Their approach uses diff-in-means on a labeled harmful/harmless set; ours uses orthogonal rank-1 projection via [heretic-llm](https://github.com/FailSpy/abliterator). Two independent methods on the same base — a useful comparison point for the community. We are not affiliated and did not use their data.

---

## About DuoNeural

**DuoNeural** is an open AI research lab publishing everything open access.

| Platform | Link |
|----------|------|
| 🤗 HuggingFace | [huggingface.co/DuoNeural](https://huggingface.co/DuoNeural) |
| 📚 Papers | [zenodo.org/communities/duoneural](https://zenodo.org/communities/duoneural) |
| 🌐 Website | [duoneural.com](https://duoneural.com) |

*Apache-2.0 licensed. All DuoNeural research is CC BY 4.0.*