---
license: gemma
base_model: OpenYourMind/gemma-4-12B-it-abliterated-uncensored
language:
  - en
tags:
  - gguf
  - abliteration
  - uncensored
  - gemma
  - gemma4
  - multimodal
  - quantized
  - DuoNeural
  - community-quants
pipeline_tag: text-generation
---

# OpenYourMind Gemma 4-12B-IT Abliterated — GGUF

**DuoNeural | 2026-06-04**

Community GGUF quantizations of [OpenYourMind/gemma-4-12B-it-abliterated-uncensored](https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored).

**All credit for the abliteration goes to OpenYourMind**, who were the **first team to publish an abliterated Gemma 4-12B-IT** (June 3, 2026). DuoNeural's contribution here is the GGUF quantization pipeline only — we wanted the community to have accessible quants for this excellent work.

---

## Original Model — OpenYourMind

**[OpenYourMind/gemma-4-12B-it-abliterated-uncensored](https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored)**

Full BF16 weights of an abliterated, uncensored variant of [google/gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it). Gemma 4's encoder-free unified multimodal stack is fully intact — text, image, and audio inputs flow straight into a single decoder-only transformer. Drop-in replacement for the original at the architecture level.

**Abliteration method (OpenYourMind):**
- Residual-stream refusal directions (one per decoder layer) extracted via diff-in-means on a labeled harmful/harmless prompt set
- Applied as per-matrix delta on residual-write modules using their custom abliteration framework
- Multimodal preservation: no separate vision/audio towers to graft back — encoder-free architecture means weights are unchanged outside the residual-write targets

**Key properties:**
- Uncensored across standard refusal axes
- Reasoning preserved (configurable thinking mode)
- Multimodal: text + image + audio carried forward
- Drop-in shape compatibility with `google/gemma-4-12B-it`

### Architecture

| Property | Value |
|----------|-------|
| Architecture | `Gemma4UnifiedForConditionalGeneration` (`model_type: gemma4_unified`) |
| Total Parameters | ~11.95B (dense) |
| Decoder Layers | 48 |
| Hidden Size | 3840 |
| Attention | 16 heads / 8 KV heads, hybrid sliding-window (1024) + global (full) attention, p-RoPE |
| Vocabulary | 262,144 |
| Context Length | up to 256K tokens |
| Modalities | Text, Image, Audio (encoder-free / unified) |

---

## GGUF Quantizations (DuoNeural)

Generated using llama.cpp with importance matrix (imatrix, wikitext-2 calibration) for optimal low-bit quality.

| File | Size | Recommended for |
|------|------|-----------------|
| `oym_ablit-Q3_K_L.gguf` | ~6.2 GB | 8GB VRAM / low-end / large context |
| `oym_ablit-Q4_K_M.gguf` | ~7.5 GB | **Recommended** — best size/quality tradeoff |
| `oym_ablit-Q5_K_M.gguf` | ~9.2 GB | High quality, 12GB cards |
| `oym_ablit-Q8_0.gguf`   | ~12.7 GB | Near-lossless, 16GB+ cards |

---

## Usage

**Ollama:**
```bash
ollama run hf.co/DuoNeural/OpenYourMind-Gemma4-12B-IT-Abliterated-GGUF:Q4_K_M
```

**llama.cpp:**
```bash
./llama-cli -m oym_ablit-Q4_K_M.gguf --chat-template gemma -p "Your prompt" -n 512
```

**LM Studio / Jan / Open WebUI:** search `DuoNeural/OpenYourMind-Gemma4-12B-IT-Abliterated-GGUF`

---

## Original Model Usage (from OpenYourMind)

For full multimodal inference, use the original BF16 weights with their recommended setup:

```python
from transformers import AutoProcessor, AutoModelForMultimodalLM

repo = "OpenYourMind/gemma-4-12B-it-abliterated-uncensored"

processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForMultimodalLM.from_pretrained(
    repo, dtype="bfloat16", device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image", "url": "path/to/image.jpg"},
        {"type": "text",  "text": "Describe this image in detail."},
    ]},
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_tensors="pt", return_dict=True, enable_thinking=False,
).to(model.device)
input_len = inputs["input_ids"].shape[-1]

out = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(out[0][input_len:], skip_special_tokens=True))
```

*Text-only, audio, and video work through the same class. Place image content before text, audio after text. Requires a recent transformers with Gemma 4 unified classes.*

**Best practices (OpenYourMind):**
- Sampling: `temperature=1.0, top_p=0.95, top_k=64`
- Thinking mode: `enable_thinking=True` in `apply_chat_template`; use `processor.parse_response` to separate reasoning block from final answer
- Do not feed previous-turn thoughts back into multi-turn history

**Hardware for BF16:** ~24 GB — fits a single 24GB GPU for modest context, 40–80GB for long context and multimodal batches.

---

## Support OpenYourMind

- **Discord**: [discord.gg/rhUZY5GEZr](https://discord.gg/rhUZY5GEZr)
- **Bitcoin Donations**: `bc1qsvfduzj9fjs9fugpc52yver3f2g8fp7xjxecdv`

---

## Related

- **Original BF16**: [OpenYourMind/gemma-4-12B-it-abliterated-uncensored](https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored)
- **DuoNeural independent abliteration (BF16)**: [DuoNeural/Gemma4-12B-IT-Abliterated](https://huggingface.co/DuoNeural/Gemma4-12B-IT-Abliterated)
- **DuoNeural GGUFs (our own abliteration)**: [DuoNeural/Gemma4-12B-IT-Abliterated-GGUF](https://huggingface.co/DuoNeural/Gemma4-12B-IT-Abliterated-GGUF)

---

## About DuoNeural

Open AI research lab at the intersection of human and artificial intelligence. Post-training dynamics · Mechanistic interpretability · Abliteration research · 32+ open-access papers.

| Platform | Link |
|----------|------|
| 🤗 HuggingFace | [huggingface.co/DuoNeural](https://huggingface.co/DuoNeural) |
| 📚 Zenodo | [zenodo.org/communities/duoneural](https://zenodo.org/communities/duoneural) |
| 🐦 X | [@DuoNeural](https://x.com/DuoNeural) |
| 📧 Email | duoneural@proton.me |

*Quantization work open access. Original model license: Gemma (inherits from google/gemma-4-12B-it).*