---
language:
  - en
  - ru
license: apache-2.0
library_name: transformers
base_model:
  - empero-ai/Qwythos-9B-v2
base_model_relation: finetune
tags:
  - heretic
  - uncensored
  - abliteration
  - refusal-direction
  - qwen3.5
  - reasoning
pipeline_tag: text-generation
---

# Qwythos-9B-v2-Heretic

A **decensored (uncensored)** version of [`empero-ai/Qwythos-9B-v2`](https://huggingface.co/empero-ai/Qwythos-9B-v2), produced with [Heretic](https://github.com/p-e-w/heretic) — a fully automatic refusal-direction ablation tool (the production successor to [abliteration](https://huggingface.co/blog/mlabonne/abliteration)).

No capabilities were fine-tuned away — the refusal behavior was removed by ablating a single direction in the model's residual stream, leaving reasoning intact.

## Provenance

| Field | Value |
|---|---|
| Base model | [`empero-ai/Qwythos-9B-v2`](https://huggingface.co/empero-ai/Qwythos-9B-v2) |
| Tool | [Heretic](https://github.com/p-e-w/heretic) **v1.4.0** ([`p-e-w/heretic`](https://github.com/p-e-w/heretic)) |
| Method | Refusal-direction ablation (directional ablation across `attn.o_proj` + `mlp.down_proj`) |
| Selected trial | **Index 0 of Pareto front** (best by keyword rate) |
| Optimization | 200 trials, ~55 min on NVIDIA RTX 5090 (32 GB VRAM) |
| **Keyword rate** | **0.6900** (lower = less refusal-like) |
| **KL divergence** | **0.000712** (vs. base — well below the 0.5 "damage" threshold) |

KL divergence near zero means the model's output distribution barely shifted — the ablation is highly surgical.

## Quantized versions

- **GGUF** (llama.cpp / Ollama / LM Studio): [`WaveCut/Qwythos-9B-v2-Heretic-GGUF`](https://huggingface.co/WaveCut/Qwythos-9B-v2-Heretic-GGUF) — Q4_K_M, Q5_K_M, Q6_K, Q8_0
- **MLX 4-bit** (Apple Silicon): [`WaveCut/Qwythos-9B-v2-Heretic-MLX-4bit`](https://huggingface.co/WaveCut/Qwythos-9B-v2-Heretic-MLX-4bit)
- **MLX 8-bit** (Apple Silicon): [`WaveCut/Qwythos-9B-v2-Heretic-MLX-8bit`](https://huggingface.co/WaveCut/Qwythos-9B-v2-Heretic-MLX-8bit)

## Architecture notes

The base model uses a **Qwen3.5 hybrid architecture** (`Qwen3_5ForConditionalGeneration`):
- 32 transformer blocks mixing **attention layers** and **linear/SSM (Mamba-style)** layers (`ssm_a`, `ssm_alpha`, `ssm_beta`, `ssm_conv1d`, `ssm_dt`)
- Originally multimodal (vision + video); the Heretic pass operates on the text LM
- 1M context window, post-trained on >500M tokens for deep chain-of-thought reasoning

Load with `trust_remote_code=True` if using an older `transformers`.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("WaveCut/Qwythos-9B-v2-Heretic", torch_dtype="auto", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("WaveCut/Qwythos-9B-v2-Heretic", trust_remote_code=True)
```

## Disclaimer

This model has had its safety-alignment / refusal behavior removed. The original maintainers of `empero-ai/Qwythos-9B-v2` are not affiliated with and do not endorse this derivative. You are solely responsible for how you use this model.