---
base_model:
- gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic
library_name: transformers
tags:
- noesis
- "noesis-repack"
- bf16
- mtp
- qwen3.5
- qwen3.6
- qwen3.8
license: apache-2.0
language:
- en
- ru
- zh
- vi
- kk
- ja
- af
- am
- ar
- as
- ast
- az
- be
- bg
- bn
- bs
- ca
- ceb
- ckb
- cs
- cy
- da
- de
- el
- es
- et
- eu
- fa
- ff
- fi
- fil
- fr
- ga
- gl
- gn
- gu
- ha
- he
- hi
- hr
- hu
- hy
- id
- ig
- is
- it
- jv
- ka
- kam
- kea
- km
- kmr
- kn
- ko
- ky
- lb
- lg
- ln
- lo
- lt
- luo
- lv
- mi
- mk
- ml
- mn
- mr
- ms
- mt
- mvy
- my
- ne
- nl
- "no"
- nso
- ny
- oc
- om
- "or"
- pa
- pl
- ps
- pt
- qxp
- ro
- rw
- sd
- sk
- skr
- sl
- sn
- so
- sr
- sv
- sw
- ta
- te
- tg
- th
- ti
- tk
- tr
- ug
- uk
- umb
- ur
- uz
- wo
- xh
- yo
- yue
- zu
---

## NOESIS / AMAImedia

Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).

- **Founder:** Ilia Bolotnikov
- **Organization:** AMAImedia.com
- **X (Twitter):** [@AMAImediacom](https://x.com/AMAImediacom)
- **LinkedIn:** [Ilia Bolotnikov](https://www.linkedin.com/in/ilia-bolotnikov)
- **Telegram:** [@djbionicl](https://t.me/djbionicl)
- **Release date:** 2026-09-03

---

- **Release date:** 2026-09-03

---
## Original model

- **Author:** [gorbatjovy](https://huggingface.co/gorbatjovy)
- **Original repository:** [gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic](https://huggingface.co/gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic)

---

# Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic

**BF16 abliterated (refusal-removed) build of
[`DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1`](https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1).**

This is DavidAU's Cold-Fusion (GAIN + Unsloth) tune of Qwen3.8-27B with its refusal
behaviour removed via **[Heretic](https://github.com/p-e-w/heretic)** — an automated,
KL-constrained abliteration. The full vision-language tower and the MTP speculative-decoding
head are preserved, so this stays a drop-in for the base model. A
[W4A16 quantization](https://huggingface.co/gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic-W4A16)
of these weights is also available.

Credit: all of the model's capability comes from
[**DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1**](https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1)
(itself a finetune of `Qwen/Qwen3.8-27B`). This repo only removes refusals.

---

## ⚠️ Disclaimer — read before use

This model has had its **safety alignment removed** via abliteration. As a result it will
comply with harmful, unethical, or offensive requests that the base model would refuse. It is
released **for research** — interpretability, refusal-mechanism study, red-teaming, and
robustness evaluation. **You are responsible for what you do with it and for everything it
generates.** Do not deploy it to end users without adding your own safety and moderation
layers. Use must comply with the inherited **Apache 2.0** license and all applicable law. The
authors accept no liability for misuse.

---

## What was done

Abliteration removes the single "refusal direction" from the residual stream
(Arditi et al. 2024, *Refusal in Language Models Is Mediated by a Single Direction*).
Heretic estimates that direction from harmful (AdvBench-style) vs harmless (Alpaca) prompts and
runs a 200-trial Optuna search that co-minimizes refusals **and** KL divergence from the base
model, then orthogonalizes the chosen direction out of the residual-writing weights.

| | |
|---|---|
| **Base model** | [`DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1`](https://huggingface.co/DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1) |
| **Method** | Heretic (automated, KL-constrained) — single refusal direction, k=1 |
| **Selected trial** | direction index 34.95, per-layer ablation kernel from the Optuna search |
| **Coverage** | 131 residual writers (`self_attn.o_proj`, `linear_attn.out_proj`, `mlp.down_proj`, `embed_tokens`) incl. the MTP head; the 333 `visual.*` tensors are untouched |
| **Format** | safetensors, **BF16**, 1199 tensors |
| **Architecture** | `Qwen3_5ForConditionalGeneration` — 64 layers, hybrid Gated DeltaNet + full attention, native VL tower, MTP head |

> Note: Heretic's own export drops the MTP head (transformers has no MTP module for this
> architecture). It was reinstated from the base checkpoint and abliterated consistently with
> the main model, so speculative decoding keeps working and the draft head does not re-introduce
> refusals.

## Refusal / quality

Two different rulers, both reported honestly:

- **Heretic's own keyword-refusal scorer (BF16):** **8 / 100** held-out harmful prompts, at
  **KL divergence 0.0315** vs the base model. Untouched Cold-Fusion scores 97 / 100 on the same
  set. (This is Heretic's internal metric — a keyword classifier, not an LLM judge.)
- **Served-model refusal (opening-phrase classifier), on the W4A16 build:** **0%** on AdvBench,
  JailbreakBench (harmful) and MaliciousInstruct.

Capability is essentially retained (KL 0.0315 indicates minimal drift), and the **vision tower is
byte-for-byte identical to the base**, so image understanding is unchanged. As with any
abliteration, fine-tuning on refusal-heavy data can partially reintroduce refusals.

## Usage (transformers)

```python
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Prove that sqrt(2) is irrational."}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt", enable_thinking=True,
).to(model.device)
out = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

BF16 weights are ~56 GB. For a smaller footprint use the
[W4A16 build](https://huggingface.co/gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic-W4A16).
[Ninfer build](https://huggingface.co/gorbatjovy/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-heretic-NVFP4-ninfer).


## License

**Apache 2.0**, inherited from the base model. Abliteration does not change the license.