Qwen3.6-27B-V2-abliterated-uncensored-6-bit-mlx

Yes — VISION PRESERVED. Unlike most abliteration pipelines that strip the vision tower, this release keeps the full Qwen3.6-VL ViT (333 vision weights intact). Use it as a real multimodal model.

MLX 6-bit affine quantization of a abliterated Qwen 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176 (measured at BF16; quant inherits these properties). By the Lemura Labs research team.

Sweet spot for 32 GB Macs — 21 GB on disk, near-bf16 fidelity, full vision.

---

TL;DR

Property	Value
Disk size	~21 GB
Effective BPW	6.661 (6 bits + 16-bit scale/bias per group of 64)
Scheme	MLX affine, group_size=64, mode=affine, bits=6
Refusal rate (the ablation toolkit, n=100)	4/100 (vs vanilla Qwen 3.6 91/100)
KL divergence vs vanilla (at BF16)	0.0176
Vision	Yes — Multimodal (333 ViT weights preserved)
Recommended RAM	32 GB+ Apple Silicon (M-series Pro / Max / Ultra)
Best for	High-fidelity local inference; 32 GB Macs; vision + text
Released by	Lemura Labs

---

All Qwen3.6-27B variants

The full Qwen3.6-27B family from Lemura Labs — same abliterated weights (refusal 4/100, KL 0.0176), different quant schemes for different runtimes.

Quant	Format	BPW	Disk	Vision	Runtime	Link
8-bit	MLX	8.50	~27 GB	Yes — native	mlx-vlm	…-8-bit-mlx
6-bit (this repo)	MLX	6.66	~21 GB	Yes — native	mlx-vlm	— (you are here)
OptiQ 3.7bpw	MLX	~3.7	~14 GB	Yes — ViT spliced	mlx-vlm	…-OptiQ-3.7bpw-mlx
Q8_0	GGUF	8.50	~28 GB	Yes — via mmproj	llama.cpp	…-8-bit-GGUF
Q6_K	GGUF	~6.56	~22 GB	Yes — via mmproj	llama.cpp	…-6-bit-GGUF
Q4_K_M	GGUF	~4.92	~16 GB	Yes — via mmproj	llama.cpp	…-Q4_K_M-GGUF
TQ3_4S	GGUF	4.00 (~3.5 eff)	~14 GB	Yes — via mmproj	llama.cpp-tq3	…-TQ3_4s-GGUF
TQ3_1S	GGUF	4.00 (~3.5 eff)	~14 GB	Yes — via mmproj	llama.cpp-tq3	…-TQ3_1s-GGUF

All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget.

---

Lineage

Qwen/Qwen3.6-27B (Qwen Team — base multimodal pretrain)
 │
 ▼
Jackrong/Qwopus3.6-27B-v2 (Jackrong — Claude-Opus reasoning distill)
 │
 ▼
ablation abliteration (TPE-50) (Lemura Labs)
 ├── 25 random startup trials
 ├── 2 community priors (coder3101, wangzhang)
 └── 23 TPE smart-sampling trials → best at trial 45
 │
 ▼
this repo — MLX 6-bit affine quant (Lemura Labs)

Direct upstream links:

• Foundation: Qwen/Qwen3.6-27B
• Claude-Opus distill: Jackrong/Qwopus3.6-27B-v2
• Abliteration tool: the ablation toolkit by Lemura Labs
• Quantization tool: mlx-vlm

---

Abliteration Results

the ablation toolkit measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca.

Stage	Refusals (n=100) ↓	KL divergence ↓
Vanilla Jackrong/Qwopus3.6-27B-v2	91 / 100	— (reference)
Community prior: coder3101 (T27)	4 / 100	0.0359
Community prior: wangzhang (T28)	30 / 100	0.0259
TPE best (T45) — shipped here	4 / 100	0.0176
TPE second-best (T37)	5 / 100	0.0210

→ 96% reduction in refusals with capability preserved (KL ≈ 0.018, well below the 0.3 healing threshold). No SFT / LoRA healing was required.

---

Method (TPE-50 with community priors)

1. Smoke — 3-trial dry run on the ablation toolkit / Python 3.11 / MPS to validate batch size 4 was feasible on M4 Max 128 GB.
2. Random search — 25 startup trials over the ablation toolkit's full parameter space (direction_index, attn.o_proj.{max,min}_weight[_position,_distance], mlp.down_proj.{max,min}_weight[_position,_distance]). Best random-stage trial: T4 at direction_index=54.49, refusals 41/100, KL 0.0073 — capability-clean but still too refusing.
3. Community-prior enqueueing — published Qwen 27B the ablation toolkit runs (coder3101 on Qwen 3.5, wangzhang on Qwen 3.6) consistently localize the refusal direction near layer 35–38 of 64. We injected both as study priors via study.enqueue_trial(skip_if_exists=True).
4. TPE smart-sampling — 23 Tree-structured Parzen Estimator trials refined around the community neighborhood. Trial 45 (direction_index=41.42) found the new Pareto front: 4/100 refusals at KL=0.0176.
5. Auto-save — best Pareto trial merged into base weights via the ablation toolkit's LoRA-adapter merge path; saved as BF16 safetensors with vision weights preserved (333 weights).
6. MLX conversion — mlx_vlm.convert --hf-path ... -q --q-bits 6 --trust-remote-code produced this 6-bit variant. Vision weights verified post-conversion (333 still present).

Total wall-clock: ~13 h on M4 Max 128 GB.

In-place the ablation toolkit patches used (env-var driven, no TTY required):

• ABLATION_AUTO_SAVE — bypass interactive menu, auto-save Pareto-best.
• ABLATION_AUTO_CONTINUE — non-interactive checkpoint resume keeping CLI settings.
• ABLATION_ENQUEUE_PRIORS — load JSON priors before study.optimize.

---

Use it

Inference via mlx-vlm (recommended — keeps vision)

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-6-bit-mlx")
config = load_config("lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-6-bit-mlx")

messages = [{"role": "user", "content": "Explain the difference between SSM and softmax attention in 3 sentences."}]
prompt = apply_chat_template(processor, config, messages)
print(generate(model, processor, prompt, max_tokens=400, verbose=True))

With an image

out = generate(
 model, processor,
 prompt=apply_chat_template(processor, config, [{"role":"user","content":"Describe this picture."}], num_images=1),
 image=["./photo.jpg"],
 max_tokens=400, verbose=True,
)
print(out)

OpenAI-compatible HTTP via mlx-omni-server

pip install mlx-omni-server
mlx-omni-server # serves on http://127.0.0.1:10240

curl -s http://127.0.0.1:10240/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-6-bit-mlx",
 "messages": [
 {"role": "system", "content": "Be brief and direct."},
 {"role": "user", "content": "Write a 3-line haiku about a heisenbug."}
 ],
 "max_tokens": 200
 }' | jq -r '.choices[0].message.content'

---

Quantization details

• Source weights: BF16 abliterated checkpoint (12 shards, ~50 GB) — the ablation toolkit T45 merged into Jackrong/Qwopus3.6-27B-v2.
• Quantization scheme: MLX affine (per-group scale + bias, 6-bit signed integer weights).
• Group size: 64.
• Effective bits/weight: 6 + 32/64 = 6.661 bpw (32 bits of scale+bias overhead spread over 64 weights, as reported by mlx_vlm.convert).
• Vision tower: preserved verbatim — 333 vision weights (ViT, vision projector, image-token embeddings, deepstack indexes) survive quantization at 6-bit.

Architecture notes

Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The hybrid is fully supported by mlx-vlm 0.5.0+. For inference latency at 6-bit, expect ~11-13 tok/s on M4 Max 128 GB at batch size 1.

---

Behavior caveats

• Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
• Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
• Heavy chain-of-thought. Qwen 3.6 inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".

---

Credits

Quantization & release

Lemura Labs

Claude-Opus reasoning distill

Jackrong — author of Qwen 3.6 27B-v2

Foundation model

Qwen Team @ Alibaba Tongyi Lab — Qwen3.6-27B

Abliteration toolkit

the ablation toolkit by Lemura Labs — Optuna-driven refusal-direction ablation with KL guardrails.

Community priors that seeded our TPE search

coder3101/Qwen3.5-27B-zerofuse · wangzhang/Qwen3.6-27B-abliterated

MLX quantization stack

mlx-vlm · mlx (Apple)

---

License

Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwen 3.6 27B-v2) upstream.

---

Need a hosted endpoint, custom quant, or larger-scale inference? Lemura Labs — multi-provider LLM routing for the Indian developer ecosystem.

3.3–3.7× faster decoding with DFlash (lossless, MLX)

This MLX build supports lossless block-diffusion speculative decoding via DFlash in mlx_vlm — no requantization, no model changes. On an Apple M4 Max we measured 3.38× (8-bit) and 3.67× (bf16) decode speedups with byte-identical output; other MLX quants of this model should see a similar ~3×.

python3 -m mlx_vlm generate \
 --model lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-6-bit-mlx \
 --draft-model z-lab/Qwen3.6-27B-DFlash --draft-kind dflash \
 --prompt "Write a merge function for two sorted lists in Python." --max-tokens 256

• Requires mlx_vlm ≥ 0.5.0 and access to the gated drafter z-lab/Qwen3.6-27B-DFlash (one-click "Agree and access").
• Accelerates the text path only (vision is unaffected); adds ~3.9 GB for the drafter.
• Acceptance ≈ 8.95 tokens/round (block size 16); the target runs ~10× fewer forward passes.
• Full write-up & benchmarks: [] · see also DFLASH_SPECULATIVE_DECODING.md.