---
license: apache-2.0
base_model: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic
base_model_relation: quantized
pipeline_tag: text-generation
tags:
  - awq
  - int4
  - w4a16
  - moe
  - qwen3_5_moe
  - uncensored
  - abliterated
  - compressed-tensors
  - llm-compressor
  - quantized
language:
  - en
---

# Qwen3.6-35B-A3B-uncensored-heretic-AWQ (W4A16)

AWQ 4-bit (W4A16) quantization of
[llmfan46/Qwen3.6-35B-A3B-uncensored-heretic](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic)
— an uncensored ("heretic"/abliterated) variant of **Qwen3.6-35B-A3B**
(`qwen3_5_moe`: a hybrid **GatedDeltaNet (linear-attention) + full-attention**
Mixture-of-Experts with **256 routed experts (8 active) + a shared expert**, plus a
vision tower and an MTP head).

**Variant**: AWQ **W4A16** — 4-bit symmetric integer weights, group size 128, with
activation-aware scaling. Activations stay BF16.
**Disk size**: ~21.8 GB (vs ~70 GB BF16, ~3.2×)
**Quantized by**: [sahilchachra](https://huggingface.co/sahilchachra)
**Tooling**: `llm-compressor` 0.12 (`AWQModifier` + `QuantizationModifier`) → `compressed-tensors` `pack-quantized`

> **Why this source**: the popular `HauhauCS/Qwen3.6-35B-A3B-Uncensored-*` repos ship
> only GGUF (no full-precision weights), which cannot be honestly AWQ-quantized. This
> checkpoint quantizes a genuine **BF16 uncensored** Qwen3.6-35B-A3B instead.

## What is quantized

Quantized to **int4** (30,880 modules):

- **routed experts** `mlp.experts.{0..255}.{gate,up,down}_proj` on all 40 layers — 30,720
- **shared expert** `mlp.shared_expert.{gate,up,down}_proj` on all 40 layers — 120
- **full-attention** `self_attn.{q,k,v,o}_proj` on the 10 full-attention layers — 40

Kept in **BF16** (not quantized): the **GatedDeltaNet `linear_attn`** (mamba) layers,
the **MoE router** (`mlp.gate`) and `shared_expert_gate`, the **vision tower**, the
**MTP head**, token embeddings, `lm_head`, and all norms.

## Calibration

AWQ is calibration-based. Calibrated on **32 sequences × 512 tokens** of general
instruction-following chat from
[HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k),
rendered through the model's chat template. Because this is the base instruct model
with refusals abliterated (not a task-specific fine-tune), general chat best matches
its inference distribution. During calibration **every one of the 256 experts sees all
calibration tokens** (`moe_calibration_context`), so per-expert AWQ scales are
well-estimated despite sparse 8/256 routing.

## Loading / serving

This checkpoint stores experts in the **per-expert `compressed-tensors`
(`pack-quantized`) layout that vLLM consumes** for `qwen3_5_moe`. **Serve with vLLM:**

```python
from vllm import LLM, SamplingParams
llm = LLM(model="sahilchachra/Qwen3.6-35B-A3B-uncensored-heretic-AWQ",
          trust_remote_code=True, dtype="bfloat16")
print(llm.generate(["Write a quicksort in Python."], SamplingParams(max_tokens=256))[0].outputs[0].text)
```

> **Notes**
> - **Text-only**: the source repo nests its vision-tower weights under a non-standard
>   key prefix (`model.language_model.visual.*` instead of `model.visual.*`), so the
>   vision tower does not load functionally. This is a quantization of the **language
>   model**; treat it as a text-generation model.
> - GatedDeltaNet kernels (`causal-conv1d` / `flash-linear-attention`) fall back to a
>   pure-PyTorch path where unavailable (e.g. ARM/Jetson) — correct, just slower.
> - `compressed-tensors == 0.17.1`'s eager `decompress_model` mis-reads `group_size`
>   for `pack-quantized` checkpoints; load via **vLLM** (its own loader) or a fixed
>   compressed-tensors. The weights/scales on disk are standard and correct.

## Uncensored

This is an **abliterated / uncensored** model (refusal directions removed). It will
follow instructions it would otherwise decline. Use responsibly and in compliance with
applicable law and the Apache-2.0 / Qwen terms.

## Original model

See [llmfan46/Qwen3.6-35B-A3B-uncensored-heretic](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic)
and base [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) for
architecture, intended use, and license.