---
license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
- mlx
- qwen3_5
- image-text-to-text
- mxfp8
- apple-silicon
- uncensored
---

# Huihui-Qwen3.8-27B-abliterated — MLX MXFP8

MLX **MXFP8** quantization of [`huihui-ai/Huihui-Qwen3.8-27B-abliterated`](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated),
uncensored via **abliteration** (refusal-direction removal on text layers 18–51; the README notes MTP and the vision tower were left unmodified) of Qwen3.8-27B. Qwen3.8-27B is a **`qwen3_5`** vision-language model with
a **hybrid GatedDeltaNet linear-attention + full-attention** text backbone (64
layers, full attention every 4th) and a Qwen3-VL vision tower. Runs on Apple
Silicon via [mlx-vlm](https://github.com/Blaizzy/mlx-vlm). Stays
**image-text-to-text** — the vision tower is kept in bf16; only the text backbone
is quantized.

| | |
| :---- | :---- |
| **Precision** | MXFP8 (E4M3 + E8M0 shared scale, group size 32) |
| **Bits per weight** | 8.381 bpw |
| **On-disk size** | 27 GB (27 shards) |
| **Quantized** | text backbone (incl. the ~1.27B `lm_head`) |
| **Kept in bf16** | Qwen3-VL vision tower |

## Quantizations

| Variant | Bits | Size | |
| :---- | :---- | :---- | :---- |
| [`Huihui-Qwen3.8-27B-abliterated-MXFP4`](https://huggingface.co/sahilchachra/Huihui-Qwen3.8-27B-abliterated-MXFP4) | 4.449 bpw | 14 GB | smaller / for 16 GB+ |
| [`Huihui-Qwen3.8-27B-abliterated-MXFP8`](https://huggingface.co/sahilchachra/Huihui-Qwen3.8-27B-abliterated-MXFP8) | 8.381 bpw | 27 GB | **← this repo** |

## Verification

This higher-fidelity build was verified **structurally** (correct tensor shapes,
`format: mlx` metadata, consistent shard index, vision tower intact in bf16). Full
token-by-token generation was not benchmarked on the 24 GB test machine because
27 GB exceeds its RAM; on a 32 GB+ Mac it runs at normal speed. Since MXFP8 uses
more bits than the **MXFP4 build — which passed text + vision smoke tests
end-to-end** — it is at least as faithful to the base model.

See the [MXFP4 build](https://huggingface.co/sahilchachra/Huihui-Qwen3.8-27B-abliterated-MXFP4) for the
full generation/vision smoke-test results.

## Usage (mlx-vlm)

```bash
pip install -U mlx-vlm   # needs the qwen3_5 architecture (>= 0.6.12)
```

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("sahilchachra/Huihui-Qwen3.8-27B-abliterated-MXFP8")
config = model.config

prompt = apply_chat_template(processor, config, "What is the capital of France?")
print(generate(model, processor, prompt, max_tokens=256, verbose=True))
```

This is a reasoning model; it produces a `<think>` channel before its answer.

## Run in LM Studio

Loads and runs in **LM Studio** (tested on 0.4.20, mlx-llm runtime): the
`qwen3_5` architecture is recognized, the model indexes cleanly (`format: mlx`
present), and the ChatML template runs as-is. This is a **reasoning model** — it
emits a thinking channel (`reasoning_content`) before the final `content`, so give
it enough `max_tokens` (e.g. 200+) or the answer can be empty while it is still
thinking. Verified: "capital of France" → reasoning + `content` = "Paris".

**Note:** at 27 GB this needs a 32 GB+ Mac to load under LM Studio's default guardrails.

## Notes & limitations

- **Uncensored model.** This is a deliberately uncensored/abliterated derivative and
  will not refuse requests the way the original might. Use responsibly and in line
  with the base model's license and your local laws.
- **MTP head dropped.** The base model's multi-token-prediction (speculative
  decoding) head is not included — mlx-vlm strips the `mtp.*` weights on load. Text
  and vision are unaffected; only spec-decode is not available.
- `tie_word_embeddings=false`, so the ~1.27B `lm_head` is a separate matrix and is
  quantized. Verified correct on the MXFP4 build above.
- Inherits all capabilities and limitations of the base model. See the
  [original model card](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated).
- Quantized by [@sahilchachra](https://huggingface.co/sahilchachra) with MLX.
  Apache-2.0.