---
license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated
base_model_relation: quantized
quantized_by: blockblockblock
library_name: exllamav3
pipeline_tag: text-generation
tags:
  - exl3
  - exllamav3
  - quantized
  - mixture-of-experts
  - qwen
quantization_format: exl3
bits_per_weight: 4.5
---

<div align="center">

# Huihui · Qwen3.6 · 35B-A3B · abliterated

<sub><code>EXL3</code> &nbsp;·&nbsp; <b>4.5&nbsp;bpw</b> &nbsp;·&nbsp; 21.6&nbsp;GB &nbsp;·&nbsp; Mixture&#8209;of&#8209;Experts &nbsp;·&nbsp; 40 layers &times; 256 experts</sub>

<br/>

[![format](https://img.shields.io/badge/format-EXL3-c63010?style=for-the-badge&labelColor=14120e)](https://github.com/turboderp-org/exllamav3)
[![bpw](https://img.shields.io/badge/bpw-4.5-6b8a76?style=for-the-badge&labelColor=14120e)](#quants)
[![size](https://img.shields.io/badge/size-21.6_GB-6b8a76?style=for-the-badge&labelColor=14120e)](#quants)
[![arch](https://img.shields.io/badge/arch-MoE_36B--A3B-c63010?style=for-the-badge&labelColor=14120e)](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated)

[![base model](https://img.shields.io/badge/Base-huihui--ai%2FHuihui--Qwen3.6--35B--A3B--abliterated-2a2620?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated)
[![quantized by](https://img.shields.io/badge/Quantized_by-blockblockblock-2a2620?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/blockblockblock)
[![collection](https://img.shields.io/badge/All_bpws-Collection-c63010?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/collections/blockblockblock/huihui-qwen36-35b-a3b-abliterated-exl3-69ef8197714f5baf5d4b802d)

</div>

---

> [!NOTE]
> An [ExLlamaV3](https://github.com/turboderp-org/exllamav3) build of [`huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated`](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated) at **4.5 bits per weight** &mdash; quality-leaning build &mdash; comfortable on a 24&nbsp;GB card with usable context (~22&nbsp;GB weights). See [Quants](#quants) for sibling repos at other bit&#8209;widths or browse the [collection](https://huggingface.co/collections/blockblockblock/huihui-qwen36-35b-a3b-abliterated-exl3-69ef8197714f5baf5d4b802d).

## Quants

<div align="center">

| BPW &nbsp; | &nbsp; Head bits &nbsp; | &nbsp; Calibration rows &nbsp; | &nbsp; Size &nbsp; | &nbsp; Status |
| :---: | :---: | :---: | ---: | :--- |
| 4.0 | 8 | 128 | 19.5&nbsp;GB | [link](https://huggingface.co/blockblockblock/Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.0bpw) |
| **4.5** | 8 | 250 | **21.6&nbsp;GB** | <kbd>this repo</kbd> |

</div>

## Inference

<table>
  <thead>
    <tr>
      <th align="left" width="32%">Loader</th>
      <th align="left">Use it for</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><a href="https://github.com/theroyallab/tabbyAPI"><b>TabbyAPI</b></a></td>
      <td>OpenAI&#8209;compatible HTTP server. Drop&#8209;in for OpenAI clients.</td>
    </tr>
    <tr>
      <td><a href="https://github.com/oobabooga/text-generation-webui"><b>text&#8209;generation&#8209;webui</b></a></td>
      <td>Local chat UI. Pick the <i>ExLlamaV3</i> loader from the model dropdown.</td>
    </tr>
    <tr>
      <td><a href="https://github.com/turboderp-org/exllamav3"><b>ExLlamaV3</b></a></td>
      <td>Direct Python API for embedding the model in your own code or pipeline.</td>
    </tr>
  </tbody>
</table>

> [!TIP]
> **VRAM at 4.5&nbsp;bpw:** ~21.6&nbsp;GB weights + ~2&nbsp;GB context. Comfortable on a single 24&nbsp;GB card; tight on 16&nbsp;GB with reduced context.

## Download

```bash
pip install -U huggingface_hub

hf download \
  blockblockblock/Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.5bpw \
  --local-dir ./Huihui-Qwen3.6-35B-A3B-abliterated-exl3-4.5bpw
```

<details>
<summary><b>Quantization recipe</b> &nbsp;<sub>(advanced &mdash; embedded in <code>quantization_config.json</code>)</sub></summary>

<br/>

| Setting | Value |
| :--- | :--- |
| Format | `EXL3` |
| Bits per weight | `4.5` |
| Head bits | `8` |
| Calibration rows | `250` |
| Codebook | `MCG` |
| Out&#8209;scales | `always` |
| Parallel mode | `enabled` (MoE expert batching) |

Loaded automatically by every ExLlamaV3 loader; reproduced here for searchability.

</details>

## License &amp; use

> [!IMPORTANT]
> Use and license **follow the [base model](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated)**.
> Quantization adds no additional restrictions. Refer to the upstream repository for terms, citation, and safety documentation.

---

<div align="center">
  <sub><i>Quantized with <a href="https://github.com/Honkware/blockquant"><b>BlockQuant</b></a> &nbsp;·&nbsp; convention&nbsp;<code>{org}/{model}-exl3-{bpw}bpw</code></i></sub>
</div>