---
base_model: huihui-ai/Huihui-Qwen3.5-9B-abliterated
library_name: gguf
tags:
- qwen
- qwen3.5
- abliterated
- uncensored
- gguf
- text-generation
---

# Huihui-Qwen3.5-9B-abliterated - GGUF

This repository contains GGUF format model files for [huihui-ai/Huihui-Qwen3.5-9B-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.5-9B-abliterated). 


## 💾 Provided Files and Quantizations

We provide a spectrum of quantizations to suit different hardware capabilities. If you are unsure which to choose, **Q4_K_M** is the recommended standard for the best balance of speed, size, and reasoning retention.

| Filename | Quant Type | File Size | Recommended RAM/VRAM | Use Case |
| :--- | :--- | :--- | :--- | :--- |
| `Huihui-Qwen3.5-9B-abliterated-Q4_K_S.gguf` | Q4_K_S | ~5.1 GB | 8 GB | Fastest inference, lowest memory footprint. |
| `Huihui-Qwen3.5-9B-abliterated-Q4_K_M.gguf` | Q4_K_M | ~5.4 GB | 8 GB | **Recommended.** Best balance of quality and size. |
| `Huihui-Qwen3.5-9B-abliterated-Q5_K_S.gguf` | Q5_K_S | ~6.1 GB | 10 GB | Slightly higher quality, needs more memory. |
| `Huihui-Qwen3.5-9B-abliterated-Q5_K_M.gguf` | Q5_K_M | ~6.3 GB | 10 GB | Excellent reasoning retention for 12GB+ GPUs. |
| `Huihui-Qwen3.5-9B-abliterated-Q6_K.gguf` | Q6_K | ~7.4 GB | 12 GB | Near-lossless quality. |
| `Huihui-Qwen3.5-9B-abliterated-Q8_0.gguf` | Q8_0 | ~9.6 GB | 16 GB | Virtually indistinguishable from the F16 base model. |

## 🚀 How to Use

These GGUF files are fully compatible with popular local AI software, including:
* **LM Studio**
* **Ollama**
* **Jan**
* **text-generation-webui (Oobabooga)**
* **llama.cpp**

### Prompt Format (ChatML)
Qwen models strictly use the ChatML format for prompting. Your system prompt and user inputs must be wrapped exactly like this:

```text
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Write a simple Python script.<|im_end|>
<|im_start|>assistant