---
base_model: huihui-ai/Huihui-Qwen3.5-9B-abliterated
tags:
- llama.cpp
- quantized
- imatrix
---
# Auto-Quantized GGUF Model
This repository contains automated GGUF quantization files for [huihui-ai/Huihui-Qwen3.5-9B-abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.5-9B-abliterated).  

The calibration data for the imatrix is targeted at Chinese novels and role-playing (RP), while preserving logic and common sense.    
imatrix 的校准数据以中文的小说、角色扮演为目标，同时保留逻辑和常识。

If the perplexity drops after quantization compared to the original precision, it might not actually be an improvement. Instead, it could be caused by differences in how llama.cpp quantization and perplexity tools handle special tokens. I will update the README generation code in the future.
如果困惑度在量化之后与原精度相比变低，可能并不是真的提升，而是llamacpp量化工具和困惑度计算工具处理特殊token行为不同导致的，我将在未来修改生成readme的代码。


## Perplexity Evaluation
*(Tested against the provided calibration dataset)*

- **Base (F16/BF16)**: PPL = 16.9588 +/- 0.14520
- **IQ4_XS**: PPL = 14.1837 +/- 0.11750
- **IQ4_NL**: PPL = 14.1609 +/- 0.11721
- **Q4_K_S**: PPL = 14.1113 +/- 0.11654
- **Q4_K_M**: PPL = 14.0858 +/- 0.11636
- **Q5_K_S**: PPL = 13.9766 +/- 0.11525
- **Q5_K_M**: PPL = 13.9731 +/- 0.11528