---
base_model:
- Qwen/Qwen3.5-397B-A17B
---
## Updates
- 5/19/2026: I've uploaded new quants that include the MTP Tensors (@ Q8_0).
- 3/10/2026: I've uploaded new quants using the new fused Up + Gate conversion, this offers up to a +10% boost in prompt processing speed from my testing.

## Description
This repo contains specialized MoE-quants for Qwen3.5-397B-A17B. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, 
it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. 
To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.

| Quant   | Size                  | Mixture                            | PPL                 | 1-(Mean PPL(Q)/PPL(base)) | KLD                 |
| :------ | :-------------------- | :--------------------------------- | :------------------ | :------------------------ | :------------------ |
| Q8_0    | 399.08 GiB (8.51 BPW) | Q8_0                               | 3.486040 ± 0.018831 | +0.0501%                  | 0.003298 ± 0.000034 |
| Q5_K_M  | 280.05 GiB (5.97 BPW) | Q8_0 / Q5_K / Q5_K / Q6_K          | 3.488222 ± 0.018848 | +0.1127%                  | 0.004782 ± 0.000042 |
| Q4_K_M  | 234.11 GiB (4.99 BPW) | Q8_0 / Q4_K / Q4_K / Q5_K          | 3.496714 ± 0.018906 | +0.3565%                  | 0.008720 ± 0.000078 |
| IQ4_XS  | 183.48 GiB (3.91 BPW) | Q8_0 / IQ3_S / IQ3_S / IQ4_XS      | 3.542040 ± 0.019138 | +1.6573%                  | 0.022898 ± 0.000190 |
| IQ3_S   | 142.87 GiB (3.05 BPW) | Q6_K / IQ2_S / IQ2_S / IQ3_S       | 3.671774 ± 0.020020 | +5.3807%                  | 0.064287 ± 0.000503 |
| IQ2_S   | 129.74 GiB (2.77 BPW) | Q6_K / IQ2_XS / IQ2_XS / IQ3_XXS   | 3.778894 ± 0.020751 | +8.4551%                  | 0.093763 ± 0.000714 |
| IQ2_XXS | 120.47 GiB (2.57 BPW) | Q4_K / IQ2_XXS / IQ2_XXS / IQ3_XXS | 3.878830 ± 0.021458 | +11.3233%                 | 0.126053 ± 0.000896 |

![kld_graph](kld_data/01_kld_vs_filesize.png "Chart showing Pareto KLD analysis of quants")
![ppl_graph](kld_data/02_ppl_vs_filesize.png "Chart showing Pareto PPL analysis of quants")