---
base_model:
- stepfun-ai/Step-3.5-Flash
---

## Updates
- 6/3/2026: I've updated all of the quants to include the updated GGUF conversion + MTP heads (as Q8_0)

This repo contains specialized MoE-quants for Step-3.5-Flash. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.

| Quant  | Size                  | Mixture                          | PPL                 | 1-(Mean PPL(Q)/PPL(base)) | KLD                 |
| :----- | :-------------------- | :------------------------------- | :------------------ | :------------------------ | :------------------ |
| Q8_0   | 197.43 GiB (8.51 BPW) | Q8_0                             | 2.419187 ± 0.011093 | +0.1197%                  | 0.005594 ± 0.000041 |
| Q5_K_M | 138.83 GiB (5.98 BPW) | Q8_0 / Q5_K / Q5_K / Q6_K        | 2.431812 ± 0.011175 | +0.6421%                  | 0.015249 ± 0.000098 |
| Q4_K_M | 116.22 GiB (5.01 BPW) | Q8_0 / Q4_K / Q4_K / Q5_K        | 2.477162 ± 0.011502 | +2.5190%                  | 0.040282 ± 0.000237 |
| IQ4_XS | 91.31 GiB (3.93 BPW)  | Q8_0 / IQ3_S / IQ3_S / IQ4_XS    | 2.685042 ± 0.012863 | +11.1222%                 | 0.127994 ± 0.000696 |
| IQ3_S  | 70.89 GiB (3.05 BPW)  | Q6_K / IQ2_S / IQ2_S / IQ3_S     | 3.313628 ± 0.017314 | +37.1367%                 | 0.355228 ± 0.001705 |
| IQ2_S  | 64.43 GiB (2.78 BPW)  | Q6_K / IQ2_XS / IQ2_XS / IQ3_XXS | 3.772340 ± 0.020688 | +56.1208%                 | 0.488752 ± 0.002225 |

![kld_graph](kld_data/01_kld_vs_filesize.png "Chart showing Pareto KLD analysis of quants")
![ppl_graph](kld_data/02_ppl_vs_filesize.png "Chart showing Pareto PPL analysis of quants")