--- license: apache-2.0 base_model: nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451 base_model_relation: quantized pipeline_tag: text-generation tags: - gguf - llama.cpp - qwen3.6 - quantized - moq - mtp - nvfp4 --- # Qwen3.6-27B Architect Polaris2 Fable B F451 - MTP MoQ GGUF Tensor-aware mixed-precision GGUF quantizations of [nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451](https://huggingface.co/nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451), including the model's MTP tensors. This repository provides several MoQ size/quality points, an experimental NVFP4 variant, the c128 importance matrix used during quantization, and the complete interactive evaluation report. ## Quantization method MoQ uses tensor-aware mixed quantization instead of assigning one quantization type to the entire model. - Weight tensors are grouped by their structural role and sensitivity. - Per-group KLD sensitivity studies are used to decide which groups require higher precision. - Less sensitive groups use more compact formats, while sensitive groups are protected with higher-precision formats. - A 128-chunk importance matrix is incorporated into the quantization process. - The large MTP tensors are explicitly retained at high precision. - Every output was audited against its tensor recipe: all 866 tensor names, shapes, and quantization types matched. The `MTP-MoQ-NVFP4` variant uses NVFP4 for selected tensor groups while retaining the rest of the mixed-precision policy. On an RTX 5090 it improves short-prompt prefill throughput substantially, although the regular MoQ-4.8 model provides better KLD quality at the same file size. No single quant is best for every use case: - `MTP-MoQ-4.6` is a compact balanced option. - `MTP-MoQ-4.8` is the recommended quality/size point around 16.15 GB. - `MTP-MoQ-5.1` provides the best quality in the regular MoQ series. - `MTP-MoQ-NVFP4` is intended for testing NVFP4 acceleration, especially on Blackwell GPUs. ## Files Sizes are decimal GB. Payload BPW is calculated from the actual GGUF tensor payload and excludes the GGUF header. | File | Size GB | Payload BPW | PPL | Mean KLD | p999 KLD | |---|---:|---:|---:|---:|---:| | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-3.2.gguf` | 10.810 | 3.1621 | 6.805998 | 0.097714 | 2.751078 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-3.6.gguf` | 12.134 | 3.5500 | 6.661694 | 0.069665 | 1.796203 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-3.8.gguf` | 12.840 | 3.7566 | 6.518863 | 0.044993 | 1.291825 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.1.gguf` | 14.266 | 4.1740 | 6.447097 | 0.027318 | 0.807045 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.3.gguf` | 15.002 | 4.3897 | 6.421027 | 0.019253 | 0.591232 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.6.gguf` | 15.242 | 4.4599 | 6.392239 | 0.015145 | 0.505663 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.8.gguf` | 16.150 | 4.7258 | 6.379456 | 0.012836 | 0.423877 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-4.9.gguf` | 16.524 | 4.8353 | 6.384560 | 0.012218 | 0.402534 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-5.1.gguf` | 17.452 | 5.1069 | 6.371239 | 0.009613 | 0.325677 | | `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-MTP-MoQ-NVFP4.gguf` | 16.150 | 4.7258 | 6.397915 | 0.015615 | 0.518097 | Additional files: - `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-BF16.gguf`: BF16 GGUF reference model. - `Qwen3.6-27B-Architect-Polaris2-Fable-B-F451.c128.imatrix.gguf`: the 128-chunk importance matrix used for these quantizations. - `Jianqiao1_MoQ_vs_DavidAU_GGUF_report.html`: self-contained interactive report with zoomable and pannable charts. ## Quality evaluation All quality results were measured under the same conditions: - WikiText-2 raw test set - Context length 512 - 580 evaluation chunks - KLD calculated against logits from the BF16 model - BF16 reference PPL: `6.320035` Lower PPL, Mean KLD, and p999 KLD are better. ### MoQ results | Model | Size GB | Payload BPW | PPL | Mean KLD | p999 KLD | |---|---:|---:|---:|---:|---:| | MTP-MoQ-3.2 | 10.810 | 3.1621 | 6.805998 | 0.097714 | 2.751078 | | MTP-MoQ-3.6 | 12.134 | 3.5500 | 6.661694 | 0.069665 | 1.796203 | | MTP-MoQ-3.8 | 12.840 | 3.7566 | 6.518863 | 0.044993 | 1.291825 | | MTP-MoQ-4.1 | 14.266 | 4.1740 | 6.447097 | 0.027318 | 0.807045 | | MTP-MoQ-4.3 | 15.002 | 4.3897 | 6.421027 | 0.019253 | 0.591232 | | MTP-MoQ-4.6 | 15.242 | 4.4599 | 6.392239 | 0.015145 | 0.505663 | | MTP-MoQ-4.8 | 16.150 | 4.7258 | 6.379456 | 0.012836 | 0.423877 | | MTP-MoQ-4.9 | 16.524 | 4.8353 | 6.384560 | 0.012218 | 0.402534 | | MTP-MoQ-5.1 | 17.452 | 5.1069 | 6.371239 | 0.009613 | 0.325677 | | MTP-MoQ-NVFP4 | 16.150 | 4.7258 | 6.397915 | 0.015615 | 0.518097 | ### Comparison with DavidAU Fable Fusion 711 GGUF The following DavidAU GGUF files were evaluated locally using the same BF16 reference logits and the same evaluation settings. | DavidAU quant | Size GB | Payload BPW | PPL | Mean KLD | p999 KLD | |---|---:|---:|---:|---:|---:| | IQ2_M | 12.125 | 3.5471 | 7.100246 | 0.140480 | 3.807066 | | IQ3_M | 14.532 | 4.2520 | 6.545219 | 0.047823 | 1.487676 | | IQ4_XS | 17.034 | 4.9846 | 6.387851 | 0.013953 | 0.514973 | | Q4_K_S | 17.537 | 5.1321 | 6.378748 | 0.013965 | 0.492694 | | IQ4_NL | 17.753 | 5.1952 | 6.386419 | 0.013701 | 0.492569 | | Q4_K_M | 18.499 | 5.4135 | 6.365021 | 0.011404 | 0.412386 | | Q5_K_S | 20.631 | 6.0379 | 6.341159 | 0.005466 | 0.211335 | | Q5_K_M | 21.182 | 6.1993 | 6.337103 | 0.004809 | 0.179674 | | Q6_K | 24.034 | 7.0343 | 6.324763 | 0.001467 | 0.059772 | ### Near-size comparison Positive percentages mean the MoQ model is smaller or has a lower metric. | MoQ model | DavidAU model | Size difference | Mean KLD lower | p999 KLD lower | PPL lower | |---|---|---:|---:|---:|---:| | MTP-MoQ-3.6 | IQ2_M | -0.08% | 50.41% | 52.82% | 6.18% | | MTP-MoQ-4.1 | IQ3_M | 1.83% | 42.88% | 45.75% | 1.50% | | MTP-MoQ-4.8 | IQ4_XS | 5.19% | 8.01% | 17.69% | 0.13% | | MTP-MoQ-5.1 | Q4_K_S | 0.49% | 31.16% | 33.90% | 0.12% | ## Quality charts ### Model size vs p999 KLD ![Model size vs p999 KLD](p999_kld.png) ### Model size vs Mean KLD ![Model size vs Mean KLD](mean_kld.png) ### Model size vs WikiText-2 PPL ![Model size vs WikiText-2 PPL](ppl.png) For interactive zooming, panning, series selection, and detailed hover values, download and open [`Jianqiao1_MoQ_vs_DavidAU_GGUF_report.html`](Jianqiao1_MoQ_vs_DavidAU_GGUF_report.html). ## RTX 5090 performance Throughput was measured with f16 KV cache and three repetitions. `MTP-MoQ-NVFP4` and `MTP-MoQ-4.8` have exactly the same 16.150 GB file size. | Model | pp512 tok/s | tg128 tok/s | pg32768,256 tok/s | |---|---:|---:|---:| | MTP-MoQ-NVFP4 | 3256.97 | 78.92 | 2377.72 | | MTP-MoQ-4.8 | 2669.45 | 78.65 | 2356.68 | | MTP-MoQ-4.6 | 2731.86 | 82.87 | 2400.43 | | NVFP4 vs MoQ-4.8 | +22.01% | +0.34% | +0.89% | ## License This repository is released under the Apache License 2.0. Users should also review and comply with the terms of the original model. ## Acknowledgements - Thanks to [nightmedia](https://huggingface.co/nightmedia) for the original Qwen3.6-27B Architect Polaris2 Fable B F451 model. - Thanks to DavidAU for the Fable Fusion 711 GGUF series used in the comparison. - Thanks to the [Qwen team](https://huggingface.co/Qwen) for Qwen3.6.