Create README.md

README.md

@@ -0,0 +1,28 @@
+---
+pipeline_tag: text-generation
+license: other
+license_name: modified-mit
+license_link: https://github.com/MiniMax-AI/MiniMax-M2.7/blob/main/LICENSE
+library_name: exllamav3
+base_model: MiniMaxAI/MiniMax-M2.7
+base_model_relation: quantized
+tags:
+  - exl3
+---
+
+Quantization was performed using [exllama3 v0.0.29](https://github.com/turboderp-org/exllamav3).
+
+[4.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.7-exl3/tree/4.0bpw)  
+[5.0bpw](https://huggingface.co/NeuroSenko/MiniMax-M2.7-exl3/tree/5.0bpw)  
+
+<details>
+<summary>Quantization Notes</summary>
+
+### Inf/NaN values in calibration data
+
+Some experts in the model produce `inf` values during calibration (e.g. experts 61 and 74 in the last layer had inf values in their down-projection calibration state). The `lm_head` layer also exhibited NaN values in its calibration state (445K NaN out of 1.5B elements).
+                                                                                                                                                                                              
+This causes Cholesky decomposition to fail during quantization, as the Hessian matrix is no longer positive-definite. ExLlamaV3 does not handle this case gracefully — quantization crashes after exhausting retry attempts. A local patch was applied to fall back to uncalibrated quantization for the affected tensors. Given that only a handful of experts out of 256 in the last layer are affected, the impact on output quality is expected to be minimal.
+                                                                  
+This appears to be a property of the model weights themselves, not a bug in the quantizer.
+</details>