Initial NVFP4 model card

README.md

@@ -0,0 +1,118 @@
+---
+license: other
+license_name: modified-mit
+license_link: https://huggingface.co/MiniMaxAI/MiniMax-M2.7/blob/main/LICENSE
+base_model: dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B
+base_model_relation: quantized
+tags:
+  - minimax
+  - moe
+  - reap
+  - nvfp4
+  - fp4
+  - blackwell
+  - compressed-tensors
+  - vllm
+  - text-generation
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4
+
+**NVFP4** quantization of [`dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B`](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B) — the first publicly available REAP-40 % pruned variant of MiniMax-M2.7 — targeting NVIDIA Blackwell (B100 / B200) for native FP4 tensor-core acceleration.
+
+| Aspect | Value |
+|---|---|
+| Base model | `dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B` (BF16) |
+| Quantization | NVFP4A16 (4-bit microscaled floating point weights, FP16 activations) |
+| Format | `compressed-tensors` (vLLM-native) |
+| Tool | [`llmcompressor`](https://github.com/vllm-project/llm-compressor) |
+| File size | ~75 GB across ~25 safetensors shards |
+| Ignored layers | `lm_head` (kept in BF16) |
+
+## What is NVFP4?
+
+NVFP4 is NVIDIA's 4-bit floating-point microscaling format introduced with the Blackwell architecture. It uses small block-wise scale factors to maintain quality at extreme compression, and benefits from dedicated FP4 tensor cores on B100/B200 hardware.
+
+Compared to INT4 / AWQ quantization, NVFP4 typically preserves quality better at the same weight budget, particularly on reasoning-heavy workloads. Our REAP-pruned base model is an ideal candidate — the structural pruning has already reduced parameter count, and NVFP4 then packs each remaining weight into 4 bits.
+
+## Hardware & deployment
+
+**Native FP4 tensor-core acceleration requires Blackwell (B100 / B200)**. The quantized weights also load and run on Hopper (H100 / H200) and Ampere (A100) via FP4-to-higher-precision upcasting — functional but not at Blackwell speed.
+
+Memory footprint: ~75 GB weights + KV cache. Recommended:
+- 1× B100 / B200 (native NVFP4, best performance)
+- 2× H100 80 GB or 1× H200 141 GB (functional, no native FP4 cores)
+- Memory-constrained: combine with KV cache quantization (see vLLM docs)
+
+## Inference
+
+### vLLM
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(
+    model="dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4",
+    tensor_parallel_size=1,       # fits on 1× Blackwell or 2× Hopper
+    trust_remote_code=True,
+    max_model_len=32768,
+)
+
+params = SamplingParams(temperature=1.0, top_p=0.95, top_k=40, max_tokens=2048)
+out = llm.generate(["Explain REAP pruning briefly."], params)
+print(out[0].outputs[0].text)
+```
+
+### TensorRT-LLM
+
+Supported via the `compressed-tensors` loader in TensorRT-LLM 0.14+ with NVFP4 scheme. Consult NVIDIA's deployment guide for Blackwell-specific kernels.
+
+## Quality
+
+Inference quality was validated on the BF16 parent via a 5 / 5 pre-publish smoke test and full HumanEval evaluation (see [parent safetensors card](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B)). NVFP4A16 is expected to track FP8 / BF16 quality very closely thanks to microscaling — activations remain in FP16 so only weights are compressed.
+
+Systematic NVFP4-on-REAP evaluation is pending; we will update this card if there is community demand.
+
+## Base model summary
+
+| Property | Value |
+|---|---|
+| Architecture | MoE, 62 layers, 154 experts (pruned from 256), top-8 routing |
+| Active parameters / token | ~10 B |
+| Total parameters | ~139 B |
+| Max position embeddings | 196,608 |
+| Vocabulary size | 200,064 |
+| Pruning | REAP 40 %, seed 42, calibration on 3 × 2,048 samples (code / math / tool) |
+
+See the [parent safetensors card](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B) for full architecture, pruning details, evaluation numbers, and the known minor layer-0 bias imperfection.
+
+## Recommended generation parameters
+
+- `temperature`: 1.0
+- `top_p`: 0.95
+- `top_k`: 40
+- `repeat_penalty`: 1.05
+
+## Companion repos
+
+- **Parent safetensors (BF16)**: [`dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B`](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B)
+- **GGUF** (Mac / llama.cpp / Ollama / LM Studio): [`dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B-GGUF`](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B-GGUF)
+- **FP8** (Hopper-native): [`dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B-FP8`](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B-FP8)
+- **AWQ-4bit** (vLLM / HF Transformers INT4): coming soon
+
+## Citation
+
+See the [safetensors repo](https://huggingface.co/dervig/m51Lab-MiniMax-M2.7-REAP-139B-A10B#citation) for full citations. Core references:
+
+- Lasby et al., **REAP the Experts** (arXiv:2510.13999)
+- MiniMax AI, [**MiniMax-M2.7**](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)
+
+## License
+
+Inherits the [Modified MIT License](https://huggingface.co/MiniMaxAI/MiniMax-M2.7/blob/main/LICENSE) from MiniMaxAI/MiniMax-M2.7.
+
+---
+
+_Published by [m51Lab](https://m51.ai) — open-source LLM contributions from the M51 AI OS group._