Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,148 @@
+---
+license: other
+license_name: minimax-h3-community-license-agreement
+license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
+tags:
+- comfyui
+- nvfp4
+- quantized
+- video
+- text-to-video
+base_model: MiniMaxAI/MiniMax-H3
+base_model_relation: quantized
+---
+
+# MiniMax H3 — NVFP4
+
+NVFP4 quantizations of the **MiniMax-H3** diffusion transformer for ComfyUI, produced from
+the **unpruned bf16** weights.
+
+- Original model: https://huggingface.co/MiniMaxAI/MiniMax-H3
+- ComfyUI repackage these were built from: https://huggingface.co/Comfy-Org/MiniMax-H3
+
+Quantized with [comfy-kitchen](https://pypi.org/project/comfy-kitchen/) 0.2.26 via
+[comfyui-starnodes-modelconverter](https://github.com/Starnodes2024/comfyui-starnodes-modelconverter),
+using a hand-authored per-layer profile (included below).
+
+**NVFP4 requires an NVIDIA Blackwell GPU** (RTX 50-series, RTX PRO 6000, B200). On older
+architectures the NVFP4 path is emulated and you should use Comfy-Org's `int8_convrot`
+files instead.
+
+## Why a custom per-layer profile
+
+H3's parameters are not evenly distributed, and the biggest single group is the one most
+hostile to aggressive quantization:
+
+| group | params | share |
+|---|---|---|
+| `adaln_proj` (AdaLN modulation) | 13.04B | **39.4%** |
+| `mlp` (fc1/fc2) | 12.02B | 36.3% |
+| `attn` (qkv/out_proj) | 8.02B | 24.2% |
+| norms / embedders / final heads | 0.04B | 0.1% |
+| **total** | **33.12B** | |
+
+AdaLN layers emit the per-block scale and shift applied to every residual stream, so error
+there is multiplicative and compounds across all 50 blocks and every sampling step — unlike
+mlp/attn error, which tends to average out.
+
+The conventional "blacklist the sensitive layers" approach is **useless here**: keeping
+13.04B params at bf16 yields a ~36 GB file, *larger* than Comfy-Org's 34 GB
+`int8_convrot`, for no speed gain. There is no conservative-and-small option. Hence two
+deliberate variants.
+
+## Variants
+
+| file | AdaLN | attn + mlp | size | s/it |
+|---|---|---|---|---|
+| `minimax_h3_ref2va_nvfp4_mixed.safetensors` | FP8 (e4m3) | NVFP4 | 24.4 GB | 1.92 |
+| `minimax_h3_ref2va_nvfp4_full.safetensors` | NVFP4 | NVFP4 | 18.7 GB | 1.91 |
+
+Norms, biases, RoPE frequencies, patch/time/condition embedders and the final output heads
+are kept at BF16 in both (0.1% of params, negligible size cost).
+
+**`mixed` is the recommended default.**
+
+Note what the numbers say: quantizing AdaLN from FP8 down to NVFP4 saves **5.7 GB but buys
+no speed at all** (1.91 vs 1.92 s/it — noise). AdaLN projections are small per-block
+matmuls that are memory-bound rather than compute-bound, so FP4 tensor cores have nothing
+to accelerate there. All of the speedup comes from attn+mlp.
+
+So `full` is **not** the "faster" variant — it is the "smaller" variant, and it pays for
+that with 4-bit weights in the most error-sensitive 39% of the model. Use it only if you
+are VRAM-constrained. Otherwise take `mixed`.
+
+## Measured performance
+
+RTX PRO 6000 Blackwell (96 GB), ComfyUI 0.30.0, ref2va, 864×480, 39 frames, 20 steps,
+`res_multistep` / `beta`:
+
+| model | source | size | s/it |
+|---|---|---|---|
+| `pruned_int8_convrot` (Comfy-Org) | pruned | 21.0 GB | 2.18 |
+| `nvfp4_mixed` (this repo) | unpruned bf16 | 24.4 GB | **1.92** |
+| `nvfp4_full` (this repo) | unpruned bf16 | 18.7 GB | 1.91 |
+
+~12% faster sampling than the pruned int8 baseline, from unpruned source weights.
+Baseline `--highvram` / `--fast` made no measurable difference (2.18 s/it either way), so
+the quantization change is the only thing that moved this number.
+
+## Honest limitations
+
+- Quality was spot-checked against the pruned-int8 baseline at matched seeds and showed no
+  visible degradation, but this is **not a rigorous evaluation** — no FVD, no human study,
+  no long-duration or 2K testing.
+- Only the **ref2va** task is converted so far. `fl2va` is not included.
+- `full` (AdaLN at NVFP4) is the more speculative variant and has had less testing. It
+  showed no visible artifacts at matched seeds, but 4-bit modulation weights are the kind
+  of thing that fails on hard cases rather than easy ones.
+- Benchmarks are single-GPU on one card at one resolution.
+
+If you find failure cases, please open a discussion — concrete artifacts are more useful
+than aggregate scores for judging whether the AdaLN policy is right.
+
+## Usage
+
+Place in `ComfyUI/models/diffusion_models/`, and pair with the text encoder and VAEs from
+Comfy-Org's repo:
+
+```
+📂 ComfyUI/
+├── 📂 models/
+│   ├── 📂 diffusion_models/
+│   │   └── minimax_h3_ref2va_nvfp4_mixed.safetensors
+│   ├── 📂 text_encoders/
+│   │   └── qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors
+│   ├── 📂 vae/
+│   │   ├── minimax_h3_audio_vae_fp32.safetensors
+│   │   └── minimax_h3_video_vae_fp16.safetensors
+```
+
+Use the official R2V template, swapping the diffusion model:
+https://github.com/Comfy-Org/workflow_templates/blob/main/templates/video_minimax_h3_r2v.json
+
+Requires a ComfyUI new enough to have native H3 support (`comfy/ldm/minimax/`,
+`comfy_extras/nodes_minimax_h3.py`) — ComfyUI ≥ 0.30.0.
+
+Notes: `CLIPLoader` type must be `minimax`; sampler `res_multistep`; frame `length` must
+satisfy 17n+5.
+
+## Reproducing
+
+`profiles/` in this repo contains the two per-layer profiles. To rebuild:
+
+```bash
+pip install comfy-kitchen
+git clone https://github.com/Starnodes2024/comfyui-starnodes-modelconverter \
+  ComfyUI/custom_nodes/comfyui-starnodes-modelconverter
+# copy profiles/*.json into that node pack's profiles/ dir, then run
+# StarUltimateModelConverterPro with:
+#   model_name = minimax_h3_ref2va_bf16.safetensors
+#   profile    = minimax_h3_nvfp4_mixed.json
+#   use_blacklist = no        (the profile controls precision per-tensor)
+#   target_quant_format = NVFP4
+```
+
+## License
+
+Inherits the MiniMax-H3 Community License Agreement from the original model. See the
+license link above.