DmitryDB
/

MiniMax-H3-ComfyUI-Quants

@@ -22,216 +22,145 @@ tags:
 Community quantized diffusion-transformer checkpoints for
 [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3).
-The repository contains separate **FL2VA** and **Ref2VA** models and a range of
-INT8, W8/W4, W4, and NVFP4 precision profiles.
-These checkpoints are not official MiniMax or ComfyOrg releases. The models in
-this repository use the standard MiniMax-H3 time-table and fused-QKV layout and
-load in an unmodified recent ComfyUI. No core patch or custom model loader is
-required.
-The filenames are intentionally short. Full implementation details, layer
-counts, compatibility notes, and measured VRAM behavior are documented below.
-## Choose a model
-Download exactly one diffusion model for the task you use:
-- **FL2VA** — text-to-audio-video and optional first frame, last frame, or both;
-- **Ref2VA** — reference-to-audio-video using reference images, videos, and/or audio.
-The same profile names are provided under both `FL2VA/` and `Ref2VA/`.
-| Canonical filename | Quant format and intent | File size |
-|---|---|---:|
-| `minimax-h3-{variant}-int8-convrot-safe.safetensors` | INT8 ConvRot with the largest VRAM margin | 20.330 GiB |
-| `minimax-h3-{variant}-int8-convrot-balanced.safetensors` | Recommended INT8 quality/size balance | 20.940 GiB |
-| `minimax-h3-{variant}-int8-convrot-max.safetensors` | INT8 ConvRot with the largest BF16 quality island | 21.908 GiB |
-| `minimax-h3-{variant}-w8w4-convrot-balanced.safetensors` | Mixed W8/W4 ConvRot | 13.565 GiB |
-| `minimax-h3-{variant}-w4-convrot-compact.safetensors` | W4 ConvRot main transformer with INT8 refiner | 10.067 GiB |
-| `minimax-h3-{variant}-w4-convrot-offload.safetensors` | Smallest universal W4 ConvRot checkpoint | 9.708 GiB |
-| `minimax-h3-{variant}-nvfp4-quality.safetensors` | NVFP4 with a BF16 quality island | 13.597 GiB |
-| `minimax-h3-{variant}-nvfp4-compact.safetensors` | Compact all-edge NVFP4 | 10.862 GiB |
-Replace `{variant}` with `fl2va` or `ref2va`. Resolution, frame count,
-attention backend, text encoder, VAE, and ComfyUI offload policy all affect
-peak VRAM; checkpoint size is not the same as full-workflow VRAM.
-## GPU recommendations and why
-| GPU generation | VRAM | Recommended starting profile | Why |
-|---|---:|---|---|
-| RTX 30 or RTX 40 | 8 GiB | `w4-convrot-offload` | Smallest portable checkpoint. It still requires CPU offload, but reduces repeated PCIe weight traffic compared with a 20+ GiB INT8 model. |
-| RTX 30 or RTX 40 | 12 GiB | `w4-convrot-compact` | Leaves more room for activations than mixed W8/W4 while retaining an INT8 token refiner. |
-| RTX 30 or RTX 40 | 16 GiB | `w8w4-convrot-balanced` | Uses W8 on the more quality-sensitive subset and W4 elsewhere; a better quality/speed compromise when 20 GiB INT8 cannot remain resident. |
-| RTX 30 or RTX 40 | 24 GiB | `int8-convrot-balanced` | Portable fast path with the full model resident on the tested RTX 4090. It retains the balanced BF16 island without the offload seen in `max`. |
-| RTX 30 or RTX 40 | 24 GiB, activation-heavy workflow | `int8-convrot-safe` | Also fully resident on the tested RTX 4090 and leaves about 0.6 GiB more free VRAM than `balanced`. |
-| RTX 30 or RTX 40 | 32 GiB or more | `int8-convrot-max` | Largest BF16 island and enough memory to avoid the small offload measured on a 24 GiB 4090. |
-| RTX 50 / Blackwell | 8–12 GiB | `nvfp4-compact` | Blackwell-native block-scaled NVFP4 gives the smallest architecture-specific checkpoint. Use W4 ConvRot as the portable fallback. |
-| RTX 50 / Blackwell | 16–24 GiB | `nvfp4-quality` | Keeps the balanced BF16 quality island while using Blackwell-native NVFP4 for the other main weights, leaving much more activation headroom. |
-| RTX 50 / Blackwell | 24 GiB, portability preferred | `int8-convrot-balanced` | Same file works across RTX 30/40/50 and avoids dependence on the NVFP4 runtime path. |
-| RTX 50 / Blackwell | 32 GiB or more | `int8-convrot-max` for quality, `nvfp4-quality` for speed/headroom | The larger card removes the reason to offload `max`; NVFP4 remains attractive when throughput and free VRAM matter more. |
-RTX 4090 recommendations are backed by the measurements below. RTX 50 NVFP4
-recommendations are architecture-based; this machine has no local RTX 5090,
-so they are not presented as measured 5090 generation results. NVFP4 is plain
-block-scaled NVFP4, not activation-aware AWQ.
-## Measured RTX 4090 loading
-FL2VA and Ref2VA were tested independently on a 24 GiB RTX 4090 through the
-stock ComfyUI loader. A real resident INT8 projection was executed after load.
-| Profile | Loaded weight fraction | Peak reserved VRAM | Free after load | Result |
 |---|---:|---:|---:|---|
 | `int8-convrot-safe` | 100% | 20.424 GiB | 2.072 GiB | PASS |
 | `int8-convrot-balanced` | 100% | 21.025 GiB | 1.471 GiB | PASS |
-| `int8-convrot-max` | 95.6% | 21.002 GiB | 1.494 GiB | PASS with about 0.955 GiB of weights offloaded |
-These are loader and quantized-kernel measurements, not full video-generation
-peak figures.
-## What was quantized
-The raw FL2VA and Ref2VA Diffusers transformers each contain 50 main
-transformer blocks. The conversion preserves every block and uses a
-layer-sensitive policy instead of uniformly quantizing all matrices.
-### INT8 profiles
-| Profile | INT8 semantic matrices | BF16 semantic matrices | Notes |
-|---|---:|---:|---|
-| `int8-convrot-safe` | 185 | 23 | Maximum VRAM headroom of the three INT8 profiles |
-| `int8-convrot-balanced` | 170 | 38 | 30 BF16 main matrices plus the BF16 token refiner |
-| `int8-convrot-max` | 145 | 63 | Largest BF16 quality island |
-INT8 weights use ConvRot/Hadamard rotation with group size 256, per-row FP32
-scales, and deterministic scale search. The BF16 islands were selected from
-independent FL2VA and Ref2VA sampled-reconstruction measurements. Norms,
-conditioning projections, patch projections, output heads, and other small or
-sensitive tensors retain their appropriate source precision.
-The exact main-transformer BF16 islands are:
-| Profile | BF16 attention output blocks | BF16 MLP `fc2` blocks | Everything else among the 200 main semantic matrices |
 |---|---|---|---|
 | `int8-convrot-safe` | 0, 1, 2, 3, 4, 5, 6, 7, 9, 15, 19, 38, 45, 49 | 49 | INT8 ConvRot |
 | `int8-convrot-balanced` | 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 27, 38, 43, 44, 45, 46, 47, 49 | 39, 45, 49 | INT8 ConvRot |
 | `int8-convrot-max` | all blocks 0–49 | 29, 39, 44, 45, 49 | INT8 ConvRot |
-For all three profiles the eight token-refiner semantic matrices remain BF16.
-The machine-readable validation reports provide the saved-tensor inventory for
-each FL2VA and Ref2VA file.
-### W4 and NVFP4 profiles
-- `w8w4-convrot-balanced`: 86 W8 and 114 W4 main matrices; token refiner remains BF16.
-- `w4-convrot-compact`: 200 W4 main matrices and 8 INT8 token-refiner matrices.
-- `w4-convrot-offload`: 208 W4 main/refiner matrices.
-- `nvfp4-quality`: 170 NVFP4 and 30 BF16 main matrices; token refiner remains BF16.
-- `nvfp4-compact`: 208 NVFP4 main/refiner matrices.
-## Stock ComfyUI layout
-The source checkpoints store Q, K, and V separately. Stock ComfyUI expects a
-single `qkv_proj`, so this release stores exact global `cat(Q,K,V)` tensors.
-The original gated MLP order is also converted from Diffusers
-`[value, gate]` to the ComfyUI `[gate, value]` order.
-The original model contains a wide FP32 time MLP followed by 51 large AdaLN
-projections. For stock compatibility, the original time path is sampled on
-4,097 points and represented by a rank-16 FP32 table coordinate system. Every
-block and the final layer keeps a separate FP32 AdaLN projection. This removes
-the redundant wide input dimension without merging the blocks.
-### Exact stock-versus-dynamic layer layout
-| Feature | This stock-compatible repository | Patched dynamic `s-QKV` repository |
 |---|---|---|
-| Task variants | FL2VA and Ref2VA | FL2VA and Ref2VA |
-| Transformer blocks | All 50 main blocks retained | All 50 main blocks retained |
-| Attention storage | One exact fused `qkv_proj = cat(Q,K,V)` tensor per attention module | Physical `q_proj`, `k_proj`, and `v_proj` tensors |
-| Attention execution | One fused projection call | Three projection calls |
-| Original `time_embedder` | Absent | Present in FP32 |
-| `time_embedder.proj_in` | Absent | Weight `[5376,256]`, bias `[5376]` |
-| `time_embedder.proj_out` | Absent | Weight `[2688,5376]`, bias `[2688]` |
 | `adaln_t_table` | FP32 `[4097,16]` | Absent |
 | `adaln_curve_basis` | Absent | FP32 `[2688,16]` |
 | `adaln_curve_mean` | Absent | FP32 `[2688]` |
 | Per-block AdaLN | 51 independent FP32 rank-16 projections | 51 independent FP32 rank-16 projections |
-| Time evaluation | Interpolation in the 4,097-point table | Original runtime FP32 time MLP, then projection into rank-16 coordinates |
-| ComfyUI requirement | Unmodified recent ComfyUI | Included ComfyUI core patch is required |
-The table path does not remove timestep conditioning: every denoising step
-still receives its own continuous time coordinates. It replaces the wide
-runtime MLP with a compact numerical representation of the same measured time
-curve. The 51 block-specific projections remain independent in both editions.
-Measured maximum interpolation error of the shared time curve is below
-`0.001%`; the sampled end-to-end AdaLN relative error is approximately
-`3e-7` to `4e-7` across 19 timesteps.
-Users who specifically want the original runtime FP32 time MLP and physically
-separate Q/K/V modules should use the
-[patch-required dynamic quant repository](https://huggingface.co/DmitryDB/MiniMax-H3-DynTime-sQKV).
 ## Validation
-Each released checkpoint must pass all of the following before publication:
-1. exact tensor-key, shape, dtype, and quantization-inventory checks;
-2. sampled reconstruction against the original HF shards;
-3. 19-timestep AdaLN numerical comparison;
-4. complete CPU load in a clean stock ComfyUI checkout at commit `14b05228`
-   as `MiniMaxH3Model`;
-5. remote file-size and LFS SHA256 verification after upload.
-The release reports are stored under `reports/release_matrix/`. BF16 samples
-are checked bit-for-bit. A representative INT8 QKV sample has relative L2
-reconstruction error `0.008814`.
-The 24 GiB INT8 profiles additionally passed the RTX 4090 load and resident
-INT8-linear smoke tests described above. A complete prompt-to-decoded-video
-A/B evaluation has not yet been run, so this repository does not claim a
-measured perceptual score.
-## Installation
-Place one selected checkpoint in your ComfyUI diffusion-model directory, for
-example:
 ```text
 ComfyUI/models/diffusion_models/
 ```
-Use the matching FL2VA or Ref2VA workflow. A complete MiniMax-H3 workflow also
-needs a compatible Qwen3-VL MiniMax-H3 text encoder and the MiniMax-H3 video
-and audio VAEs. The text encoder is intentionally maintained separately and is
-not part of this quant matrix.
-## Auxiliary files
-The repository also contains the shared stock MiniMax-H3 VAE components:
 | File | Role |
 |---|---|
 | `vae/minimax_h3_video_vae_fp16.safetensors` | Video latent encode/decode |
 | `vae/minimax_h3_audio_vae_fp32.safetensors` | Audio latent encode/decode |
-The same VAE files are used by FL2VA and Ref2VA and are not quant variants.
-No text encoder is included; the MiniMax-H3 Qwen3-VL text encoder is maintained
-in a separate repository.
-## Naming
-Technical recipe markers were intentionally removed from checkpoint filenames.
-All stock models use the same fused-QKV, rank-16, 4,097-point time-table
-conversion described on this page. Profile names now communicate only the
-weight format and practical memory tier.
 ## License and attribution
 Use is subject to the included MiniMax-H3 community license. The base model is
-by MiniMax. ComfyUI and ComfyUI-compatible quantization runtimes are separate
-upstream projects. This conversion is a community release and is not endorsed
-by MiniMax or ComfyOrg.

 Community quantized diffusion-transformer checkpoints for
 [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3).
+This repository provides the same eight precision profiles for both **FL2VA**
+and **Ref2VA**. All files use the stock ComfyUI fused-QKV and time-table layout:
+no custom node or core patch is required.
+These are community conversions, not official MiniMax or ComfyOrg releases.
+## 1. Choose FL2VA or Ref2VA
+- **FL2VA** — text-to-audio-video, optionally conditioned by a first frame,
+  last frame, or both.
+- **Ref2VA** — reference-to-audio-video using reference images, video, and/or
+  audio.
+Download one diffusion checkpoint from the matching column below.
+## 2. Choose a quant
+| Profile | Direct downloads | Size | What it contains and when to use it |
+|---|---|---:|---|
+| **INT8 ConvRot balanced** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-int8-convrot-balanced.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-int8-convrot-balanced.safetensors?download=true) | 20.940 GiB | **Recommended for RTX 3090/4090 24 GiB.** 170 INT8 + 38 BF16 semantic matrices. Best tested quality/VRAM balance and fully resident in the RTX 4090 loader test. |
+| **INT8 ConvRot safe** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-int8-convrot-safe.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-int8-convrot-safe.safetensors?download=true) | 20.330 GiB | 185 INT8 + 23 BF16 semantic matrices. Choose this on a 24 GiB RTX 30/40 card when the rest of the workflow needs more VRAM. |
+| **INT8 ConvRot max** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-int8-convrot-max.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-int8-convrot-max.safetensors?download=true) | 21.908 GiB | 145 INT8 + 63 BF16 semantic matrices. Largest BF16 quality island; recommended for 32 GiB or more. About 0.955 GiB was offloaded in the 24 GiB RTX 4090 loader test. |
+| **W8/W4 ConvRot balanced** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-w8w4-convrot-balanced.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-w8w4-convrot-balanced.safetensors?download=true) | 13.565 GiB | 86 W8 + 114 W4 main matrices; BF16 token refiner. Starting point for 16 GiB RTX 30/40 cards. |
+| **W4 ConvRot compact** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-w4-convrot-compact.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-w4-convrot-compact.safetensors?download=true) | 10.067 GiB | 200 W4 main matrices + 8 INT8 token-refiner matrices. Recommended starting point for 12 GiB cards. |
+| **W4 ConvRot offload** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-w4-convrot-offload.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-w4-convrot-offload.safetensors?download=true) | 9.708 GiB | All 208 main/refiner matrices use W4. Smallest portable profile; intended for 8 GiB cards with CPU offload. |
+| **NVFP4 quality** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-nvfp4-quality.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-nvfp4-quality.safetensors?download=true) | 13.597 GiB | 170 NVFP4 + 30 BF16 main matrices; BF16 token refiner. Recommended for RTX 50/Blackwell 16–24 GiB when NVFP4 support is available. |
+| **NVFP4 compact** | [FL2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/FL2VA/minimax-h3-fl2va-nvfp4-compact.safetensors?download=true) · [Ref2VA](https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants/resolve/main/Ref2VA/minimax-h3-ref2va-nvfp4-compact.safetensors?download=true) | 10.862 GiB | All 208 main/refiner matrices use block-scaled NVFP4. Smallest Blackwell-specific profile for 8–12 GiB cards. |
+### Short answer
+- **RTX 4090 24 GiB:** start with `int8-convrot-balanced`; use `safe` if the
+  workflow needs more activation memory.
+- **RTX 5090 / Blackwell 16–24 GiB:** start with `nvfp4-quality` for headroom,
+  or INT8 balanced when portability matters.
+- **RTX 30/40 16 GiB:** start with `w8w4-convrot-balanced`.
+- **12 GiB:** start with `w4-convrot-compact`.
+- **8 GiB:** use `w4-convrot-offload` and CPU offload.
+- **32 GiB or more:** `int8-convrot-max` has the largest BF16 island.
+Checkpoint size is not full-workflow peak VRAM. Resolution, frame count,
+attention backend, text encoder, VAE, and ComfyUI offload settings also matter.
+RTX 50 recommendations are architecture-based; no RTX 5090 generation run was
+performed on this machine. NVFP4 here is plain block-scaled NVFP4, not AWQ.
+## Measured RTX 4090 loader results
+FL2VA and Ref2VA were tested independently through stock ComfyUI. Each test
+also executed a real quantized INT8 projection.
+| Profile | Loaded weights | Peak reserved | Free after load | Result |
 |---|---:|---:|---:|---|
 | `int8-convrot-safe` | 100% | 20.424 GiB | 2.072 GiB | PASS |
 | `int8-convrot-balanced` | 100% | 21.025 GiB | 1.471 GiB | PASS |
+| `int8-convrot-max` | 95.6% | 21.002 GiB | 1.494 GiB | PASS; about 0.955 GiB offloaded |
+These are loader/kernel measurements, not full prompt-to-decoded-video peaks.
+## Compatibility
+All 16 files in this repository:
+- retain all 50 transformer blocks;
+- use fused `qkv_proj = cat(Q,K,V)` tensors expected by stock ComfyUI;
+- use a rank-16 FP32, 4,097-point time table;
+- retain 51 independent FP32 AdaLN projections;
+- load without a custom loader or core patch in the tested ComfyUI revision.
+For the original runtime FP32 time MLP and physically separate Q/K/V modules,
+use the patch-required
+[MiniMax-H3-DynTime-sQKV](https://huggingface.co/DmitryDB/MiniMax-H3-DynTime-sQKV)
+repository instead.
+<details>
+<summary><strong>Advanced: exact INT8 BF16 islands and time/QKV layout</strong></summary>
+INT8 weights use ConvRot/Hadamard rotation with group size 256, per-row FP32
+scales, and deterministic scale search. Norms, conditioning projections, patch
+projections, output heads, and other small or sensitive tensors retain their
+source precision.
+| Profile | BF16 attention-output blocks | BF16 MLP `fc2` blocks | Other main semantic matrices |
 |---|---|---|---|
 | `int8-convrot-safe` | 0, 1, 2, 3, 4, 5, 6, 7, 9, 15, 19, 38, 45, 49 | 49 | INT8 ConvRot |
 | `int8-convrot-balanced` | 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 27, 38, 43, 44, 45, 46, 47, 49 | 39, 45, 49 | INT8 ConvRot |
 | `int8-convrot-max` | all blocks 0–49 | 29, 39, 44, 45, 49 | INT8 ConvRot |
+The eight token-refiner semantic matrices remain BF16 in all three INT8
+profiles.
+| Feature | This stock repository | Patched DynTime `s-QKV` repository |
 |---|---|---|
+| Attention | One fused projection call | Separate Q, K, and V calls |
+| Original FP32 `time_embedder` | Absent | Present |
 | `adaln_t_table` | FP32 `[4097,16]` | Absent |
 | `adaln_curve_basis` | Absent | FP32 `[2688,16]` |
 | `adaln_curve_mean` | Absent | FP32 `[2688]` |
 | Per-block AdaLN | 51 independent FP32 rank-16 projections | 51 independent FP32 rank-16 projections |
+| ComfyUI | Stock | Core patch required |
+The time table does not remove timestep conditioning. It interpolates a compact
+representation of the original measured time curve. Maximum measured table
+interpolation error is below `0.001%`; sampled end-to-end AdaLN relative error
+is approximately `3e-7` to `4e-7` across 19 timesteps.
+</details>
 ## Validation
+Every released checkpoint passed:
+1. exact key, shape, dtype, and quantization-inventory checks;
+2. sampled reconstruction against its original FL2VA or Ref2VA HF shards;
+3. a 19-timestep FP32 AdaLN numerical comparison;
+4. complete CPU load as `MiniMaxH3Model` in clean ComfyUI commit `14b05228`;
+5. remote byte-size and LFS SHA-256 verification.
+Reports are stored under `reports/release_matrix/`. BF16 samples are checked
+bit-for-bit. A representative INT8 QKV sample has relative L2 error `0.008814`.
+A prompt-to-decoded-video perceptual A/B score has not yet been measured.
+## Installation and required components
+Place one selected FL2VA or Ref2VA checkpoint in:
 ```text
 ComfyUI/models/diffusion_models/
 ```
+A complete workflow also needs the separately maintained Qwen3-VL MiniMax-H3
+text encoder and these shared VAE files:
 | File | Role |
 |---|---|
 | `vae/minimax_h3_video_vae_fp16.safetensors` | Video latent encode/decode |
 | `vae/minimax_h3_audio_vae_fp32.safetensors` | Audio latent encode/decode |
+No text encoder is included in this repository.
 ## License and attribution
 Use is subject to the included MiniMax-H3 community license. The base model is
+by MiniMax. This community conversion is not endorsed by MiniMax or ComfyOrg.