Update README.md for short checkpoint names

README.md

@@ -10,158 +10,228 @@ tags:
   - comfyui
   - quantization
   - int8
-  - convrot
+  - w4
+  - nvfp4
   - video
   - audio
   - fl2va
   - ref2va
 ---
 
-# MiniMax-H3 INT8 Lean ConvRot — Stock ComfyUI Edition
+# MiniMax-H3 Quants for ComfyUI
 
-This repository contains stock-compatible, quality-oriented mixed-precision
-INT8 ConvRot conversions of
-[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3), plus the
-standard MiniMax-H3 visual and audio VAEs. FL2VA and Ref2VA are separate because
-their diffusion-transformer weights are task-specific.
+Community quantized diffusion-transformer checkpoints for
+[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3).
+The repository contains separate **FL2VA** and **Ref2VA** models and a range of
+INT8, W8/W4, W4, and NVFP4 precision profiles.
 
-These are community derivatives, not official MiniMax or ComfyOrg checkpoints.
-The diffusion checkpoints use the standard MiniMax-H3 `adaln_t_table` layout
-and load in unmodified ComfyUI. No core patch or custom node is required.
+These checkpoints are not official MiniMax or ComfyOrg releases. The models in
+this repository use the standard MiniMax-H3 time-table and fused-QKV layout and
+load in an unmodified recent ComfyUI. No core patch or custom model loader is
+required.
 
-The experimental edition with the original FP32 time MLP and physically
-separate Q/K/V projections has moved to
-[`DmitryDB/MiniMax-H3-INT8-Lean-ConvRot-Dynamic-Time-Separate-QKV`](https://huggingface.co/DmitryDB/MiniMax-H3-INT8-Lean-ConvRot-Dynamic-Time-Separate-QKV).
-It requires a ComfyUI core patch and is intentionally kept separate to avoid
-confusion.
+The filenames are intentionally short. Full implementation details, layer
+counts, compatibility notes, and measured VRAM behavior are documented below.
 
-## Included files
+## Choose a model
 
-| Path | Purpose |
-|---|---|
-| `FL2VA/minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors` | Stock-compatible FL2VA diffusion transformer, 20.940 GiB |
-| `Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors` | Stock-compatible Ref2VA diffusion transformer, 20.940 GiB |
-| `vae/minimax_h3_video_vae_fp16.safetensors` | Stock ComfyUI MiniMax-H3 visual VAE, FP16, 4.850 GiB |
-| `vae/minimax_h3_audio_vae_fp32.safetensors` | Stock ComfyUI MiniMax-H3 stereo audio VAE, FP32, 0.564 GiB |
-| `reports/validation.json` / `validation_ref2va.json` | Structural, table-error, and clean-Comfy CPU-load validation |
-| `reports/layer_policy.json` / `layer_policy_ref2va.json` | Exact BF16/INT8 layer policy |
-| `reports/mm_quant_profile_fl2va.json` / `mm_quant_profile_ref2va.json` | Row-sampled reconstruction profiles |
+Download exactly one diffusion model for the task you use:
 
-The MiniMax-H3 Qwen3-VL text encoder is not included in this repository. A
-standard workflow needs one selected diffusion checkpoint, both VAEs, and a
-compatible MiniMax-H3 text encoder installed separately.
+- **FL2VA** — text-to-audio-video and optional first frame, last frame, or both;
+- **Ref2VA** — reference-to-audio-video using reference images, videos, and/or audio.
 
-## Quantization policy
+The same profile names are provided under both `FL2VA/` and `Ref2VA/`.
 
-The original FL2VA and Ref2VA transformers each contain 50 main transformer
-blocks. Each block has four attention projections and two MLP matrices, for 300
-main matrices. The quality21 policy keeps the high-risk matrices in their
-source precision and compresses the rest:
+| Canonical filename | Quant format and intent | File size |
+|---|---|---:|
+| `minimax-h3-{variant}-int8-convrot-safe.safetensors` | INT8 ConvRot with the largest VRAM margin | 20.330 GiB |
+| `minimax-h3-{variant}-int8-convrot-balanced.safetensors` | Recommended INT8 quality/size balance | 20.940 GiB |
+| `minimax-h3-{variant}-int8-convrot-max.safetensors` | INT8 ConvRot with the largest BF16 quality island | 21.908 GiB |
+| `minimax-h3-{variant}-w8w4-convrot-balanced.safetensors` | Mixed W8/W4 ConvRot | 13.565 GiB |
+| `minimax-h3-{variant}-w4-convrot-compact.safetensors` | W4 ConvRot main transformer with INT8 refiner | 10.067 GiB |
+| `minimax-h3-{variant}-w4-convrot-offload.safetensors` | Smallest universal W4 ConvRot checkpoint | 9.708 GiB |
+| `minimax-h3-{variant}-nvfp4-quality.safetensors` | NVFP4 with a BF16 quality island | 13.597 GiB |
+| `minimax-h3-{variant}-nvfp4-compact.safetensors` | Compact all-edge NVFP4 | 10.862 GiB |
 
-- 170 main matrices use INT8 ConvRot with Hadamard group size 256;
-- 30 high-risk main matrices remain BF16;
-- token-refiner attention/MLP weights remain BF16;
-- normalization tensors remain BF16;
-- patch projections and video/audio output heads retain source FP32;
-- 51 AdaLN projections are represented through separate FP32 rank-16 table
-  projections.
+Replace `{variant}` with `fl2va` or `ref2va`. Resolution, frame count,
+attention backend, text encoder, VAE, and ComfyUI offload policy all affect
+peak VRAM; checkpoint size is not the same as full-workflow VRAM.
 
-Q, K, and V originate as separate Diffusers tensors. For stock ComfyUI they are
-packed as global `cat(Q,K,V)` into `qkv_proj`, which is the layout expected by
-the unmodified runtime.
+## GPU recommendations and why
 
-The BF16 islands were selected by deterministic row-sampled reconstruction
-analysis rather than by a blanket first/last-block rule. FL2VA and Ref2VA were
-profiled independently and selected the same 30 matrices.
+| GPU generation | VRAM | Recommended starting profile | Why |
+|---|---:|---|---|
+| RTX 30 or RTX 40 | 8 GiB | `w4-convrot-offload` | Smallest portable checkpoint. It still requires CPU offload, but reduces repeated PCIe weight traffic compared with a 20+ GiB INT8 model. |
+| RTX 30 or RTX 40 | 12 GiB | `w4-convrot-compact` | Leaves more room for activations than mixed W8/W4 while retaining an INT8 token refiner. |
+| RTX 30 or RTX 40 | 16 GiB | `w8w4-convrot-balanced` | Uses W8 on the more quality-sensitive subset and W4 elsewhere; a better quality/speed compromise when 20 GiB INT8 cannot remain resident. |
+| RTX 30 or RTX 40 | 24 GiB | `int8-convrot-balanced` | Portable fast path with the full model resident on the tested RTX 4090. It retains the balanced BF16 island without the offload seen in `max`. |
+| RTX 30 or RTX 40 | 24 GiB, activation-heavy workflow | `int8-convrot-safe` | Also fully resident on the tested RTX 4090 and leaves about 0.6 GiB more free VRAM than `balanced`. |
+| RTX 30 or RTX 40 | 32 GiB or more | `int8-convrot-max` | Largest BF16 island and enough memory to avoid the small offload measured on a 24 GiB 4090. |
+| RTX 50 / Blackwell | 8–12 GiB | `nvfp4-compact` | Blackwell-native block-scaled NVFP4 gives the smallest architecture-specific checkpoint. Use W4 ConvRot as the portable fallback. |
+| RTX 50 / Blackwell | 16–24 GiB | `nvfp4-quality` | Keeps the balanced BF16 quality island while using Blackwell-native NVFP4 for the other main weights, leaving much more activation headroom. |
+| RTX 50 / Blackwell | 24 GiB, portability preferred | `int8-convrot-balanced` | Same file works across RTX 30/40/50 and avoids dependence on the NVFP4 runtime path. |
+| RTX 50 / Blackwell | 32 GiB or more | `int8-convrot-max` for quality, `nvfp4-quality` for speed/headroom | The larger card removes the reason to offload `max`; NVFP4 remains attractive when throughput and free VRAM matter more. |
 
-## Time conditioning
+RTX 4090 recommendations are backed by the measurements below. RTX 50 NVFP4
+recommendations are architecture-based; this machine has no local RTX 5090,
+so they are not presented as measured 5090 generation results. NVFP4 is plain
+block-scaled NVFP4, not activation-aware AWQ.
 
-The raw model uses a wide FP32 time MLP followed by 51 large AdaLN projections.
-The release samples the original FP32 time path on 4,097 points, derives a
-shared 16-dimensional basis, and stores one independent FP32 projection for
-every transformer block plus the final layer.
+## Measured RTX 4090 loading
 
-The saved `adaln_t_table` has shape `[4097, 16]`. ComfyUI linearly interpolates
-the 16 coordinates and applies the per-layer projection. On 19 validation
-timesteps, relative AdaLN output error against the raw HF dynamic path was:
+FL2VA and Ref2VA were tested independently on a 24 GiB RTX 4090 through the
+stock ComfyUI loader. A real resident INT8 projection was executed after load.
 
-- FL2VA: about `3.04e-7` to `3.82e-7`;
-- Ref2VA: about `3.42e-7` to `4.46e-7`.
+| Profile | Loaded weight fraction | Peak reserved VRAM | Free after load | Result |
+|---|---:|---:|---:|---|
+| `int8-convrot-safe` | 100% | 20.424 GiB | 2.072 GiB | PASS |
+| `int8-convrot-balanced` | 100% | 21.025 GiB | 1.471 GiB | PASS |
+| `int8-convrot-max` | 95.6% | 21.002 GiB | 1.494 GiB | PASS with about 0.955 GiB of weights offloaded |
 
-This is substantially more accurate than using a direct wide cached table at
-the same grid density while remaining compatible with stock ComfyUI.
+These are loader and quantized-kernel measurements, not full video-generation
+peak figures.
 
-## Comparison with official ComfyOrg files
+## What was quantized
 
-| Checkpoint | Size | Main matrices | Time/AdaLN representation |
-|---|---:|---:|---|
-| Official full INT8 ConvRot | 27.557 GiB | 300 INT8 | Wide FP32 AdaLN table |
-| Official pruned INT8 ConvRot | 19.530 GiB | 200 INT8 | Wide FP32 AdaLN table |
-| This stock quality21 edition | 20.940 GiB | 170 INT8 + 30 BF16 | FP32 `[4097,16]` table + 51 projections |
+The raw FL2VA and Ref2VA Diffusers transformers each contain 50 main
+transformer blocks. The conversion preserves every block and uses a
+layer-sensitive policy instead of uniformly quantizing all matrices.
 
-Each file is about 1.410 GiB larger than its official pruned INT8 counterpart.
-The additional space is intentionally spent on the rank-16 time representation
-and BF16 islands. The sampled weight-reconstruction proxy improved from about
-`0.9345%` mean relative error for official-style absmax INT8 to about `0.7639%`
-for this mixed profile. This is a proxy, not an end-to-end perceptual result.
+### INT8 profiles
+
+| Profile | INT8 semantic matrices | BF16 semantic matrices | Notes |
+|---|---:|---:|---|
+| `int8-convrot-safe` | 185 | 23 | Maximum VRAM headroom of the three INT8 profiles |
+| `int8-convrot-balanced` | 170 | 38 | 30 BF16 main matrices plus the BF16 token refiner |
+| `int8-convrot-max` | 145 | 63 | Largest BF16 quality island |
+
+INT8 weights use ConvRot/Hadamard rotation with group size 256, per-row FP32
+scales, and deterministic scale search. The BF16 islands were selected from
+independent FL2VA and Ref2VA sampled-reconstruction measurements. Norms,
+conditioning projections, patch projections, output heads, and other small or
+sensitive tensors retain their appropriate source precision.
+
+The exact main-transformer BF16 islands are:
+
+| Profile | BF16 attention output blocks | BF16 MLP `fc2` blocks | Everything else among the 200 main semantic matrices |
+|---|---|---|---|
+| `int8-convrot-safe` | 0, 1, 2, 3, 4, 5, 6, 7, 9, 15, 19, 38, 45, 49 | 49 | INT8 ConvRot |
+| `int8-convrot-balanced` | 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 27, 38, 43, 44, 45, 46, 47, 49 | 39, 45, 49 | INT8 ConvRot |
+| `int8-convrot-max` | all blocks 0–49 | 29, 39, 44, 45, 49 | INT8 ConvRot |
+
+For all three profiles the eight token-refiner semantic matrices remain BF16.
+The machine-readable validation reports provide the saved-tensor inventory for
+each FL2VA and Ref2VA file.
+
+### W4 and NVFP4 profiles
+
+- `w8w4-convrot-balanced`: 86 W8 and 114 W4 main matrices; token refiner remains BF16.
+- `w4-convrot-compact`: 200 W4 main matrices and 8 INT8 token-refiner matrices.
+- `w4-convrot-offload`: 208 W4 main/refiner matrices.
+- `nvfp4-quality`: 170 NVFP4 and 30 BF16 main matrices; token refiner remains BF16.
+- `nvfp4-compact`: 208 NVFP4 main/refiner matrices.
+
+## Stock ComfyUI layout
+
+The source checkpoints store Q, K, and V separately. Stock ComfyUI expects a
+single `qkv_proj`, so this release stores exact global `cat(Q,K,V)` tensors.
+The original gated MLP order is also converted from Diffusers
+`[value, gate]` to the ComfyUI `[gate, value]` order.
+
+The original model contains a wide FP32 time MLP followed by 51 large AdaLN
+projections. For stock compatibility, the original time path is sampled on
+4,097 points and represented by a rank-16 FP32 table coordinate system. Every
+block and the final layer keeps a separate FP32 AdaLN projection. This removes
+the redundant wide input dimension without merging the blocks.
+
+### Exact stock-versus-dynamic layer layout
+
+| Feature | This stock-compatible repository | Patched dynamic `s-QKV` repository |
+|---|---|---|
+| Task variants | FL2VA and Ref2VA | FL2VA and Ref2VA |
+| Transformer blocks | All 50 main blocks retained | All 50 main blocks retained |
+| Attention storage | One exact fused `qkv_proj = cat(Q,K,V)` tensor per attention module | Physical `q_proj`, `k_proj`, and `v_proj` tensors |
+| Attention execution | One fused projection call | Three projection calls |
+| Original `time_embedder` | Absent | Present in FP32 |
+| `time_embedder.proj_in` | Absent | Weight `[5376,256]`, bias `[5376]` |
+| `time_embedder.proj_out` | Absent | Weight `[2688,5376]`, bias `[2688]` |
+| `adaln_t_table` | FP32 `[4097,16]` | Absent |
+| `adaln_curve_basis` | Absent | FP32 `[2688,16]` |
+| `adaln_curve_mean` | Absent | FP32 `[2688]` |
+| Per-block AdaLN | 51 independent FP32 rank-16 projections | 51 independent FP32 rank-16 projections |
+| Time evaluation | Interpolation in the 4,097-point table | Original runtime FP32 time MLP, then projection into rank-16 coordinates |
+| ComfyUI requirement | Unmodified recent ComfyUI | Included ComfyUI core patch is required |
+
+The table path does not remove timestep conditioning: every denoising step
+still receives its own continuous time coordinates. It replaces the wide
+runtime MLP with a compact numerical representation of the same measured time
+curve. The 51 block-specific projections remain independent in both editions.
+
+Measured maximum interpolation error of the shared time curve is below
+`0.001%`; the sampled end-to-end AdaLN relative error is approximately
+`3e-7` to `4e-7` across 19 timesteps.
+
+Users who specifically want the original runtime FP32 time MLP and physically
+separate Q/K/V modules should use the
+[patch-required dynamic quant repository](https://huggingface.co/DmitryDB/MiniMax-H3-INT8-Lean-ConvRot-Dynamic-Time-Separate-QKV).
+
+## Validation
+
+Each released checkpoint must pass all of the following before publication:
+
+1. exact tensor-key, shape, dtype, and quantization-inventory checks;
+2. sampled reconstruction against the original HF shards;
+3. 19-timestep AdaLN numerical comparison;
+4. complete CPU load in a clean stock ComfyUI checkout at commit `14b05228`
+   as `MiniMaxH3Model`;
+5. remote file-size and LFS SHA256 verification after upload.
+
+The release reports are stored under `reports/release_matrix/`. BF16 samples
+are checked bit-for-bit. A representative INT8 QKV sample has relative L2
+reconstruction error `0.008814`.
+
+The 24 GiB INT8 profiles additionally passed the RTX 4090 load and resident
+INT8-linear smoke tests described above. A complete prompt-to-decoded-video
+A/B evaluation has not yet been run, so this repository does not claim a
+measured perceptual score.
 
 ## Installation
 
-Place one or both diffusion checkpoints in:
+Place one selected checkpoint in your ComfyUI diffusion-model directory, for
+example:
 
 ```text
 ComfyUI/models/diffusion_models/
 ```
 
-Place both VAE files in:
-
-```text
-ComfyUI/models/vae/
-```
-
-Install a compatible MiniMax-H3 Qwen3-VL text encoder separately, then use the
-matching stock MiniMax-H3 FL2VA or Ref2VA workflow.
-
-## Validation performed
+Use the matching FL2VA or Ref2VA workflow. A complete MiniMax-H3 workflow also
+needs a compatible Qwen3-VL MiniMax-H3 text encoder and the MiniMax-H3 video
+and audio VAEs. The text encoder is intentionally maintained separately and is
+not part of this quant matrix.
 
-- safetensors structure, shapes, and dtypes;
-- 170/170 INT8 ConvRot blobs with group size 256;
-- all 30 BF16 main matrices present according to the published policy;
-- non-quantized source tensors compared against the raw MiniMax-H3 shards;
-- FP32 rank-16 time tables and 51 projections checked numerically;
-- full CPU load through clean, unmodified ComfyUI commit `14b05228` as
-  `ModelPatcher -> MiniMaxH3Model`, table mode enabled and no dynamic time MLP;
-- both VAE files fully loaded through the same clean ComfyUI;
-- VAE checksums match the standard ComfyOrg repackaged files.
+## Auxiliary files
 
-SHA-256:
+The repository also contains the shared stock MiniMax-H3 VAE components:
 
-```text
-ad0cf8b50ba92068b5a4477527d5da73412a22db99cded35b07195504e78cf6a  FL2VA/minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors
-f4b2a8b4a53446f5d696a39deb693b09ac7ce37013793a8b19340644e38b154f  Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors
-7c1f131492e7eddacaac9069a61b81bdd39de5cc96561e677c5eab1cdce5e522  vae/minimax_h3_video_vae_fp16.safetensors
-8e505d95dd1561d47abd43d4238fd40d9bb1ae9e147ed0a4cba778d76ae4db48  vae/minimax_h3_audio_vae_fp32.safetensors
-```
-
-## Validation not yet completed
-
-- GPU loading and peak-VRAM measurement;
-- fixed-seed end-to-end FL2VA/Ref2VA generation;
-- video and audio A/B quality comparison against BF16 and official INT8.
+| File | Role |
+|---|---|
+| `vae/minimax_h3_video_vae_fp16.safetensors` | Video latent encode/decode |
+| `vae/minimax_h3_audio_vae_fp32.safetensors` | Audio latent encode/decode |
 
-The 20.940 GiB file size is a disk-weight target, not a guarantee that every
-24 GiB workflow will fit. Runtime memory also depends on activation size,
-resolution, frame count, attention implementation, and offload settings.
+The same VAE files are used by FL2VA and Ref2VA and are not quant variants.
+No text encoder is included; the MiniMax-H3 Qwen3-VL text encoder is maintained
+in a separate repository.
 
-## License
+## Naming
 
-MiniMax H3 is distributed under the MiniMax H3 Community License Agreement.
-Read `LICENSE` before using or redistributing these derivatives. The
-modification summary is provided in `NOTICE`.
+Technical recipe markers were intentionally removed from checkpoint filenames.
+All stock models use the same fused-QKV, rank-16, 4,097-point time-table
+conversion described on this page. Profile names now communicate only the
+weight format and practical memory tier.
 
-## Links
+## License and attribution
 
-- [Original MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3)
-- [Experimental dynamic-time / separate-QKV edition](https://huggingface.co/DmitryDB/MiniMax-H3-INT8-Lean-ConvRot-Dynamic-Time-Separate-QKV)
-- [Official ComfyOrg MiniMax-H3 files](https://huggingface.co/Comfy-Org/MiniMax-H3)
-- [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
+Use is subject to the included MiniMax-H3 community license. The base model is
+by MiniMax. ComfyUI and ComfyUI-compatible quantization runtimes are separate
+upstream projects. This conversion is a community release and is not endorsed
+by MiniMax or ComfyOrg.