--- license: other license_name: minimax-h3-community-license-agreement license_link: LICENSE base_model: MiniMaxAI/MiniMax-H3 library_name: comfyui pipeline_tag: image-text-to-video tags: - minimax-h3 - comfyui - quantization - int8 - convrot - video - audio - fl2va - ref2va --- # MiniMax-H3 INT8 Lean ConvRot This repository contains a quality-oriented mixed-precision ComfyUI derivative of [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3). The released checkpoints cover both the **FL2VA** and **Ref2VA** diffusion transformers. Each is designed as a first 24 GiB GPU candidate while preserving more of the original model than the corresponding official pruned INT8 checkpoint. Two layouts are provided: a stock-compatible dense time-table edition and a patched edition with the original FP32 time MLP and physically separate Q, K, and V modules. These are not official MiniMax or ComfyOrg checkpoints. The `table-*` files use the standard MiniMax-H3 `adaln_t_table` format and load in an unmodified ComfyUI. The `dynamic-*-separate-qkv-*` files require the small core patch included in [`patches/`](patches/). ## Included files | Path | Purpose | |---|---| | [`FL2VA/minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors`](FL2VA/minimax-h3-fl2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors) | Stock-compatible FL2VA diffusion transformer, 20.940 GiB | | [`Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors`](Ref2VA/minimax-h3-ref2va-int8-lean-convrot-table-k16-g4097-quality21.safetensors) | Stock-compatible Ref2VA diffusion transformer, 20.940 GiB | | [`FL2VA/minimax-h3-fl2va-int8-lean-convrot-dynamic-k16-separate-qkv-quality21.safetensors`](FL2VA/minimax-h3-fl2va-int8-lean-convrot-dynamic-k16-separate-qkv-quality21.safetensors) | Patched FL2VA: original FP32 time MLP and separate Q/K/V, 20.999 GiB | | [`Ref2VA/minimax-h3-ref2va-int8-lean-convrot-dynamic-k16-separate-qkv-quality21.safetensors`](Ref2VA/minimax-h3-ref2va-int8-lean-convrot-dynamic-k16-separate-qkv-quality21.safetensors) | Patched Ref2VA: original FP32 time MLP and separate Q/K/V, 20.999 GiB | | [`vae/minimax_h3_video_vae_fp16.safetensors`](vae/minimax_h3_video_vae_fp16.safetensors) | Stock ComfyUI MiniMax-H3 visual VAE, FP16, 4.850 GiB | | [`vae/minimax_h3_audio_vae_fp32.safetensors`](vae/minimax_h3_audio_vae_fp32.safetensors) | Stock ComfyUI MiniMax-H3 stereo audio VAE, FP32, 0.564 GiB | | [`patches/comfyui-minimax-h3-dynamic-time-separate-qkv.patch`](patches/comfyui-minimax-h3-dynamic-time-separate-qkv.patch) | Required ComfyUI core patch for the two dynamic/separate files | | [`reports/layer_policy.json`](reports/layer_policy.json) / [`layer_policy_ref2va.json`](reports/layer_policy_ref2va.json) | Exact per-layer precision policies | | [`reports/validation.json`](reports/validation.json) / [`validation_ref2va.json`](reports/validation_ref2va.json) | Structural, numerical, and CPU-load results | | [`reports/validation_dynamic_separate_fl2va.json`](reports/validation_dynamic_separate_fl2va.json) / [`validation_dynamic_separate_ref2va.json`](reports/validation_dynamic_separate_ref2va.json) | Bitwise split and dynamic-time validation for the patched files | | [`reports/loadtest_dynamic_separate_fl2va.json`](reports/loadtest_dynamic_separate_fl2va.json) / [`loadtest_dynamic_separate_ref2va.json`](reports/loadtest_dynamic_separate_ref2va.json) | Full patched-Comfy CPU loader results | | [`reports/mm_quant_profile_fl2va.json`](reports/mm_quant_profile_fl2va.json) | Row-sampled reconstruction profile for all 200 main matrices | | [`reports/mm_quant_profile_ref2va.json`](reports/mm_quant_profile_ref2va.json) | Matching Ref2VA reconstruction profile | This repository intentionally does **not** include the MiniMax-H3 text encoder. It is being prepared as a separate repository. ComfyUI does not require the Diffusers scheduler, tokenizer, processor, or remote-code Python bundles as separate downloads: a standard workflow needs one selected diffusion model, the two included VAEs, and a compatible MiniMax-H3 text encoder. ## Shared quantization policy The original FL2VA and Ref2VA Diffusers shards each contain 50 transformer blocks with separate Q, K, and V tensors. Both release layouts use the same quality21 precision policy: - swaps the two `fc1` halves from `[value, gate]` to `[gate, value]`; - stores 170 selected main attention/MLP matrices as INT8 ConvRot with group size 256 and per-row FP32 scales; - keeps 30 high-error main matrices as exact BF16 copies; - keeps the two-block token refiner, norms, Q/K norms, and condition projection at their source precision; - keeps patch projections and video/audio output heads in source FP32; - replaces only the redundant input width of the 51 large AdaLN projections, while keeping every block and final AdaLN projection separate in FP32. The stock-compatible files additionally pack attention rows as global `cat(Q,K,V)` and evaluate the original FP32 time embedder offline into a dense FP32 table with 4,097 rows and 16 coordinates. Each stock-compatible checkpoint contains 872 tensor entries: 170 INT8 weights, 170 quantization configuration blobs, 250 BF16 entries, and 282 FP32 entries. ## Patched dynamic-time, separate-QKV edition The two `dynamic-k16-separate-qkv` files retain the original four FP32 time-MLP tensors: ```text 256 -> 5,376 -> 2,688 full_t = SiLU(original_time_embedder(t)) coords = (full_t - mean) @ basis[2,688 x 16] AdaLN_i(t) = separate_linear_i(coords) ``` The 16-dimensional dynamic basis compresses only the shared input subspace of the 51 very wide AdaLN projections. It does not replace the time MLP with an interpolation table, and it does not merge the 51 per-block/final AdaLN layers. Q, K, and V are also separate modules all the way through loading and forward: - 50 main blocks contain independent INT8 ConvRot `q_proj`, `k_proj`, and `v_proj` weights, scales, and descriptors; - both token-refiner blocks contain independent BF16 Q, K, and V projections; - the checkpoint contains no `qkv_proj` key; - the patch performs three projection calls rather than concatenating the tensors at load time. This changes storage structure, not the quantization values. All 156 resulting Q/K/V projection tensors are bit-identical to the corresponding contiguous thirds of the validated fused release. Per-row scales were already independent, so splitting QKV alone is not claimed as a quality improvement. It may be slower than fused QKV because the current ConvRot path can rotate the same input three times and launches three projections. GPU speed and peak VRAM have not yet been measured. Each patched checkpoint contains 1,181 tensor entries and 270 INT8 ConvRot modules. The total quantized QKV payload is unchanged; the approximately 0.059 GiB size increase comes primarily from restoring the original FP32 time MLP. ## Precision policy Every DiT block has four main matrices: QKV, attention output, MLP `fc1`, and MLP `fc2`, for 200 matrices total per checkpoint. The `quality21` profile stores 170 in INT8 and 30 in BF16. The BF16 island is: - attention output projections in blocks 0–15, 17, 19, 20, 27, 38, 43–47, and 49 (27 matrices); - MLP `fc2` in blocks 39, 45, and 49 (3 matrices). These were selected from deterministic reconstruction audits using 64 output rows and every input column of each matrix. Ref2VA independently selected the same 30-layer set as FL2VA, with a per-layer error correlation of `0.999676`. Attention output projections were the most error-prone family and are considerably cheaper to retain in BF16 than QKV or `fc1`. The exact machine-readable lists are in [`reports/layer_policy.json`](reports/layer_policy.json) and [`reports/layer_policy_ref2va.json`](reports/layer_policy_ref2va.json). ## Stock-compatible time conditioning The time embedder maps the current diffusion timestep/noise level to a 2,688-dimensional signal. Every transformer block turns this signal into its own AdaLN shifts, scales, and gates, telling the network which denoising stage is being evaluated. At inference this frozen mapping traces a smooth one-dimensional curve because its input is a single scalar timestep. This release evaluates the original FP32 time MLP offline, projects the curve onto a shared 16-dimensional basis, and stores those coordinates at 4,097 evenly spaced points: ```text u(t) = SiLU(original_time_embedder(t)) table[j] = project(u(j / 4096)) AdaLN_i(t) = linear_i(interpolate(table, t)) ``` Each of the 50 block AdaLN projections and the final AdaLN projection remains a different FP32 linear layer. Only their shared input curve is compressed. On 19 validation timesteps, the relative AdaLN output error against the raw HF weights was `3.04e-7` to `3.82e-7` for FL2VA and `3.42e-7` to `4.46e-7` for Ref2VA across blocks 0, 24, 49, and the final layer. For context, the official pruned rank-8 FP16 tables measured about `0.016%–0.021%` on the same projections. This comparison covers only the time/AdaLN path, not end-to-end generation quality. ## Comparison with official ComfyOrg files | Checkpoint | Size | Main matrices | Time/AdaLN representation | |---|---:|---:|---| | Official regular INT8 ConvRot | about 31.70 GiB | 200 INT8 | full AdaLN weights | | Official pruned INT8 ConvRot | 19.530 GiB | 200 INT8 | FP16 rank-8 table | | Each stock `table-k16 quality21` release | 20.940 GiB | 170 INT8 + 30 BF16 | FP32 rank-16 table, 4,097 rows | | Each patched `dynamic-k16 separate-QKV quality21` release | 20.999 GiB | Same 170 semantic main matrices; 270 physical INT8 modules | Original FP32 time MLP + FP32 dynamic rank-16 basis | Each stock file is about 1.410 GiB (7.22%) larger than its official pruned INT8 counterpart. The patched edition adds about 0.059 GiB over the stock quality21 file by restoring the original FP32 time MLP. The remaining added space is intentionally spent on a higher-precision time representation and a BF16 island for the most sensitive main matrices. The full 200-matrix sampled reconstruction proxies produced: | Variant | Official-style absmax INT8 | This profile | |---|---:|---:| | FL2VA | 0.9345% | 0.7639% | | Ref2VA | 0.9345% | 0.7639% | For these profiles, selected BF16 matrices count as zero storage reconstruction error. On the remaining 170 INT8 matrices, searched scales reduced the mean proxy from about 0.9109% to 0.8987%. The approximately 18.25% overall reduction is a sampled weight reconstruction result, **not** a perceptual or generation benchmark. ## ComfyUI installation 1. Use a current ComfyUI checkout with MiniMax-H3 and native INT8 ConvRot support. The stock table files were CPU-load-tested on a clean, unmodified ComfyUI commit `14b05228cef127ce529bc0c08660770d4af3e9a8`. 2. If using a `dynamic-k16-separate-qkv` file, apply the included patch from the root of that ComfyUI checkout. This is not required for the `table-k16` files: ```powershell git apply --check C:\path\to\comfyui-minimax-h3-dynamic-time-separate-qkv.patch git apply C:\path\to\comfyui-minimax-h3-dynamic-time-separate-qkv.patch ``` The patch was verified with `git apply --check` against commit `14b05228`. It updates MiniMax-H3 model detection/runtime and makes partial-offload bookkeeping tolerate packed INT8 layers without a regular `.weight` field. 3. Place the FL2VA and/or Ref2VA safetensors file in `ComfyUI/models/diffusion_models/`. FL2VA is for text/image first-last-frame workflows; Ref2VA is for reference image/video/audio workflows. 4. Place both files from `vae/` in `ComfyUI/models/vae/`. 5. Load one diffusion checkpoint and both VAEs with the standard loaders in the matching MiniMax-H3 FL2VA or Ref2VA workflow. Supply a compatible text encoder separately. ```text ComfyUI/models/ |-- diffusion_models/ | `-- |-- vae/ | |-- minimax_h3_video_vae_fp16.safetensors | `-- minimax_h3_audio_vae_fp32.safetensors `-- text_encoders/ `-- ``` No custom node is required. Only the two explicitly named dynamic/separate checkpoints require the included core patch. ## Validation status Completed: - safetensors structure, shapes, dtypes, and all 170 ConvRot descriptors; - SHA-256: FL2VA `ad0cf8b50ba92068b5a4477527d5da73412a22db99cded35b07195504e78cf6a`; Ref2VA `f4b2a8b4a53446f5d696a39deb693b09ac7ce37013793a8b19340644e38b154f`; - BF16 main island: 30/30 matrices bitwise equal to raw HF; - FL2VA: 871/871 tensors outside the time representation preserved bit-for-bit during the final stock-table conversion; - Ref2VA: 258/258 direct source-copy tensors (4,146,503,680 bytes) bitwise equal to raw HF; - FP32 table interpolation checked at every one of the 4,096 interval midpoints; FL2VA grid rows reproduced exactly, and the independent Ref2VA FP64-SVD rebuild differed by at most `4.55e-13`; - full CPU load through a clean, unmodified ComfyUI as `ModelPatcher -> MiniMaxH3Model`, table mode `(4097, 16)`, for both diffusion checkpoints; - patched editions: 1,181 keys, 270 INT8 ConvRot descriptors at group size 256, zero fused-QKV keys, and all 156 Q/K/V projection tensors bitwise equal to the corresponding thirds of the stock quality21 checkpoints; - patched editions: all 719 common non-QKV tensors bitwise equal to their stock quality21 counterparts and all four FP32 time-MLP tensors bitwise equal to the selected raw HF variant; - patched FL2VA dynamic coordinates reproduce the released 4,097-row table bit-for-bit; the independently reconstructed Ref2VA dynamic path differs by `2.89e-7` relative (`1.31e-6` maximum absolute coordinate error); - patched AdaLN output relative errors on 19 timesteps are `2.85e-7` to `3.32e-7` for FL2VA and `2.96e-7` to `3.31e-7` for Ref2VA across blocks 0, 24, 49, and the final layer; - both patched files fully load on CPU through the patched ComfyUI as `MiniMaxH3Model`, with dynamic basis enabled, table mode disabled, and independent Q/K/V modules in both the main stack and token refiner; - patched SHA-256: FL2VA `495b9d565b17194196646ef39b0c04ac7dde4a8df695b103b3fcbe04b5da00a6`; Ref2VA `82d16de12bee9d006c83c80b8843f2bb7e05cd25320724113ca7d128c6be6a04`; - both companion VAE files fully loaded on CPU through the same clean ComfyUI: `MiniMaxH3VideoVAE` in FP16 with native internal tiling, and `MiniMaxH3AudioVAE` in FP32 with 32 kHz stereo output; - companion VAE SHA-256 checksums match the official Comfy-Org repackaged files: `7c1f131492e7eddacaac9069a61b81bdd39de5cc96561e677c5eab1cdce5e522` (video) and `8e505d95dd1561d47abd43d4238fd40d9bb1ae9e147ed0a4cba778d76ae4db48` (audio). Not completed at release time: - CUDA/GPU loading of these exact checkpoints; - peak VRAM measurement for a target resolution and frame count; - end-to-end FL2VA or Ref2VA generation; - same-seed visual/audio A/B against BF16 and official pruned variants. Therefore, 20.940 GiB (stock table) or 20.999 GiB (patched dynamic/separate) is the on-disk tensor payload, not a promise that every workflow will remain under 24 GiB. Activations, runtime buffers, resolution, frame count, batch size, and offloading policy determine peak VRAM. ## Conversion provenance The converter reads the original top-level `transformer/` or `transformer_ref/` Diffusers shards from MiniMaxAI directly; it does not create a 61+ GiB merged BF16 intermediate. The nested task-specific transformer folders were not used because their already-packed QKV is head-major. The stock-compatible edition uses global `cat(Q,K,V)` packing expected by unmodified ComfyUI; the patched edition retains the top-level source's separate Q, K, and V ordering. The official FL2VA and Ref2VA pruned files, [`minimax_h3_fl2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors) and [`minimax_h3_ref2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors), were used only as layout references and as the sources of the corresponding FP32 `rope.inv_freq` tensors absent from the raw Diffusers state dicts. The two files under `vae/` are unmodified, bit-identical copies of the standard [`Comfy-Org/MiniMax-H3`](https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/vae) repackages. The original remote-code VAE bundles are more complicated: the visual VAE constructs a causal 3D-CNN encoder and 36-layer ViT decoder from Python and configuration files, while the audio VAE constructs a DAC-lineage encoder and BigVGAN decoder. ComfyUI implements both architectures natively. The single-file visual repack casts all 560 source weight tensors from FP32 to FP16 and embeds the latent statistics; the audio repack preserves 743 tensors, folds 172 weight-normalized pairs into ordinary FP32 weights, and embeds its latent statistics. No remote Python code or external VAE config is required. ## Limitations and intended use This release is intended for research and local ComfyUI inference. It is a new, structurally validated operating point, not a completed quality benchmark. Both variants were profiled separately. Their independently selected quality21 BF16 islands are identical, but this does not establish perceptual quality without end-to-end A/B generation. The separate-QKV runtime also trades the fused projection for three independent ConvRot calls, so it should be treated as an experimental quality/architecture variant until GPU speed and VRAM are measured. ## License This derivative is distributed under the [MiniMax H3 Community License Agreement](LICENSE). The license limits use and distribution to its defined Applicable Territory, excludes the European Union, United Kingdom, Republic of Korea, and United States, and includes additional use and redistribution conditions. Read the full license and [`NOTICE`](NOTICE) before downloading, using, or redistributing these files. ## Links - [Original MiniMax-H3 repository](https://huggingface.co/MiniMaxAI/MiniMax-H3) - [Official ComfyOrg MiniMax-H3 files](https://huggingface.co/Comfy-Org/MiniMax-H3) - [ComfyUI](https://github.com/Comfy-Org/ComfyUI)