| --- |
| license: mit |
| base_model: MiniMaxAI/MiniMax-H3 |
| tags: |
| - minimax-h3 |
| - lora |
| - video |
| - comfyui |
| - motion |
| library_name: diffusers |
| pipeline_tag: image-to-video |
| --- |
| |
| # MiniMax-H3 motion adapter (pilot, r16) |
|
|
| A small LoRA for MiniMax-H3 that makes the de-rope pass in |
| [ComfyUI-MAINodes](https://github.com/matlowai/ComfyUI-MAINodes) behave |
| better on fast motion. It is a pilot: rank 16, trained for a few hours on |
| 38 clips we generated ourselves. It works, it is measurable, and it has |
| clear rough edges listed below. We are publishing it as an intermediate |
| release while we try something considerably more ambitious that may or may |
| not work (an all-in-one adapter that internalises the whole two-pass |
| pipeline). If that lands it will replace this file; if it does not, this |
| one still does a useful job. |
|
|
| ## What it does |
|
|
| MAINodes' Motion Lab regenerates the fast span of a clip on a stretched |
| clock (the "de-rope") and recovers it back to real time. That pass has a |
| known bad habit: on very fast motion the base model over-produces, and the |
| result alternates between advance and snap frame to frame. This adapter |
| was trained on that exact task (frames held out of the middle of a burst, |
| the model asked to fill them back in), and it teaches the base model to |
| spend the extra clock on smoothness instead of invention. |
|
|
| Measured on four clips the adapter never saw during training (a fight, |
| an anime sword ring, a character swap, a dialogue close-up), against the |
| same graph without the adapter and against a same-seed noise floor: |
|
|
| - advance/snap alternation improved on 4 of 4 clips (4.6 to 6.1 times the |
| floor) |
| - frame-to-frame over-production dropped from 1.76 to 2.93 down to 1.09 |
| to 1.23 on all four |
| - blind operator picks, adapter vs no adapter, same seed: 7 for, 0 |
| against, 1 no preference |
| - it transfers to the Ref2VA checkpoint (same wins on both), so one file |
| serves both fl2va and ref2va graphs |
|
|
| Where it is not great (also measured): |
|
|
| - it costs about 1 dB of anchor fidelity on native keyframes and it |
| over-corrects calm chains (it smooths things that did not need |
| smoothing); keep it off on calm content |
| - on the Ref2VA checkpoint it mutes colour. A prismatic creature came out |
| as a plain calico; mean saturation dropped 46 to 38 at denoise 0.40 and |
| 54 to 40 at 0.70. If your reference has strong colour, either lower the |
| strength or accept the trade |
| - it is anime-heavy in effect: on very fast anime it wins by a mile at |
| every setting; on live-action character work it needs the lower inject |
| values below |
|
|
| ## Settings that work |
|
|
| - Strength 1.0. Strength and inject turned out to be the same dial, and |
| 1.0 wins on alternation in every paired cell we ran; 0.75 was |
| in-distribution tuning on one fight plate and does not hold elsewhere. |
| - Inject is the dial. Starting points: 0.45 for character or dialogue |
| where the base model over-produces hard; 0.30 where identity or props |
| are the deliverable and the base is only mildly inflated; on very fast |
| anime it does not matter (every setting wins). |
| - Keep the tail guide on. Turning it off costs identity about 3 times the |
| floor across seeds and checkpoints, adapter or not. |
| - Apply it to the de-rope pass only, not to the first (text-to-video) |
| pass. |
|
|
| ## Use in ComfyUI |
|
|
| Put the file in `models/loras/minimax_h3/`. Load it with a stock |
| `LoraLoaderModelOnly` at strength 1.0 on the model that feeds the de-rope |
| pass. A ready graph is in ComfyUI-MAINodes: |
| `examples/motion_pipeline_adapter_api.json` (API format, loads through the |
| API and through the frontend's API import). The stock |
| `LoraLoaderModelOnly` is fine for this file; the key mismatch that bites |
| turbo LoRAs does not apply here. |
|
|
| ## Training facts |
|
|
| - Base: MiniMax-H3 (fl2va), rank 16, alpha 16, 375 steps, lr 1e-4, |
| adamw8bit, bf16, about 4.6 h on one GPU |
| - Data: 38 holdout-infilling items cut from 15 clips we generated with |
| MiniMax-H3 (physics scenes, fights, chains, a dialogue plate); no real |
| footage, no third-party footage |
| - Task: hold out the frames inside a motion burst, keep the rest as clean |
| context (kept_label 1.0), predict the held tokens |
| - Trainer: our ai-toolkit extension for H3 (span-aware packing verified |
| bit-exact against ComfyUI) |
| |
| ## Files |
| |
| - `minimax_h3_motion_adapter_pilot_r16.safetensors` (63 MB) |
| - `motion_pipeline_adapter_api.json` (the same graph as in the node pack) |
|
|
| Examples and before/after clips, including the colour-muting case, are on |
| the demo page: https://matlowai.github.io/ComfyUI-MAINodes/#adapter |
| The node pack, the example graph and the tuning notes are at |
| https://github.com/matlowai/ComfyUI-MAINodes (see README, "The motion |
| adapter (pilot)", and TUNING.md). |
|
|
