Hugging Face's logo

MATLOWAI
/
MiniMax-H3-Motion-Adapter

Image-to-Video
Diffusers
MiniMax H3
lora
video
comfyui
motion
Model card Files
xet
 Community
2
MiniMax-H3-Motion-Adapter / README.md
matlod's picture
matlod
model card: link the demo page anchor and the node pack
f3e9df0 verified
preview code
|
Raw
History Blame
4.67 kB
metadata 
license: mit
base_model: MiniMaxAI/MiniMax-H3
tags:
  - minimax-h3
  - lora
  - video
  - comfyui
  - motion
library_name: diffusers
pipeline_tag: image-to-video

MiniMax-H3 motion adapter (pilot, r16)

A small LoRA for MiniMax-H3 that makes the de-rope pass in ComfyUI-MAINodes behave better on fast motion. It is a pilot: rank 16, trained for a few hours on 38 clips we generated ourselves. It works, it is measurable, and it has clear rough edges listed below. We are publishing it as an intermediate release while we try something considerably more ambitious that may or may not work (an all-in-one adapter that internalises the whole two-pass pipeline). If that lands it will replace this file; if it does not, this one still does a useful job.

What it does

MAINodes' Motion Lab regenerates the fast span of a clip on a stretched clock (the "de-rope") and recovers it back to real time. That pass has a known bad habit: on very fast motion the base model over-produces, and the result alternates between advance and snap frame to frame. This adapter was trained on that exact task (frames held out of the middle of a burst, the model asked to fill them back in), and it teaches the base model to spend the extra clock on smoothness instead of invention.

Measured on four clips the adapter never saw during training (a fight, an anime sword ring, a character swap, a dialogue close-up), against the same graph without the adapter and against a same-seed noise floor:

  • advance/snap alternation improved on 4 of 4 clips (4.6 to 6.1 times the floor)
  • frame-to-frame over-production dropped from 1.76 to 2.93 down to 1.09 to 1.23 on all four
  • blind operator picks, adapter vs no adapter, same seed: 7 for, 0 against, 1 no preference
  • it transfers to the Ref2VA checkpoint (same wins on both), so one file serves both fl2va and ref2va graphs

Where it is not great (also measured):

  • it costs about 1 dB of anchor fidelity on native keyframes and it over-corrects calm chains (it smooths things that did not need smoothing); keep it off on calm content
  • on the Ref2VA checkpoint it mutes colour. A prismatic creature came out as a plain calico; mean saturation dropped 46 to 38 at denoise 0.40 and 54 to 40 at 0.70. If your reference has strong colour, either lower the strength or accept the trade
  • it is anime-heavy in effect: on very fast anime it wins by a mile at every setting; on live-action character work it needs the lower inject values below

Settings that work

  • Strength 1.0. Strength and inject turned out to be the same dial, and 1.0 wins on alternation in every paired cell we ran; 0.75 was in-distribution tuning on one fight plate and does not hold elsewhere.
  • Inject is the dial. Starting points: 0.45 for character or dialogue where the base model over-produces hard; 0.30 where identity or props are the deliverable and the base is only mildly inflated; on very fast anime it does not matter (every setting wins).
  • Keep the tail guide on. Turning it off costs identity about 3 times the floor across seeds and checkpoints, adapter or not.
  • Apply it to the de-rope pass only, not to the first (text-to-video) pass.

Use in ComfyUI

Put the file in models/loras/minimax_h3/. Load it with a stock LoraLoaderModelOnly at strength 1.0 on the model that feeds the de-rope pass. A ready graph is in ComfyUI-MAINodes: examples/motion_pipeline_adapter_api.json (API format, loads through the API and through the frontend's API import). The stock LoraLoaderModelOnly is fine for this file; the key mismatch that bites turbo LoRAs does not apply here.

Training facts

  • Base: MiniMax-H3 (fl2va), rank 16, alpha 16, 375 steps, lr 1e-4, adamw8bit, bf16, about 4.6 h on one GPU
  • Data: 38 holdout-infilling items cut from 15 clips we generated with MiniMax-H3 (physics scenes, fights, chains, a dialogue plate); no real footage, no third-party footage
  • Task: hold out the frames inside a motion burst, keep the rest as clean context (kept_label 1.0), predict the held tokens
  • Trainer: our ai-toolkit extension for H3 (span-aware packing verified bit-exact against ComfyUI)

Files

  • minimax_h3_motion_adapter_pilot_r16.safetensors (63 MB)
  • motion_pipeline_adapter_api.json (the same graph as in the node pack)

Examples and before/after clips, including the colour-muting case, are on the demo page: https://matlowai.github.io/ComfyUI-MAINodes/#adapter The node pack, the example graph and the tuning notes are at https://github.com/matlowai/ComfyUI-MAINodes (see README, "The motion adapter (pilot)", and TUNING.md).