AGENTS.md

Deploying this checkpoint

A working recipe for getting one clip out of MiniMax-H3 with Veda block-sparse attention, written so an agent can follow it top to bottom. Every command is meant to be run from the root of a Miowtion clone.

The model is a 33B audio-video DiT. What makes it fit on small cards is that the transformer blocks live in host memory and are streamed to the GPU one at a time; what makes it fast is that attention only visits 10% of the key tiles, chosen per head by the predictor in this repository.

0. What you need

GPU	one CUDA device, compute capability 8.9 / 9.0 / 10.x / 12.x (see §2)
VRAM	24 GB is enough for every geometry with --offload-blocks 50
Host RAM	~40 GB free per process, pinned. This is the binding constraint, not VRAM
Disk	~70 GB for the base model, 263 MB for this predictor
Python	3.10+; 3.12 is what we run
CUDA	a PyTorch build matching your driver; we use torch 2.14 + cu126

The host memory requirement is not negotiable and it is what usually kills a deployment: a second process on the same box gets OOM-killed by the kernel with no traceback, right after it finishes reading weights. If a run dies silently at load weights ... done, you ran out of host RAM, not VRAM.

1. Install

git clone https://github.com/veda-sparse/Miowtion.git
cd Miowtion
python -m venv .venv && . .venv/bin/activate
pip install -e '.[gpu,encode]'

• gpu pulls Triton and flash-attn-4 (CuTe DSL), pinned to commit d15f153 — the sparse path is built against that exact revision and validates it at runtime. Do not upgrade it casually.
• encode pulls the text tower and VAE dependencies, needed once to turn a prompt into a sample cache.

Then fetch the weights:

hf download MiniMaxAI/MiniMax-H3 --local-dir weights/MiniMax-H3
hf download Veda-Sparse/Minimax-H3-T2VA-Veda-8NFE-600Step-Preview \
  --local-dir weights/veda/h3-t2va-8nfe-600

Eight-step generation also needs the few-step Turbo LoRA for FL2VA; point --adapter at it. Without it, run the base schedule at its own step count instead — the predictor in this repository was distilled against the 8-step teacher and expects that trajectory.

Paths above are repo-relative on purpose: weights/, artifacts/ and runs/ are the three directories every config writes to, and they are usually symlinks to wherever the disk actually is.

2. Per-architecture notes

The sparse kernel is FlashAttention-4's CuTe block-sparse path. What differs per architecture is only how the block mask reaches it, and all of it is handled by miowtion/kernels/fa4.py — you do not pass an architecture flag.

SM89 — Ada (RTX 4090, L40S, RTX 6000 Ada)

Upstream FA4 does not implement block sparsity on SM8x, and — this is the dangerous part — it does not say so: passing block-sparse tensors to the SM80 kernel silently computes dense attention at dense cost. Miowtion therefore ships a vendored, patched copy of five FA4 modules (miowtion/kernels/fa4_sm8x) that adds a real block-sparse main loop.

Nothing to configure, but two rules:

1. Never import flash_attn.cute before Miowtion does. The package's __init__ imports the interface, which imports the three other modules, and the patch can only be installed before that happens. install() detects a pre-imported FA4 and raises rather than running unpatched.
2. The patch is pinned to flash-attn-4 4.0.0b32 and verifies the sha256 of the five original modules. A different FA4 version raises instead of patching.

Check it:

python -c "
from miowtion.kernels import fa4
print('block sparsity available:', fa4.available())"

False here means you will get a NotImplementedError on the sparse path rather than a silent dense fallback — that is deliberate.

24 GB cards: use --offload-blocks 50 (stream every block) and --mlp-chunk-rows 4096 at latent_t 102, and export

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,garbage_collection_threshold:0.8

Allocation at 104k tokens is tight, and without expandable segments the allocator fragments and fails on a geometry switch even though the total fits.

SM90 — Hopper (H100, H200)

The native path, and the one FA4 targets first. With head_dim 128 the forward tile is 128×128 with q_stage = 1, which lines up exactly with Veda's 128-row query tiles, so the dense block mask is packed into full/partial index lists and handed straight to the kernel. No patch, no vendored module.

80 GB cards do not need host offloading: drop --offload-blocks to 0 to keep every block resident and skip the H2D stream. That is the single biggest win on Hopper — on a 24 GB card a good fraction of each step is PCIe time.

SM120 — Blackwell (RTX 5090, RTX PRO 6000)

Install exactly as for Hopper; consumer Blackwell needs a CUDA 12.8+ build of PyTorch and an FA4 built for sm_120, so check that

python -c "import torch; print(torch.cuda.get_device_capability())"

reports (12, 0) and that a dense FA4 call works before trying the sparse path. Block sparsity is gated by an architecture whitelist in miowtion/kernels/fa4.py (_SPARSE_MAJOR_ARCHS); majors 9, 10 and 11 are enabled there. If your build provides SM120 block-sparse kernels, add 12 to that tuple — the rest of the pipeline is architecture-independent. With 32 GB of VRAM, --offload-blocks 50 is still the safe setting at latent_t 102.

Datacenter Blackwell (SM100, B200) is enabled: the only wrinkle is that the interface forces q_stage = 2 — and thus 256-row query blocks — whenever seqlen_q > 128, so the wrapper overrides it back to 1 to keep Veda's 128-row tiles. That override checks for the FA4 internal symbol and raises if it has moved.

3. Generate one clip

# 3.1 encode the prompt once
echo '{"id": "demo", "task": "t2va", "prompt": "<structured T2VA prompt>"}' \
  > prompts.jsonl
python scripts/encode_samples.py --root weights/MiniMax-H3 \
  --manifest prompts.jsonl --out artifacts/samples/demo

# 3.2 denoise and decode
CUDA_VISIBLE_DEVICES=0 python scripts/generate.py \
  --root weights/MiniMax-H3 --variant FL2VA \
  --schedule turbo --num-steps 8 \
  --adapter weights/turbo_lora/<8-step-lora>.safetensors \
  --sample-cache artifacts/samples/demo --sample-id demo \
  --geometry 16:9@37 --seed 0 \
  --attention veda \
  --predictor weights/veda/h3-t2va-8nfe-600/minimax_h3_t2va_veda_8nfe_600step_preview_fp8.safetensors \
  --offload-blocks 50 --mlp-chunk-rows 4096 \
  --out-dir artifacts/generate/demo

Output: veda.mp4, veda_latents.pt and summary.json (per-step wall time, attention GPU time, and the speedups) under --out-dir.

The prompt must be a structured T2VA prompt, not a bare sentence; data.validate_prompt rejects the wrong shape at encode time. See the data/prompts/ directory in the Miowtion repository for 1003 expanded examples.

Useful variations:

• --attention dense veda renders both from the same noise and also writes dense_vs_veda.mp4, titled and with both audio tracks. This is the honest way to look at what sparsity costs; step 0 of each mode includes kernel compilation and is excluded from the reported speedups.
• --sample-id a b c --geometry 16:9@37 9:16@72 1:1@102 runs several clips from one model load; with several visible GPUs they run concurrently, one clip per GPU.
• --decode-only re-decodes <mode>_latents.pt from an earlier run without touching the DiT.
• --config configs/infer_*.yaml keeps all of the above in a versioned file; command-line flags override it.

4. Verifying a deployment

pytest tests/unit -q          # CPU, must be green everywhere
pytest tests/gpu -q -m gpu    # needs a CUDA device; auto-skips otherwise

tests/gpu includes the check that matters most here: the block-sparse kernel against a reference implementation over the same mask. If your architecture silently ignores the mask, that test fails instead of quietly producing dense results at dense cost.

For a numeric sanity check of the predictor itself, scripts/predictor_precision.py scores one trajectory with one or more bundles and reports recall against the oracle mask.

5. When it goes wrong

Symptom	Cause	Fix
Process dies after load weights ... done, no traceback	Host OOM killer; ~40 GB pinned per process	Run one process at a time; check free RAM before launching
NotImplementedError: FA4 block sparsity unavailable on sm..	Architecture not whitelisted, or the SM8x patch failed to install	Read the attached reason; on Ada make sure nothing imported flash_attn.cute first
Sparse run is as slow as dense	You are on an unpatched SM8x FA4	fa4.available() must be True; if it is, check that the mask actually reaches the kernel via tests/gpu
expandable_segments: memory mapping failed at latent_t 102	VRAM genuinely exhausted	--offload-blocks 50, --mlp-chunk-rows 4096, and the PYTORCH_CUDA_ALLOC_CONF above
KeyError on the geometry	That aspect/latent_t pair has no plan in the bundle	Use one of the twelve geometries in config.json
Predictor loads but output is noise	A predictor paired with a plan table it was not trained against	Use --predictor (plans travel inside the file) rather than --checkpoint + --plan-dir

6. What this checkpoint is not

It is a preview: 600 updates from scratch plus 100 full-trajectory ones, trained on 5.17 s and 14.4 s clips across all four aspect ratios; the 10.1 s geometries are packed and work, but were never seen in training. It runs under an 8-step Turbo LoRA and was distilled under it — at other step counts the selection is out of distribution. It changes attention block selection and nothing else — not the denoiser, not the schedule, not the VAE. Sparse output is not bit-exact against dense and is not meant to be: at a 10% keep ratio even an oracle mask recovers only about 0.63 of the attention mass at 5.17 s (0.71 at 14.4 s), so treat dense as the reference and look at the side-by-side before shipping anything.