# Deploying this checkpoint A working recipe for getting one clip out of MiniMax-H3 with Veda block-sparse attention, written so an agent can follow it top to bottom. Every command is meant to be run from the root of a `Miowtion` clone. The model is a 33B audio-video DiT. What makes it fit on small cards is that the transformer blocks live in **host memory** and are streamed to the GPU one at a time; what makes it fast is that attention only visits 10% of the key tiles, chosen per head by the predictor in this repository. ## 0. What you need | | | |---|---| | GPU | one CUDA device, compute capability 8.9 / 9.0 / 10.x / 12.x (see §2) | | VRAM | 24 GB is enough for every geometry with `--offload-blocks 50` | | Host RAM | **~40 GB free per process**, pinned. This is the binding constraint, not VRAM | | Disk | ~70 GB for the base model, 263 MB for this predictor | | Python | 3.10+; 3.12 is what we run | | CUDA | a PyTorch build matching your driver; we use torch 2.14 + cu126 | The host memory requirement is not negotiable and it is what usually kills a deployment: a second process on the same box gets OOM-killed by the kernel with **no traceback**, right after it finishes reading weights. If a run dies silently at `load weights ... done`, you ran out of host RAM, not VRAM. ## 1. Install ```bash git clone https://github.com/veda-sparse/Miowtion.git cd Miowtion python -m venv .venv && . .venv/bin/activate pip install -e '.[gpu,encode]' ``` - `gpu` pulls Triton and **flash-attn-4 (CuTe DSL), pinned to commit `d15f153`** — the sparse path is built against that exact revision and validates it at runtime. Do not upgrade it casually. - `encode` pulls the text tower and VAE dependencies, needed once to turn a prompt into a sample cache. Then fetch the weights: ```bash hf download MiniMaxAI/MiniMax-H3 --local-dir weights/MiniMax-H3 hf download Veda-Sparse/Minimax-H3-T2VA-Veda-8NFE-600Step-Preview \ --local-dir weights/veda/h3-t2va-8nfe-600 ``` Eight-step generation also needs the few-step Turbo LoRA for FL2VA; point `--adapter` at it. Without it, run the base schedule at its own step count instead — the predictor in this repository was distilled against the 8-step teacher and expects that trajectory. Paths above are repo-relative on purpose: `weights/`, `artifacts/` and `runs/` are the three directories every config writes to, and they are usually symlinks to wherever the disk actually is. ## 2. Per-architecture notes The sparse kernel is FlashAttention-4's CuTe block-sparse path. What differs per architecture is only how the block mask reaches it, and all of it is handled by `miowtion/kernels/fa4.py` — you do not pass an architecture flag. ### SM89 — Ada (RTX 4090, L40S, RTX 6000 Ada) Upstream FA4 does **not** implement block sparsity on SM8x, and — this is the dangerous part — it does not say so: passing block-sparse tensors to the SM80 kernel silently computes dense attention at dense cost. Miowtion therefore ships a vendored, patched copy of five FA4 modules (`miowtion/kernels/fa4_sm8x`) that adds a real block-sparse main loop. Nothing to configure, but two rules: 1. **Never import `flash_attn.cute` before Miowtion does.** The package's `__init__` imports the interface, which imports the three other modules, and the patch can only be installed before that happens. `install()` detects a pre-imported FA4 and raises rather than running unpatched. 2. The patch is pinned to flash-attn-4 `4.0.0b32` and verifies the sha256 of the five original modules. A different FA4 version raises instead of patching. Check it: ```bash python -c " from miowtion.kernels import fa4 print('block sparsity available:', fa4.available())" ``` `False` here means you will get a `NotImplementedError` on the sparse path rather than a silent dense fallback — that is deliberate. 24 GB cards: use `--offload-blocks 50` (stream every block) and `--mlp-chunk-rows 4096` at latent_t 102, and export ```bash export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,garbage_collection_threshold:0.8 ``` Allocation at 104k tokens is tight, and without expandable segments the allocator fragments and fails on a geometry switch even though the total fits. ### SM90 — Hopper (H100, H200) The native path, and the one FA4 targets first. With head_dim 128 the forward tile is 128×128 with `q_stage = 1`, which lines up exactly with Veda's 128-row query tiles, so the dense block mask is packed into full/partial index lists and handed straight to the kernel. No patch, no vendored module. 80 GB cards do not need host offloading: drop `--offload-blocks` to 0 to keep every block resident and skip the H2D stream. That is the single biggest win on Hopper — on a 24 GB card a good fraction of each step is PCIe time. ### SM120 — Blackwell (RTX 5090, RTX PRO 6000) Install exactly as for Hopper; consumer Blackwell needs a CUDA 12.8+ build of PyTorch and an FA4 built for `sm_120`, so check that ```bash python -c "import torch; print(torch.cuda.get_device_capability())" ``` reports `(12, 0)` and that a dense FA4 call works before trying the sparse path. Block sparsity is gated by an architecture whitelist in `miowtion/kernels/fa4.py` (`_SPARSE_MAJOR_ARCHS`); majors 9, 10 and 11 are enabled there. If your build provides SM120 block-sparse kernels, add `12` to that tuple — the rest of the pipeline is architecture-independent. With 32 GB of VRAM, `--offload-blocks 50` is still the safe setting at latent_t 102. Datacenter Blackwell (SM100, B200) is enabled: the only wrinkle is that the interface forces `q_stage = 2` — and thus 256-row query blocks — whenever `seqlen_q > 128`, so the wrapper overrides it back to 1 to keep Veda's 128-row tiles. That override checks for the FA4 internal symbol and raises if it has moved. ## 3. Generate one clip ```bash # 3.1 encode the prompt once echo '{"id": "demo", "task": "t2va", "prompt": ""}' \ > prompts.jsonl python scripts/encode_samples.py --root weights/MiniMax-H3 \ --manifest prompts.jsonl --out artifacts/samples/demo # 3.2 denoise and decode CUDA_VISIBLE_DEVICES=0 python scripts/generate.py \ --root weights/MiniMax-H3 --variant FL2VA \ --schedule turbo --num-steps 8 \ --adapter weights/turbo_lora/<8-step-lora>.safetensors \ --sample-cache artifacts/samples/demo --sample-id demo \ --geometry 16:9@37 --seed 0 \ --attention veda \ --predictor weights/veda/h3-t2va-8nfe-600/minimax_h3_t2va_veda_8nfe_600step_preview_fp8.safetensors \ --offload-blocks 50 --mlp-chunk-rows 4096 \ --out-dir artifacts/generate/demo ``` Output: `veda.mp4`, `veda_latents.pt` and `summary.json` (per-step wall time, attention GPU time, and the speedups) under `--out-dir`. The prompt must be a **structured T2VA prompt**, not a bare sentence; `data.validate_prompt` rejects the wrong shape at encode time. See the `data/prompts/` directory in the Miowtion repository for 1003 expanded examples. Useful variations: - `--attention dense veda` renders both from the same noise and also writes `dense_vs_veda.mp4`, titled and with both audio tracks. This is the honest way to look at what sparsity costs; step 0 of each mode includes kernel compilation and is excluded from the reported speedups. - `--sample-id a b c --geometry 16:9@37 9:16@72 1:1@102` runs several clips from one model load; with several visible GPUs they run concurrently, one clip per GPU. - `--decode-only` re-decodes `_latents.pt` from an earlier run without touching the DiT. - `--config configs/infer_*.yaml` keeps all of the above in a versioned file; command-line flags override it. ## 4. Verifying a deployment ```bash pytest tests/unit -q # CPU, must be green everywhere pytest tests/gpu -q -m gpu # needs a CUDA device; auto-skips otherwise ``` `tests/gpu` includes the check that matters most here: the block-sparse kernel against a reference implementation over the same mask. If your architecture silently ignores the mask, that test fails instead of quietly producing dense results at dense cost. For a numeric sanity check of the predictor itself, `scripts/predictor_precision.py` scores one trajectory with one or more bundles and reports recall against the oracle mask. ## 5. When it goes wrong | Symptom | Cause | Fix | |---|---|---| | Process dies after `load weights ... done`, no traceback | Host OOM killer; ~40 GB pinned per process | Run one process at a time; check free RAM before launching | | `NotImplementedError: FA4 block sparsity unavailable on sm..` | Architecture not whitelisted, or the SM8x patch failed to install | Read the attached reason; on Ada make sure nothing imported `flash_attn.cute` first | | Sparse run is as slow as dense | You are on an unpatched SM8x FA4 | `fa4.available()` must be `True`; if it is, check that the mask actually reaches the kernel via `tests/gpu` | | `expandable_segments: memory mapping failed` at latent_t 102 | VRAM genuinely exhausted | `--offload-blocks 50`, `--mlp-chunk-rows 4096`, and the `PYTORCH_CUDA_ALLOC_CONF` above | | `KeyError` on the geometry | That aspect/latent_t pair has no plan in the bundle | Use one of the twelve geometries in `config.json` | | Predictor loads but output is noise | A predictor paired with a plan table it was not trained against | Use `--predictor` (plans travel inside the file) rather than `--checkpoint` + `--plan-dir` | ## 6. What this checkpoint is not It is a **preview**: 600 updates from scratch plus 100 full-trajectory ones, trained on 5.17 s and 14.4 s clips across all four aspect ratios; the 10.1 s geometries are packed and work, but were never seen in training. It runs under an 8-step Turbo LoRA and was distilled under it — at other step counts the selection is out of distribution. It changes attention block selection and nothing else — not the denoiser, not the schedule, not the VAE. Sparse output is not bit-exact against dense and is not meant to be: at a 10% keep ratio even an oracle mask recovers only about 0.63 of the attention mass at 5.17 s (0.71 at 14.4 s), so treat dense as the reference and look at the side-by-side before shipping anything.