Add prompting guide: H3 expects a structured IR, not prose
Browse files
README.md
CHANGED
|
@@ -93,6 +93,43 @@ quantization error is immediately visible. There is nothing to gain by quantizin
|
|
| 93 |
Note also that the text encoder being NVFP4 buys **VRAM, not speed** (15.7 GB vs 27 GB for
|
| 94 |
int8): it runs once per prompt, not once per sampling step.
|
| 95 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
## Measured performance
|
| 97 |
|
| 98 |
RTX PRO 6000 Blackwell (96 GB), ComfyUI 0.30.0, ref2va, 864×480, 39 frames, 20 steps,
|
|
|
|
| 93 |
Note also that the text encoder being NVFP4 buys **VRAM, not speed** (15.7 GB vs 27 GB for
|
| 94 |
int8): it runs once per prompt, not once per sampling step.
|
| 95 |
|
| 96 |
+
## Prompting: H3 wants a structured IR, not prose
|
| 97 |
+
|
| 98 |
+
**Worth reading before you blame the weights for bad output.** H3 was trained on the
|
| 99 |
+
structured output of **H3-Context-IR**, a preprocessing model that rewrites a plain request
|
| 100 |
+
into labelled sections; MiniMax's model card calls it "critical to the quality of the final
|
| 101 |
+
output". ComfyUI passes your raw string straight to the DiT, so you have to write that
|
| 102 |
+
structure yourself.
|
| 103 |
+
|
| 104 |
+
Official guides: [`docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md)
|
| 105 |
+
and [`..._ref_en.md`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md).
|
| 106 |
+
|
| 107 |
+
```text
|
| 108 |
+
For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.
|
| 109 |
+
|
| 110 |
+
integrated_multimodal_description: [Shot 1] Live-action, cinematic, the young woman shown in
|
| 111 |
+
<Picture 1> remains beside the rain-covered train window, preserving her appearance and the
|
| 112 |
+
carriage layout. The camera trucks right with small amplitude at slow speed as she lifts her
|
| 113 |
+
gaze toward the passing city lights. The quiet, breathy young woman (S1) says:
|
| 114 |
+
<d>[English] I get off at the next station.</d> She folds the letter along its existing crease.
|
| 115 |
+
|
| 116 |
+
overall_soundscape: The train wheels produce a steady metallic rhythm beneath a low
|
| 117 |
+
ventilation hum. Rain ticks against the window while paper rustles softly in her hands.
|
| 118 |
+
|
| 119 |
+
non_diegetic_music: Sustained cello notes at a slow tempo with widely spaced piano tones.
|
| 120 |
+
```
|
| 121 |
+
|
| 122 |
+
**Dialogue must be explicit or you get gibberish.** Speech is generated jointly with video,
|
| 123 |
+
so saying *that* someone speaks without giving the words yields correct prosody and mouth
|
| 124 |
+
shapes with no lexical content. Put speaker identity, action and delivery *outside* `<d>`,
|
| 125 |
+
and only the language tag plus verbatim words *inside* it. Use stable speaker IDs `(S1)`,
|
| 126 |
+
`(S2)`, `(S1,S2)` for simultaneous speech.
|
| 127 |
+
|
| 128 |
+
Other essentials: `[Shot 1]` carries no timestamp, later shots use `[Shot N] At MM:SS.mmm`;
|
| 129 |
+
aim for 350-500 words of description; write camera motion as type + amplitude + speed;
|
| 130 |
+
reference tags must appear in the order the inputs were connected. ref2va takes up to 9 ref
|
| 131 |
+
images, and 3-4 varied shots hold identity far better than one.
|
| 132 |
+
|
| 133 |
## Measured performance
|
| 134 |
|
| 135 |
RTX PRO 6000 Blackwell (96 GB), ComfyUI 0.30.0, ref2va, 864×480, 39 frames, 20 steps,
|