lilcheaty commited on
Commit
a916479
·
verified ·
1 Parent(s): 5d03260

Add prompting guide: H3 expects a structured IR, not prose

Browse files
Files changed (1) hide show
  1. README.md +37 -0
README.md CHANGED
@@ -93,6 +93,43 @@ quantization error is immediately visible. There is nothing to gain by quantizin
93
  Note also that the text encoder being NVFP4 buys **VRAM, not speed** (15.7 GB vs 27 GB for
94
  int8): it runs once per prompt, not once per sampling step.
95
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96
  ## Measured performance
97
 
98
  RTX PRO 6000 Blackwell (96 GB), ComfyUI 0.30.0, ref2va, 864×480, 39 frames, 20 steps,
 
93
  Note also that the text encoder being NVFP4 buys **VRAM, not speed** (15.7 GB vs 27 GB for
94
  int8): it runs once per prompt, not once per sampling step.
95
 
96
+ ## Prompting: H3 wants a structured IR, not prose
97
+
98
+ **Worth reading before you blame the weights for bad output.** H3 was trained on the
99
+ structured output of **H3-Context-IR**, a preprocessing model that rewrites a plain request
100
+ into labelled sections; MiniMax's model card calls it "critical to the quality of the final
101
+ output". ComfyUI passes your raw string straight to the DiT, so you have to write that
102
+ structure yourself.
103
+
104
+ Official guides: [`docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_base_en.md)
105
+ and [`..._ref_en.md`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md).
106
+
107
+ ```text
108
+ For the target video, at 0.00 seconds into the target video, <Picture 1> (from [Shot 1]) is fully referenced.
109
+
110
+ integrated_multimodal_description: [Shot 1] Live-action, cinematic, the young woman shown in
111
+ <Picture 1> remains beside the rain-covered train window, preserving her appearance and the
112
+ carriage layout. The camera trucks right with small amplitude at slow speed as she lifts her
113
+ gaze toward the passing city lights. The quiet, breathy young woman (S1) says:
114
+ <d>[English] I get off at the next station.</d> She folds the letter along its existing crease.
115
+
116
+ overall_soundscape: The train wheels produce a steady metallic rhythm beneath a low
117
+ ventilation hum. Rain ticks against the window while paper rustles softly in her hands.
118
+
119
+ non_diegetic_music: Sustained cello notes at a slow tempo with widely spaced piano tones.
120
+ ```
121
+
122
+ **Dialogue must be explicit or you get gibberish.** Speech is generated jointly with video,
123
+ so saying *that* someone speaks without giving the words yields correct prosody and mouth
124
+ shapes with no lexical content. Put speaker identity, action and delivery *outside* `<d>`,
125
+ and only the language tag plus verbatim words *inside* it. Use stable speaker IDs `(S1)`,
126
+ `(S2)`, `(S1,S2)` for simultaneous speech.
127
+
128
+ Other essentials: `[Shot 1]` carries no timestamp, later shots use `[Shot N] At MM:SS.mmm`;
129
+ aim for 350-500 words of description; write camera motion as type + amplitude + speed;
130
+ reference tags must appear in the order the inputs were connected. ref2va takes up to 9 ref
131
+ images, and 3-4 varied shots hold identity far better than one.
132
+
133
  ## Measured performance
134
 
135
  RTX PRO 6000 Blackwell (96 GB), ComfyUI 0.30.0, ref2va, 864×480, 39 frames, 20 steps,