Upload README (5).md
Browse files- README (5).md +138 -0
README (5).md
ADDED
|
@@ -0,0 +1,138 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: minimax-h3-community-license
|
| 3 |
+
pipeline_tag: video-to-video
|
| 4 |
+
tags:
|
| 5 |
+
- minimax-h3
|
| 6 |
+
- comfyui
|
| 7 |
+
- workflow
|
| 8 |
+
- ref2v
|
| 9 |
+
- prompt-enhancer
|
| 10 |
+
- character-replacement
|
| 11 |
+
- v2v
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Nugget H3 EasyR2V β Prompt Enhancer & Full Workflow
|
| 15 |
+
|
| 16 |
+
Why spend 30 minutes doing something when you can spend 2 weeks making a tool to automate it. That's basically the whole reason this exists.
|
| 17 |
+
|
| 18 |
+
I wanted an easy to use workflow for us simple folk who want to do **Ref2V** on MiniMax H3 but keep running into trouble replacing characters. A good prompt does 80β90% of the work with H3 R2V β so instead of writing them by hand every time, this workflow writes them for you.
|
| 19 |
+
|
| 20 |
+
Think of it as **"Ref2Video for dummies"**. But it's not just limited to that.
|
| 21 |
+
|
| 22 |
+

|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
## What it does
|
| 27 |
+
|
| 28 |
+
- **Scans your video** (if you're using one) to caption it and transcribe the audio
|
| 29 |
+
- **Captions all your images** β so it also works as a pure image-to-video workflow
|
| 30 |
+
- **Loads a small LLM of your choice** and writes your H3 R2V prompt in the correct format
|
| 31 |
+
- **If you're on the Full workflow**, it generates the video too
|
| 32 |
+
|
| 33 |
+
### The clever part
|
| 34 |
+
|
| 35 |
+
- If your video doesn't change, a second run doesn't trigger a new transcription
|
| 36 |
+
- If your images and user prompt don't change, it doesn't write a new prompt β so you can re-run on a new seed to test without paying the LLM cost every time
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
|
| 40 |
+
## Two workflows
|
| 41 |
+
|
| 42 |
+
| File | What it does |
|
| 43 |
+
|---|---|
|
| 44 |
+
| `Nugget_H3_EasyR2V_Full_WF_v03.json` | The full thing β transcribe + prompt + generate video |
|
| 45 |
+
| `Nugget_H3_EasyR2V_Prompter_Only_v03.json` | Just writes the H3 prompt. Copy it out, or wire it into your own H3 graph |
|
| 46 |
+
|
| 47 |
+
Prompter-only is handy if you've already got your own H3 setup dialled in and just want a better prompt without ripping your workflow apart.
|
| 48 |
+
|
| 49 |
+

|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## Examples
|
| 54 |
+
|
| 55 |
+
**Character replacement** β swap the person in a clip for one from your reference images:
|
| 56 |
+
|
| 57 |
+
<video controls src="https://huggingface.co/datasets/PoopMan333/Examples/resolve/main/Prompt_Enhancer/Character_replacement_example.mp4"></video>
|
| 58 |
+
|
| 59 |
+
Side by side, original vs replaced:
|
| 60 |
+
|
| 61 |
+
<video controls src="https://huggingface.co/datasets/PoopMan333/Examples/resolve/main/Prompt_Enhancer/example%20side%20by%20side.mp4"></video>
|
| 62 |
+
<video controls src="https://huggingface.co/datasets/PoopMan333/Examples/resolve/main/Prompt_Enhancer/example%20side%20by%20side%202.mp4"></video>
|
| 63 |
+
|
| 64 |
+
**Background + character replacement** β swap both in one go:
|
| 65 |
+
|
| 66 |
+
<video controls src="https://huggingface.co/datasets/PoopMan333/Examples/resolve/main/Prompt_Enhancer/BG_and_Char_Replacement_Example.mp4"></video>
|
| 67 |
+
|
| 68 |
+
**Voice line + character replacement** β new person, new dialogue:
|
| 69 |
+
|
| 70 |
+
<video controls src="https://huggingface.co/datasets/PoopMan333/Examples/resolve/main/Prompt_Enhancer/Voice_Line_and_Char_Replacement_Example.mp4"></video>
|
| 71 |
+
|
| 72 |
+

|
| 73 |
+
|
| 74 |
+
---
|
| 75 |
+
|
| 76 |
+
## Setup
|
| 77 |
+
|
| 78 |
+
Both workflow files have a big **START HERE β Downloads & Setup** markdown note pinned inside them, with every model link, folder path and install step. Rather than duplicate it all here, just open the workflow and read the note β it's more accurate than a copy of it would be.
|
| 79 |
+
|
| 80 |
+
The short version:
|
| 81 |
+
|
| 82 |
+
- **Custom nodes**: ComfyUI-Nugget, ComfyUI-KJNodes, and (Full workflow only) ComfyUI-PlagueKind-Nodes
|
| 83 |
+
- **Run the Nugget install script** if you want dialogue transcribed β it installs `faster-whisper` into ComfyUI's own Python. A normal `pip install` goes into the wrong interpreter and it will still say the package is missing
|
| 84 |
+
- **H3 models** from [π€ Comfy-Org/MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3)
|
| 85 |
+
- **Prompt enhancer model** β start with Qwen3-VL 8B fp8_scaled. Drop to nvfp4 on 12 GB cards
|
| 86 |
+
|
| 87 |
+
If the file won't open in ComfyUI, a node pack is missing. Install it, restart, reopen. Bypassing won't help.
|
| 88 |
+
|
| 89 |
+
---
|
| 90 |
+
|
| 91 |
+
## Tips for good character replacement
|
| 92 |
+
|
| 93 |
+
- **If the video is easy, replacement should be easy too.** H3 has a quirk though β if the original person and the new person look too similar, it sometimes converges back to the original. A prompt won't always fix that. If you hit it, look into the **Greenman method** or the **Sam3 noise method** as alternatives *(links to be added)*
|
| 94 |
+
- **More than one person in the scene?** Describe the scene properly. `replace the man wearing white shorts with the man in <picture 1>` beats `replace the man with <picture 1>` every time
|
| 95 |
+
- **Give the LLM some context.** A one-liner in the user prompt like `<video 1> is a video of two girls eating a cup of chocolate ice cream` really helps the LLM understand what it's looking at. Especially useful with multiple scenes
|
| 96 |
+
- **H3 is a tool, you're the one using it.** If you don't specify something, expect expressionless results. **Slop prompt in, slop video out**
|
| 97 |
+
- **Don't expect miracles.** Sometimes less is more, sometimes more is more. You'll get a feel for it
|
| 98 |
+
- **Limit the resolution and length.** There seems to be an arbitrary token limit that's affected by higher res/longer time. Keep it smaller and your success rate goes up
|
| 99 |
+
|
| 100 |
+
---
|
| 101 |
+
|
| 102 |
+
## Caveats (being honest here)
|
| 103 |
+
|
| 104 |
+
- This is still working within the limits of H3, so sometimes it'll go funny. Not much I can do about that from a workflow
|
| 105 |
+
- Clips run best around **8β20 seconds**. Longer than that and the transcriber gets memory-hungry β a minute of 1080p is more RAM than most machines want to spend. Trim first
|
| 106 |
+
- I'm still testing more challenging v2v scenarios. If you hit something weird, let me know
|
| 107 |
+
|
| 108 |
+
---
|
| 109 |
+
|
| 110 |
+
## Re-running cheaply
|
| 111 |
+
|
| 112 |
+
Keep both `seed` widgets on **fixed**. If nothing upstream has changed, ComfyUI skips the transcriber and the prompt enhancer entirely β only the sampler re-runs. Change the sampler seed for a new take, don't touch the others.
|
| 113 |
+
|
| 114 |
+
Set the transcriber or enhancer seed to `randomize` and you'll re-transcribe the whole video every queue for nothing.
|
| 115 |
+
|
| 116 |
+
To skip the video entirely, bypass **Load source video** (Ctrl+B). Prompt is written from the images and your text alone.
|
| 117 |
+
|
| 118 |
+
---
|
| 119 |
+
|
| 120 |
+
## Credit
|
| 121 |
+
|
| 122 |
+
Workflow build, tuning and prompt-enhancer chain by **C_Nugget**.
|
| 123 |
+
|
| 124 |
+
Original single-image workflow by **mackyb** (H3 Basic prompt enhancer v2). This is a modified variant, not their release.
|
| 125 |
+
|
| 126 |
+
---
|
| 127 |
+
|
| 128 |
+
## Licence stuff
|
| 129 |
+
|
| 130 |
+
The workflow files are mine, do what you like with them.
|
| 131 |
+
|
| 132 |
+
**The model isn't.** MiniMax H3 has a community licence that excludes the EU, UK, South Korea and the USA, and it covers **outputs** as well as the weights. If you're planning anything commercial, go read [the actual licence](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE) rather than taking my word for it. I'm not a lawyer.
|
| 133 |
+
|
| 134 |
+
---
|
| 135 |
+
|
| 136 |
+
**[If this has helped you, consider chucking a few dollars my way. Any tips help with the power bills. Thank you.](https://ko-fi.com/c_nugget)**
|
| 137 |
+
|
| 138 |
+
By C_Nugget
|