INSTRUCTIONS_LITE.md

Multishot Lite — full instructions

Everything here is also written into note blocks inside the workflow itself, so you can work from the graph alone. This file is the long version.

---

1. Install

There is nothing to install. Every node in this graph ships with ComfyUI.

If a node shows up red on load, your ComfyUI is older than ManualSigmas or ComfySwitchNode — update ComfyUI and reopen.

Open the workflow: Workflows → Open, or drag workflow/LTX23_Multishot_Lite.json onto the canvas.

2. Models — six loaders, five different folders

This trips up nearly everyone: each loader reads a different folder, and a model in the wrong one simply will not appear in its dropdown.

#	node	file	folder
1	Diffusion model	ltx-2.3-22b-distilled-1.1 or another distilled build	models/diffusion_models/
1b	ID-LoRA	LTX-2.3-ID-LoRA-TalkVid-3K.safetensors	models/loras/
2	Video VAE	LTX-2.3 video VAE	models/vae/
3	Gemma + text projection	Gemma-3-12B and the LTX text projection	models/text_encoders/ + models/checkpoints/
4	Audio VAE	LTX-2.3 audio VAE	models/checkpoints/

Node 3 takes two files on purpose: Gemma produces the embeddings, and LTX's text projection maps them into the DiT's conditioning space. Gemma alone is not enough.

The weight_dtype widget on node 1 will drop the model to fp8 without needing a separate file — useful on 24 GB cards.

3. Your two inputs

• 5. Voice reference — 3–5 seconds of clean, solo, continuous speech. No music, no second speaker, no long silences. Put it in ComfyUI/input/.
• 6. Shot 1 first frame — the image shot 1 opens on. Also ComfyUI/input/.

Both ship as <YOUR ...> placeholders, so they will show as unset until you pick your own — that is intentional.

4. First render

1. Set the six loaders and your two inputs.
2. Leave everything else at defaults (960×544, 241 frames ≈ 9.6 s per shot).
3. Queue.
4. Watch multishot_lite/FINAL — that is the joined, refined output. The per-shot files are previews.

5. Audio modes

Mode 1 (default) — your reference voice. The character speaks the quoted line from your prompt in the voice from your sample. Requires the ID-LoRA.

Mode 2 — the model invents a voice. Select both reference nodes (one per shot) and press Ctrl+B. They turn purple. Everything else is unchanged.

There is no track substitution in either mode — what you hear is always what the model generated.

If the audio garbles

Symptom: your line comes out plus extra garbled words.

1. Shorten the voice reference toward 3 seconds. The whole clip becomes conditioning tokens; a long one gives the model words to echo. This is usually the entire fix.
2. Confirm the ID-LoRA is loaded (node 1b). Without it the model has no trained path for identity transfer and echoes fragments instead.
3. Lower identity_guidance_scale toward 1.5 where that widget exists. Note that identity_guidance_scale = 0 is not a clean "no reference" test — it only skips the guidance pass; the reference tokens stay attached. Only Ctrl+B removes them.

5b. Tuning: checkpoint first, then widgets

Start with your checkpoint, not a slider. Lip sync lives in the video branch's cross-attention to the audio conditioning - not in the audio branch. Video-side merges and finetunes routinely rewrite exactly those layers, so a checkpoint can be excellent at scenery and markedly worse at mouths while its audio weights sit untouched. Measured on one merge vs stock ltx-2.3-22b-distilled-1.1: audio branch bit-identical, only 2 of 24 sampled video cross-attention tensors unchanged.

If mouths are poor, A/B a stock checkpoint first. Change only the UNETLoader, keep the seed, change nothing else. Match the schedule: the sigmas here are the 8-step distilled ladder at cfg 1, so compare against a distilled build - never -dev, which needs a many-step cfg>1 schedule and will produce garbage on this ladder and have you blaming the wrong thing.

strength (guide node) - ships at 1.0. Sets frame 0's noise mask to 1.0 - strength, i.e. how free the model is to move that frame. 1.0 pins it to your guide and gives the tightest identity hold; on a healthy checkpoint it still animates.

• Mouth frozen -> 0.8, then 0.7.
• Needing below ~0.6 for any movement -> checkpoint problem, see above.
• Face drifting off your reference / looks like T2V -> back toward 1.0.

img_compression (LTXVPreprocess) - ships at 25. Not a motion dial. LTX is trained on video frames, which carry codec artifacts; a pristine photo is out-of-distribution as a "frame", so a light H.264 round-trip helps the model read it as one. But the value is passed straight to x264 as CRF, so it is a quality-destruction knob:

• x264 clamps CRF at 51 - every value from 51 to 100 is byte-identical. Running it at 100 is not "maximum motion", it is CRF 51.
• At CRF 51 fine detail drops to ~83% of source. Patterned fabric, curtains, hair and skin texture turn to mush and the model invents replacements - it restyles the whole frame, not just the face.

Leave it at 25. If you are tempted past 40, the problem is elsewhere.

identity_guidance_scale + end_percent - ship at 3.0 / 0.5. Guidance amplifies the reference across the WHOLE denoised tensor - audio and video - so run across every step it restyles the face while fixing the voice. Identity is decided in the early high-noise steps; detail forms late. end_percent 0.5 switches guidance off halfway.

• Face stylized -> lower end_percent (0.3-0.4), not the scale.
• Voice not holding -> raise the scale (4-5), leave end_percent.

Change one thing at a time. Every wrong turn in this workflow's history came from moving two dials at once and crediting the wrong one.

6. Length and resolution

Both are set once in the GLOBAL group. length must be 8n+1 (97, 145, 241, 265, 329...), duration = length / 24, dimensions divisible by 32.

Everything downstream of length now follows it automatically (the last-frame picker uses negative indexing, the trims clamp). The one manual value: the AV-extend node's video_end_time/audio_end_time = (length + 72) / 24. At the shipped 241 that is 13.04; for a ~20 s shot set length 265 and end times 14.04.

7. Prompting

Only text inside "double quotes" should be spoken. But the model has a narrator prior and will read your scene description aloud unless told not to — quoting alone is not enough.

The shipped prompts carry an exclusivity block:

They are the only person speaking, and the only voice on the audio track is theirs. One continuous line with natural breaths and brief pauses. No narration, no voice-over, no other speech, and nothing else is read aloud.

Keep that when you rewrite. Other rules that carry over:

• ~26–32 spoken words per shot. Longer and delivery drifts.
• One speaker per shot. Two quoted lines in one prompt tend to come out simultaneously, in the same voice.
• Name the accent explicitly ("a casual American accent") — it drifts otherwise.
• If you are using a first-frame image, describe what should happen, not a fixed person or place. A prompt that contradicts the image makes the model fight the guide, which reads as the image being ignored.

8. The chain (v2: a true extension)

Shot 2 does not restart — it extends shot 1. The last 73 frames (3.04 s) of shot 1's video and audio are encoded as latent context on the AV-extend node (LTXVAudioVideoMask, pad mode), the model generates forward from an ongoing utterance, and the context region is dropped from the output before the join.

Because the audio never restarts, the voice carries over automatically — the chained shot has no reference-audio node and needs none.

To add a shot 3: copy the whole SHOT 2 group and feed its two context nodes from shot 2's decodes instead of shot 1's. Identity drifts slowly over many extensions; for long chains use the full JoyAI-Echo pack (memory bank).

9. The FINISH group

Both shots are joined (video and audio) and refined with a deterministic 1.5× bicubic upscale plus a gentle sharpen. Deterministic on purpose — generative refiners re-invent detail per frame, which reads as texture shimmer.

• Turn refine off: set scale_by to 1.0.
• The sharpen node ships bypassed. Core ImageSharpen moves the whole batch to the GPU and allocates two more full-size copies, so it needs roughly 3× the batch in VRAM — fine for a few hundred frames, an OOM on a 2000-frame run at any resolution. Ctrl+B to enable it on short renders. For long ones, sharpen the finished file instead:

ffmpeg -i FINAL.mp4 -vf "cas=0.55" -c:v libx264 -crf 16 -preset medium \
       -tune grain -bf 0 -c:a copy FINAL_sharp.mp4

• Shot 2's first frame is shot 1's last frame, so the joint holds one duplicated frame — 1/24th of a second, invisible at a cut.

10. Troubleshooting

symptom	cause
A loader dropdown is empty / red	model is in the wrong folder — see §2
Line spoken plus garbled words	voice reference too long, or ID-LoRA missing — §5
The model reads your description aloud	exclusivity block removed from the prompt — §7
Mouth barely moves	A/B a stock distilled checkpoint first — see §5b
Face soft / stylized / "smoothed"	lower img_compression, then end_percent — §5b
Shot 2 looks unrelated to shot 1	ImageFromBatch → batch_index ≠ length − 1
ComfyUI dies during load, RAM at 100%, VRAM idle	system RAM — raise the Windows pagefile to 64–128 GB
OOM in ImageSharpen on a long render	expected; leave it bypassed and use ffmpeg — §9
Video looks soft	scale_by is doing a plain upscale; enable sharpen on short runs