INSTRUCTIONS.md

JoyAI-Echo Multishot — Full Instructions

This is the complete, step-by-step guide. If you only read one file, read this one. It covers install, your first render, where the reference image goes, LoRAs, performance settings, and every common failure with its fix.

---

1. What this workflow actually is (read this first — it prevents confusion)

This is a multi-shot text-to-video pipeline with joint audio (voice and picture generated together, not dubbed). You write one or more SHOTS; each shot renders separately, and a cross-shot memory bank carries your character's face, wardrobe, and voice from shot to shot. The thing that makes it work is simple but strict:

Repeat your character's description sentence WORD-FOR-WORD IDENTICAL in every shot. The memory bank keys on it. Reword it even slightly and the face drifts.

It is not a classic image-to-video workflow. You do not have to provide an image at all — text alone holds identity. You can add a reference image to lock the face harder (section 6), but it works differently from normal i2v: it pre-seeds the memory bank rather than becoming frame one.

---

2. What you need

thing	where it goes	notes
RealRebelAI's ComfyUI_JoyAI_Echo_GGUF_Nodes	ComfyUI/custom_nodes/	the BASE pack — install it first
This patch (the folder in this zip)	copied OVER the base pack	replace files when asked
A model build (see table below)	ComfyUI/models/diffusion_models/ (safetensors) or models/unet/ (GGUF)	one of: bf16, fp8, Q8_0, Q5_0
A FULL checkpoint (bf16 or fp8)	ComfyUI/models/diffusion_models/	required even when using a GGUF — the GGUF is the transformer only; the VAEs and vocoder load from the full checkpoint
Gemma-3-12B-it text encoder	ComfyUI/models/text_encoders/	use TRUE BASE Gemma — single-file safetensors or the clean Q8_0 encoder GGUF. Do not use the circulating "joyecho" Gemma GGUF: it is a different model and produces duplicated people
ComfyUI-Easy-Use + ComfyUI-Custom-Scripts	custom_nodes/	two helper packs used by two convenience nodes; or delete those two nodes

Which model build:

you have	use
RTX 40/50-series, 24 GB+	fp8 (fast) or Q8_0 GGUF (highest quant fidelity)
RTX 30-series or older, 24 GB	Q8_0 GGUF
16 GB card	Q5_0 GGUF
32 GB+ and patience	bf16

System RAM and the Windows pagefile - READ THIS IF COMFYUI DIES WHILE LOADING

A 24-32 GB GPU is not the binding constraint. System RAM is. Loading a model here materializes the whole checkpoint in system RAM as owned copies (not memory-mapped), and the quantize / LoRA-fuse pass stacks more on top. The bf16 build is 46 GB on disk, so peak system usage runs well past what 64 GB of RAM alone can provide.

Symptoms (all the same root cause):

• RAM climbs to ~100% while VRAM stays almost idle, then ComfyUI exits
• Windows fatal exception: access violation with a stack ending in sft_loader.py / create_vae_wrappers
• the whole machine hangs or reboots during the first load

There is no catchable Python error for this - the process is killed by the OS.

Fixes, cheapest first:

1. Raise the Windows pagefile. This is the usual fix and costs only disk. Settings > System > About > Advanced system settings > Performance Settings > Advanced > Virtual memory > Change. Untick "Automatically manage", select an SSD, choose Custom size, and set 65536-131072 MB (64-128 GB). Reboot. The peak is brief; with a large pagefile it spills to disk instead of dying. Machines with 64 GB RAM and a default pagefile are the ones that crash.
2. Raise resident_blocks on the Generate node. With sequential_offload on, blocks that are NOT resident live in system RAM. resident_blocks=1 means 47 of 48 blocks sit in RAM while your VRAM goes unused. Push it up until VRAM is nearly full - roughly 24-30 on a 32 GB card, 12 on 24 GB. This moves load off RAM and onto the card you bought.
3. Use a GGUF in model_file - GGUF weights stay packed and memory-mapped rather than copied into RAM.
4. Avoid the fp8 FILE unless you are also using an fp8 toggle. Loaded on its own it is upcast to bf16 at load, so a 25 GB file becomes ~50 GB in RAM. It saves disk, not memory.
5. Close other applications, and load no LoRAs on the first successful run (fusion allocates extra at the peak).

Recent versions print a system-memory estimate before loading and warn when it looks too tight, instead of letting Windows fall over silently.

3. Install

1. Install the base pack (RealRebelAI's) into ComfyUI/custom_nodes/ if you have not already.
2. Copy this zip's ComfyUI_JoyAI_Echo_GGUF_Nodes folder into ComfyUI/custom_nodes/, REPLACING files when asked.
3. Restart ComfyUI.
4. Open workflow/JoyEcho_Multishot_Workflow_PUBLIC.json (drag it onto the ComfyUI canvas, or Workflows > Open).

4. First render in five minutes

1. Copy workflow/example_multishot.json into ComfyUI/input/joyecho_prompts/ (create the folder if it does not exist).
2. In the Model Loader node: pick your DiT in model_file (GGUF or safetensors). If you picked a GGUF, ALSO set checkpoint_path to your full bf16/fp8 checkpoint. Pick your Gemma in gemma_file.
3. In the Prompt Source node: pick example_multishot.json.
4. Queue. First run encodes the prompts (a few minutes); every later run of the same prompts skips encoding entirely (you will see Conditioning cache HIT).

Where your output goes — THREE places, by design:

file	what it is
output/joyecho/shot_000.mp4, shot_001.mp4, ...	each shot, saved the moment it finishes (watch progress live)
output/output_#####.mp4	the assembled full video from SaveVideo (native resolution, with shot transitions)
output/joyecho/<name>_<timestamp>_MASTER.mp4	the upscaled master, built automatically in the background a few minutes after the run ends

5. Writing your own shots

Your prompt file is either a .json file: {"prompts": ["shot 1 text", "shot 2 text", ...]} — or a .txt in the same format the example shows. Rules that matter:

• The identity sentence is byte-identical in every shot. Copy-paste it. Include: appearance, wardrobe, and a voice description WITH an explicit accent ("in a casual American accent" — the model defaults to British if you do not say otherwise).
• One shot is ~12.5 seconds at the default 313 frames. A comfortable spoken line for one shot is about 20-30 words, in double quotes.
• For talking shots, keep the framing medium close-up or tighter and the resolution height 864+. In wide shots the mouth is smaller than the model can resolve and lip sync suffers no matter what you write.
• Describe sound affirmatively in every shot ("Quiet diegetic sound only: rain on the window and a ticking radiator") — otherwise the model invents drones and music.
• Do not write "no X" in a prompt — it renders the X. State what IS there.

6. THE REFERENCE IMAGE — "where do I insert the source image?"

As of v1.4 the graph ships with a Reference Image (OPTIONAL) node just below the Generate node, deliberately unconnected - unconnected nodes never run, so it is safe to ignore. To use it: load your image there and drag its IMAGE output to the reference_image input on the left edge of JoyEcho_Generate. If you are on an older graph without the node:

1. Right-click the canvas > Add Node > image > Load Image.
2. Load your character image (a portrait render or photo-style still works best — clear face, front or three-quarter angle).
3. Drag from the Load Image node's IMAGE output to the reference_image input socket on the left edge of the JoyEcho_Generate node.
4. Queue as normal.

What it does: the image is planted into the cross-shot memory bank as a permanent anchor, so EVERY shot is conditioned on that face and look — on top of (not instead of) your identity sentence. Keep the identity sentence anyway, and make it describe the same person as the image.

The reference_zoom widget (default 1.2) crops the reference in ~20% before conditioning. Why: a reference at exactly the render size is treated as continuable content — the first shot opens ON your image like classic i2v, which usually is not what you want in a multi-shot piece. 1.2 keeps the identity and drops that behavior. If you WANT shot 1 to start from your image (true i2v-style opening), set reference_zoom to 1.0.

Advanced: the RefPicker node can auto-load reference folders per character by name (a folder ComfyUI/input/joyecho_refs/alice/ is used whenever "alice" appears in your prompts). For single-character work the Load Image route above is simpler.

7. LoRAs

As of v1.4 the loader takes multiple LoRAs, and the workflow ships with the node for it: the LoRA Stack node, already wired into the Model Loader.

• Pick up to three LoRAs in its dropdowns, each with its own strength slider. Slots left on (none) do nothing.
• Need more than three? Add another LoRA Stack node (Add Node > JoyAI-Echo > JoyEcho LoRA Stack) and chain it: first node's lora_stack output into the second node's lora_stack input, second node into the Model Loader.
• Everything stacks with the Model Loader's own lora_file pick.

Power users / automation: lora_path also accepts a text list — one entry per line or comma-separated, each with an optional strength suffix (my_style.safetensors@0.7). Same result, no extra nodes.

All LoRAs are fused into the model at load, so there is no per-step cost. Note: LoRAs apply on the safetensors DiT path only — they are ignored when a GGUF is selected as the DiT (existing limitation, stated in the tooltip).

8. Performance settings by VRAM

widget	what to set
sequential_offload	ON below 32 GB VRAM. Streams transformer blocks over PCIe.
resident_blocks	With offload ON: how many of the 48 blocks stay on the GPU. Rough guide: 24 GB card + fp8/GGUF -> ~20-24; 16 GB -> 8-12; raise until VRAM is nearly full.
fp8_scaled_mm	RTX 40/50-series ONLY. Native fp8 compute, real speedup. Leave OFF on 30-series and older.
low_vram (encoder)	ON if Gemma encoding OOMs; slower but ~3 GB.
resolution	544x960 is a fast preview. For dialogue, raise height to 864+.
hires_factor	1.0 = off (default). The background AutoFinish upscale usually covers you.
head_trim_frames	Leave at 14 (default). Trims the unstable first ~0.5s of each shot, where identity morph and lip-sync error concentrate.

9. After updating this patch — one-time step

Open your workflow, glance at the LAST few widgets on the Generate node (reference_zoom / resident_blocks / hires), fix anything that looks wrong, and SAVE. ComfyUI stores widget values by position; a graph saved under an older pack version can load a few trailing widgets scrambled until you re-save it once. If you ever find yourself re-entering the same two or three values after every restart — this is why, and one save ends it.

10. Troubleshooting

symptom	cause / fix
Error: "checkpoint_path is empty"	You picked a GGUF in model_file. GGUFs are the transformer only — set checkpoint_path to a full bf16/fp8 checkpoint too.
Two copies of your character / wrong wardrobe / garbled identity	Wrong text encoder. Use TRUE BASE Gemma-3-12B-it (safetensors or the clean Q8_0 encoder GGUF). The "joyecho" Gemma GGUF is a different model — replace it.
Face drifts between shots	Your identity sentence is not byte-identical across shots. Copy-paste it; do not reword.
Voice is British	State the accent in every shot: "in a casual American accent".
Random music / horror drones	Add an explicit sound line to every shot; leave negative_scale_audio at 0.3.
Lip sync poor in wide shots	Physics of resolution: mouth too small. Medium close-up + height 864+ for talking shots.
Widgets reset every restart	Section 9 — re-save the graph once.
First run very slow	Normal: prompt encoding + caches build. Subsequent runs with the same prompts skip encoding entirely.
Master video missing	It arrives a few minutes AFTER the run ends (background worker). Check output/joyecho/ for *_MASTER.mp4 and the _autofinish_*.log next to it.
OOM during generation	sequential_offload ON, lower resident_blocks, lower resolution, low_vram ON for the encoder.

---

Model downloads, measurements, and a live demo Space are linked from the model listings. AI-generated content must be disclosed as such (LTX-2 Community License; the JoyAI-Echo component is research / non-commercial).