---
license: other
license_name: ltx-2-community-license
license_link: https://huggingface.co/Lightricks/LTX-2/blob/main/LICENSE.txt
tags:
  - comfyui
  - joyai-echo
  - ltx-video
  - multishot
  - patch
---

# JoyAI-Echo GGUF nodes — multishot fixes + automation patch

A set of bug fixes and features layered on top of the community
`ComfyUI_JoyAI_Echo_GGUF_Nodes` pack (the Rebels GGUF loader stack around
JoyAI-Echo). Everything here targets the **multi-shot** path (`JoyEcho_Generate`
+ the discrete Rebels loaders / `JoyEcho_ModelLoader`).

This is a **patch drop**, not a standalone pack: copy these files over a working
install of the same pack (back up first). The files are interdependent — in
particular `nodes.py` calls new signatures added to the two `libs/` files, so
apply them together.

Tested on an RTX 5090 (32 GB) and a 3090 (24 GB), ComfyUI 0.26–0.27,
torch 2.8–2.11, with the JoyAI-Echo bf16 release and self-built Q8 GGUFs.

---

## Files in this package

```
nodes.py                                     # JoyEcho_TextEncode / _Generate / _ModelLoader / _LLMEnhance
__init__.py                                  # registrations for the new nodes
rebels_loaders.py                            # discrete GGUF loaders (text-encoder fixes)
joyecho_prompt_source.py   (new node)        # one dropdown: .txt briefs + .json scripts
joyecho_ref_picker.py      (new node)        # auto reference-image picker by character name
joyecho_ref_batch.py       (new node)        # None-tolerant image batcher
joyecho_script_picker.py   (new node)        # JSON dropdown (superseded by PromptSource)
libs/ltx_distillation/models/ltx_wrapper.py  # fp8 quantization passthrough
libs/ltx_distillation/utils.py               # tiled VAE decode
prompts/long_story_writer_system_prompt.md   # (optional) de-musicked + character-age edits
```

---

## Bug fixes

### 1. `enable_audio_memory=False` silently disabled ALL cross-shot memory
The pack computed `audio_memory_latent=None` when audio memory was off, and the
video **memory-bank save was gated on that latent being non-None** — so with
audio memory off (the standard anti-drone setting) the bank never filled and
cross-shot **identity** silently died (symptom: `memory_size=0` every shot even
with `memory_max_size=7`; a new face each shot).
Fix: memory storage is now unconditional; `enable_audio_memory` gates only the
audio-memory **injection** path. Verify: console `memory_size=` should climb
0,1,2,… capped at your `memory_max_size`. (`nodes.py`)

### 2. GGUF text-encoder loader (`RebelsJE_TextEncoder`)
Two fixes so a text-only Gemma-3 GGUF loads cleanly:
- **meta-strip**: drop `vision_tower` / `multi_modal_projector` / `lm_head`
  (the text-only GGUF has no weights for them → "Cannot copy out of meta tensor").
- **device-unify**: pin the embeddings-processor to the encoder's actual device
  (GGUF Gemma runs on CPU while the connector was on cuda → addmm device mismatch).
(`rebels_loaders.py`)

---

## Features

### 3. Split per-domain negative lever (`JoyEcho_TextEncode`)
The DMD pipeline has no CFG, so the only steering lever is embedding-space.
Instead of one `negative_prompt`/`negative_scale` that steers both branches,
this splits it:
- `negative_prompt_video` / `negative_scale_video` — kills burned-in
  captions/subtitles. Working value ~0.5. **Above ~0.8 it over-rotates the
  video context and locks every shot to shot 1's composition** (scene-lock).
- `negative_prompt_audio` / `negative_scale_audio` — kills invented
  music/score. Keep ≤ ~0.4 or dialogue suffers.
Steering is norm-preserving (RescaleCFG-style): `cond' = renorm(cond + s*(cond − neg))`.
Old single-widget names still work as a fallback. (`nodes.py`)

### 4. Passthrough mode (`JoyEcho_LLMEnhance`)
`mode = "passthrough (raw JSON, skip LLM)"` — feed a finished
`{"prompts":[...]}` script straight through with no LLM call / no API key.
Auto-detects when `story_idea` already parses as that JSON. (`nodes.py`)

### 5. Reference-image conditioning — I2V-as-reference (`JoyEcho_Generate`)
New `reference_image` (IMAGE batch, up to 4). Identity references are prepended
as **video-only conditioning clips** at the memory-encode step — they are
**never** written into the paired audio/video bank. (An earlier attempt that
seeded refs into the bank with zero-filled audio latents injected loud
background noise with 2+ refs; video-only conditioning avoids it entirely.)
Also new: `head_trim_frames` (auto 8 with refs) drops the first N frames of each
shot, where the model morphs out of the reference/memory content. (`nodes.py`)

### 6. Shot transitions (`JoyEcho_Generate`)
`transition`: `cut` (original) / `dissolve` (overlap cross-dissolve + equal-power
audio crossfade) / `vhs_glitch` (analog static burst at each boundary: snow,
tear bands, dropout lines + a raised-cosine tape-noise audio bed).
`transition_frames`, `glitch_intensity` tune it. (`nodes.py`)

### 7. fp8 transformer quantization (`JoyEcho_ModelLoader`)
New `fp8_transformer` toggle. Quantizes the DiT's attention/FF linear weights to
`float8_e4m3fn` **at load, from the normal bf16 checkpoint** (uses the vendored
`ltx_core.quantization.QuantizationPolicy.fp8_cast()` — upcasts per-layer at
inference). Roughly halves DiT weight memory and halves sequential-offload PCIe
traffic; keeps memory training + all tensors; VAEs/text-encoder/non-linears stay
bf16. Ignored when a GGUF DiT is selected (already quantized).
(`nodes.py` + `libs/ltx_distillation/models/ltx_wrapper.py` — new `quantization`
param; the quantized build path skips the post-load dtype cast that would
otherwise silently upcast fp8 back to bf16.)

### 8. Tiled VAE decode (`JoyEcho_Generate`)
Decoding a long high-res shot (e.g. 241f @ 1280×736) in one pass hard-aborts the
VAE decode on a 24–32 GB card (fatal cuDNN abort mid-conv, not a catchable OOM).
New `decode_tiling` (`auto`/`on`/`off`) routes decode through the vendored
`VideoDecoder.tiled_decode` — **temporal-only** 64-frame chunks with 24-frame
blended overlap (no spatial tiles → no spatial seams), streaming each chunk to
CPU. `auto` engages only above a size threshold, so small renders keep the
original single-pass decode bit-for-bit.
(`nodes.py` + `libs/ltx_distillation/utils.py` — `decode_benchmark_sample` gains
a `video_tiling_config` kwarg + `_decode_video_tiled_uint8`.)

### 9. Model dropdown (`JoyEcho_ModelLoader`)
New `model_file` combo lists every `.safetensors` / `.gguf` under the ComfyUI
`checkpoints` / `diffusion_models` / `unet` dirs. Pick a `.safetensors` → full
checkpoint (replaces `checkpoint_path`); pick a `.gguf` → DiT loaded from GGUF
while `checkpoint_path` still supplies the VAEs / vocoder / text connectors.
`"(use checkpoint_path)"` keeps the old typed-path behavior. Plus a clear
early error if `gemma_path` is a `.gguf`/file/sidecar-less dir (this loader
needs the HF `gemma-3-12b-it` folder; GGUF Gemma only works via
`RebelsJE_TextEncoder`). (`nodes.py`)

### 10. Automation / batching nodes (new)
- **`JoyEcho_PromptSource`** — one dropdown listing LPFF-style `.txt` briefs
  (from the inspire-pack prompts tree) **and** passthrough `.json` scripts
  (`input/joyecho_prompts/`). Multi-block briefs fan out like
  LoadPromptsFromFile. Emits `story_idea` (→ LLMEnhance) + `character`
  (→ RefPicker) + `count`. Replaces the LPFF→UnzipPrompt chain and lets you
  switch prompt sources with one dropdown instead of rewiring.
- **`JoyEcho_RefPicker`** — auto-selects a character reference image from a
  folder tree keyed by character name (typed, or wired from PromptSource, or
  scanned from the prompt — dialogue mentions are stripped so only the on-screen
  subject wins). `on_no_match=no_reference` returns nothing so a batch keeps
  running.
- **`JoyEcho_RefBatch`** — None-tolerant image batcher: combines up to 4
  optional IMAGE inputs (e.g. two RefPickers for a two-character shot), skips
  missing refs, resizes mismatched sizes to the first image, outputs `None` if
  all are missing (Generate then just skips identity seeding). The stock KJNodes
  `ImageBatchMulti` crashes with `'NoneType' has no attribute 'shape'` on a
  missing ref; this replaces it.
- **`JoyEcho_ScriptPicker`** — JSON dropdown (superseded by PromptSource; kept
  for compatibility).

---

## Applying

1. Back up your existing pack folder.
2. Copy each file over the same relative path in
   `ComfyUI/custom_nodes/ComfyUI_JoyAI_Echo_GGUF_Nodes/`.
3. Restart ComfyUI. New widgets append at the **end** of existing nodes, so
   saved graphs keep their values; the four new nodes appear under the
   `JoyAI-Echo` category. Press `R` after adding model files to refresh the
   `model_file` dropdown.

The `libs/` files must match the vendored `ltx_core` / `ltx_distillation` in
your pack (same JoyAI-Echo release). If your `libs/` differ substantially,
cherry-pick the changes described above rather than overwriting.

Not included (intentionally): model weights, the `gemma_assets/` tokenizer
binaries, `.bak` snapshots, and `__pycache__`.