realrebelai

/

JoyAI-Echo_GGUF

like 13

First the thanks: your ComfyUI_JoyAI_Echo_GGUF_Nodes pack is the foundation my entire multishot patch sits on, and your Q6_K_RM file taught me the single most important lesson in this ecosystem - that the four text_embedding_projection tensors have to ride inside the GGUF (4448 tensors, not 4444).

Studying why your "RM" file worked when third-party conversions produced robotic voices, prompt-reading, and a new face every shot is what every GGUF I've since built is based on. All of them mirror your tensor canon, credited in the README:
https://huggingface.co/joeygambino/joyai-echo-multishot-workflow

Second, and probably more useful to you: while building on the pack I found and fixed four bugs that live in code the base pack ships, so your users are still exposed to them. All fixes are in the patch repo above; happy to PR them into your pack or hand over patches in whatever form is convenient:

1. Memory bank trim is a NO-OP whenever memory_max_size <= num_fix_frames (the common 3/3 config): tail[-0:] returns the whole list, so the bank grows unbounded and every shot conditions on EVERY prior shot - this is the silent cause of the "quality degrades as the video goes on" complaints (waxy skin, contrast crush by the late shots). One-line fix in memory_multishot.py's _trim.

2. VIDEO_FPS is hardcoded to 24.0 in ltx_wrapper.py's rope position math (inherited from JoyAI-Echo's release code). At 25fps output, video rope-time runs 4% fast against audio's true-seconds rope - lip sync accumulates ~40ms/s of mouth-ahead drift and goes visibly off ~10 seconds into every shot. Fix: divide by the actual render fps. This one masqueraded as a model limitation for weeks.

3. Quantized loads skip the global bf16 cast (correctly, to protect fp8 weights) but that lets float32 tensors through on checkpoints that store modulation tables in F32 - crashes with "mat1 and mat2 must have the same dtype". Fix: cast the float32 stragglers individually after load.

4. fp8 Gemma files in the standard HF layout (.weight + .weight_scale) silently load with ZERO modules swapped - the encoder stays bf16 with no warning. Fix: accept both scale-key layouts and warn loudly when a file matches neither.

Your pack made all of this possible - would love to see the fixes reach everyone running it.