Instructions to use FuzzPuppy/LTX-2.3-Foley-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("FuzzPuppy/LTX-2.3-Foley-LoRA") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - LTX.io
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download FuzzPuppy/LTX-2.3-Foley-LoRA --local-dir models/LTX-2.3-Foley-LoRA hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Text/image-to-video with the LoRA on the HQ two-stage base pipeline uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path path/to/checkpoint.safetensors \ --distilled-lora path/to/distilled_lora.safetensors 0.8 \ --spatial-upsampler-path path/to/spatial_upsampler.safetensors \ --gemma-root models/gemma-3-12b \ --lora models/LTX-2.3-Foley-LoRA/<weights>.safetensors 1.0 \ --prompt "your prompt here" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Upload V2A/Foley LoRA 400-step checkpoint
Browse files
README.md
CHANGED
|
@@ -23,6 +23,10 @@ This LoRA is especially useful when LTX-2.3 adds background music, score, or
|
|
| 23 |
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 24 |
effects.
|
| 25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
## Compatibility
|
| 27 |
|
| 28 |
This LoRA is intended for use with LTX-2.3 base or distilled video-to-audio
|
|
@@ -66,14 +70,32 @@ music, melody, song, singing, vocals, score, soundtrack, beat, rhythm bed, instr
|
|
| 66 |
|
| 67 |
Example videos generated with the LoRA:
|
| 68 |
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
| 76 |
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 24 |
effects.
|
| 25 |
|
| 26 |
+
## Comparison
|
| 27 |
+
|
| 28 |
+
<video src="comparisons/barista-comparison.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 960px; border-radius: 8px;"></video>
|
| 29 |
+
|
| 30 |
## Compatibility
|
| 31 |
|
| 32 |
This LoRA is intended for use with LTX-2.3 base or distilled video-to-audio
|
|
|
|
| 70 |
|
| 71 |
Example videos generated with the LoRA:
|
| 72 |
|
| 73 |
+
### Door close
|
| 74 |
+
|
| 75 |
+
<video src="examples/door-close-modal-worked.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|
| 76 |
+
|
| 77 |
+
### Man diving
|
| 78 |
+
|
| 79 |
+
<video src="examples/man-diving-modal-worked.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|
| 80 |
+
|
| 81 |
+
### Pineapple slicing
|
| 82 |
+
|
| 83 |
+
<video src="examples/pineapple-modal-worked.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|
| 84 |
+
|
| 85 |
+
### Race car
|
| 86 |
+
|
| 87 |
+
<video src="examples/race-car2-modal-1x-worked.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|
| 88 |
+
|
| 89 |
+
### Squash
|
| 90 |
+
|
| 91 |
+
<video src="examples/squash-modal-1x-works.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|
| 92 |
|
| 93 |
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
| 94 |
|
| 95 |
+
### Shar pei comparison
|
| 96 |
+
|
| 97 |
+
<video src="comparisons/sharpei-comparison.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|
| 98 |
+
|
| 99 |
+
### Tennis comparison
|
| 100 |
+
|
| 101 |
+
<video src="comparisons/tennis-comparison.mp4" controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;"></video>
|