Instructions to use FuzzPuppy/LTX-2.3-Foley-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("FuzzPuppy/LTX-2.3-Foley-LoRA") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - LTX.io
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download FuzzPuppy/LTX-2.3-Foley-LoRA --local-dir models/LTX-2.3-Foley-LoRA hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Text/image-to-video with the LoRA on the HQ two-stage base pipeline uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path path/to/checkpoint.safetensors \ --distilled-lora path/to/distilled_lora.safetensors 0.8 \ --spatial-upsampler-path path/to/spatial_upsampler.safetensors \ --gemma-root models/gemma-3-12b \ --lora models/LTX-2.3-Foley-LoRA/<weights>.safetensors 1.0 \ --prompt "your prompt here" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Upload V2A/Foley LoRA 400-step checkpoint
Browse files
README.md
CHANGED
|
@@ -23,9 +23,12 @@ This LoRA is especially useful when LTX-2.3 adds background music, score, or
|
|
| 23 |
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 24 |
effects.
|
| 25 |
|
| 26 |
-
##
|
| 27 |
|
| 28 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## Compatibility
|
| 31 |
|
|
@@ -72,30 +75,51 @@ Example videos generated with the LoRA:
|
|
| 72 |
|
| 73 |
### Door close
|
| 74 |
|
| 75 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 76 |
|
| 77 |
### Man diving
|
| 78 |
|
| 79 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 80 |
|
| 81 |
### Pineapple slicing
|
| 82 |
|
| 83 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 84 |
|
| 85 |
### Race car
|
| 86 |
|
| 87 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
### Squash
|
| 90 |
|
| 91 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 92 |
|
| 93 |
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
| 94 |
|
| 95 |
### Shar pei comparison
|
| 96 |
|
| 97 |
-
<video
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
### Tennis comparison
|
| 100 |
|
| 101 |
-
<video
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 24 |
effects.
|
| 25 |
|
| 26 |
+
## Demo with and without LoRA
|
| 27 |
|
| 28 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 960px; border-radius: 8px;">
|
| 29 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/barista-comparison.mp4" type="video/mp4">
|
| 30 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/barista-comparison.mp4">Download video</a>
|
| 31 |
+
</video>
|
| 32 |
|
| 33 |
## Compatibility
|
| 34 |
|
|
|
|
| 75 |
|
| 76 |
### Door close
|
| 77 |
|
| 78 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 79 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/door-close-modal-worked.mp4" type="video/mp4">
|
| 80 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/door-close-modal-worked.mp4">Download video</a>
|
| 81 |
+
</video>
|
| 82 |
|
| 83 |
### Man diving
|
| 84 |
|
| 85 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 86 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/man-diving-modal-worked.mp4" type="video/mp4">
|
| 87 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/man-diving-modal-worked.mp4">Download video</a>
|
| 88 |
+
</video>
|
| 89 |
|
| 90 |
### Pineapple slicing
|
| 91 |
|
| 92 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 93 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/pineapple-modal-worked.mp4" type="video/mp4">
|
| 94 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/pineapple-modal-worked.mp4">Download video</a>
|
| 95 |
+
</video>
|
| 96 |
|
| 97 |
### Race car
|
| 98 |
|
| 99 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 100 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/race-car2-modal-1x-worked.mp4" type="video/mp4">
|
| 101 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/race-car2-modal-1x-worked.mp4">Download video</a>
|
| 102 |
+
</video>
|
| 103 |
|
| 104 |
### Squash
|
| 105 |
|
| 106 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 107 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/squash-modal-1x-works.mp4" type="video/mp4">
|
| 108 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/examples/squash-modal-1x-works.mp4">Download video</a>
|
| 109 |
+
</video>
|
| 110 |
|
| 111 |
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
| 112 |
|
| 113 |
### Shar pei comparison
|
| 114 |
|
| 115 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 116 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/sharpei-comparison.mp4" type="video/mp4">
|
| 117 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/sharpei-comparison.mp4">Download video</a>
|
| 118 |
+
</video>
|
| 119 |
|
| 120 |
### Tennis comparison
|
| 121 |
|
| 122 |
+
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 123 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/tennis-comparison.mp4" type="video/mp4">
|
| 124 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/tennis-comparison.mp4">Download video</a>
|
| 125 |
+
</video>
|