Instructions to use FuzzPuppy/LTX-2.3-Foley-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("FuzzPuppy/LTX-2.3-Foley-LoRA") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - LTX.io
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download FuzzPuppy/LTX-2.3-Foley-LoRA --local-dir models/LTX-2.3-Foley-LoRA hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Text/image-to-video with the LoRA on the HQ two-stage base pipeline uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path path/to/checkpoint.safetensors \ --distilled-lora path/to/distilled_lora.safetensors 0.8 \ --spatial-upsampler-path path/to/spatial_upsampler.safetensors \ --gemma-root models/gemma-3-12b \ --lora models/LTX-2.3-Foley-LoRA/<weights>.safetensors 1.0 \ --prompt "your prompt here" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Upload V2A/Foley LoRA 400-step checkpoint
Browse files
README.md
CHANGED
|
@@ -10,10 +10,6 @@ tags:
|
|
| 10 |
- lora
|
| 11 |
- safetensors
|
| 12 |
pipeline_tag: other
|
| 13 |
-
widget:
|
| 14 |
-
- text: A barista uses an espresso machine to steam milk. No speech is present. No music is present
|
| 15 |
-
output:
|
| 16 |
-
url: comparisons/barista-comparison.mp4
|
| 17 |
---
|
| 18 |
|
| 19 |
# LTX-2.3 Foley
|
|
@@ -26,8 +22,6 @@ This LoRA is especially useful when LTX-2.3 adds background music, score, or
|
|
| 26 |
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 27 |
effects.
|
| 28 |
|
| 29 |
-
<Gallery />
|
| 30 |
-
|
| 31 |
## Compatibility
|
| 32 |
|
| 33 |
This LoRA is intended for use with LTX-2.3 base or distilled video-to-audio
|
|
@@ -69,8 +63,16 @@ music, melody, song, singing, vocals, score, soundtrack, beat, rhythm bed, instr
|
|
| 69 |
|
| 70 |
## Examples
|
| 71 |
|
| 72 |
-
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
-
|
| 76 |
-
|
|
|
|
|
|
| 10 |
- lora
|
| 11 |
- safetensors
|
| 12 |
pipeline_tag: other
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# LTX-2.3 Foley
|
|
|
|
| 22 |
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 23 |
effects.
|
| 24 |
|
|
|
|
|
|
|
| 25 |
## Compatibility
|
| 26 |
|
| 27 |
This LoRA is intended for use with LTX-2.3 base or distilled video-to-audio
|
|
|
|
| 63 |
|
| 64 |
## Examples
|
| 65 |
|
| 66 |
+
Example videos generated with the LoRA:
|
| 67 |
+
|
| 68 |
+
- [Door close](examples/door-close-modal-worked.mp4)
|
| 69 |
+
- [Man diving](examples/man-diving-modal-worked.mp4)
|
| 70 |
+
- [Pineapple slicing](examples/pineapple-modal-worked.mp4)
|
| 71 |
+
- [Race car](examples/race-car2-modal-1x-worked.mp4)
|
| 72 |
+
- [Squash](examples/squash-modal-1x-works.mp4)
|
| 73 |
+
|
| 74 |
+
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
| 75 |
|
| 76 |
+
- [Barista comparison](comparisons/barista-comparison.mp4)
|
| 77 |
+
- [Shar pei comparison](comparisons/sharpei-comparison.mp4)
|
| 78 |
+
- [Tennis comparison](comparisons/tennis-comparison.mp4)
|