Instructions to use FuzzPuppy/LTX-2.3-Foley-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("FuzzPuppy/LTX-2.3-Foley-LoRA") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - LTX.io
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download FuzzPuppy/LTX-2.3-Foley-LoRA --local-dir models/LTX-2.3-Foley-LoRA hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Text/image-to-video with the LoRA on the HQ two-stage base pipeline uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path path/to/checkpoint.safetensors \ --distilled-lora path/to/distilled_lora.safetensors 0.8 \ --spatial-upsampler-path path/to/spatial_upsampler.safetensors \ --gemma-root models/gemma-3-12b \ --lora models/LTX-2.3-Foley-LoRA/<weights>.safetensors 1.0 \ --prompt "your prompt here" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Upload V2A/Foley LoRA 400-step checkpoint
Browse files- .gitattributes +8 -0
- README.md +73 -2
- comparisons/barista-comparison.mp4 +3 -0
- comparisons/sharpei-comparison.mp4 +3 -0
- comparisons/tennis-comparison.mp4 +3 -0
- examples/door-close-modal-worked.mp4 +3 -0
- examples/man-diving-modal-worked.mp4 +3 -0
- examples/pineapple-modal-worked.mp4 +3 -0
- examples/race-car2-modal-1x-worked.mp4 +3 -0
- examples/squash-modal-1x-works.mp4 +3 -0
- ltx-2.3-foley-400-steps.safetensors +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
comparisons/barista-comparison.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
comparisons/sharpei-comparison.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
comparisons/tennis-comparison.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
examples/door-close-modal-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
examples/man-diving-modal-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
examples/pineapple-modal-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
examples/race-car2-modal-1x-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
examples/squash-modal-1x-works.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,5 +1,76 @@
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
-
|
| 4 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
+
base_model: Lightricks/LTX-2.3
|
| 4 |
+
tags:
|
| 5 |
+
- ltx
|
| 6 |
+
- ltx-2.3
|
| 7 |
+
- video-to-audio
|
| 8 |
+
- foley
|
| 9 |
+
- sound-effects
|
| 10 |
+
- lora
|
| 11 |
+
- safetensors
|
| 12 |
+
pipeline_tag: other
|
| 13 |
+
widget:
|
| 14 |
+
- text: A barista uses an espresso machine to steam milk. No speech is present. No music is present
|
| 15 |
+
output:
|
| 16 |
+
url: comparisons/barista-comparison.mp4
|
| 17 |
---
|
| 18 |
+
|
| 19 |
+
# LTX-2.3 Foley
|
| 20 |
+
|
| 21 |
+
LTX-2.3 Foley is a video-to-audio LoRA for LTX-2.3. It adds realistic,
|
| 22 |
+
visually synchronized Foley and sound effects to video without adding a music
|
| 23 |
+
overlay.
|
| 24 |
+
|
| 25 |
+
This LoRA is especially useful when LTX-2.3 adds background music, score, or
|
| 26 |
+
rhythmic soundtrack material but the desired output is audible real-world sound
|
| 27 |
+
effects.
|
| 28 |
+
|
| 29 |
+
<Gallery />
|
| 30 |
+
|
| 31 |
+
## Compatibility
|
| 32 |
+
|
| 33 |
+
This LoRA is intended for use with LTX-2.3 base or distilled video-to-audio
|
| 34 |
+
models.
|
| 35 |
+
|
| 36 |
+
## License
|
| 37 |
+
|
| 38 |
+
This model is released under the LTX Community License configured on the
|
| 39 |
+
Hugging Face repository.
|
| 40 |
+
|
| 41 |
+
## Files
|
| 42 |
+
|
| 43 |
+
- `ltx-2.3-foley-400-steps.safetensors`: 400-step Foley LoRA weights
|
| 44 |
+
|
| 45 |
+
## Recommended Usage
|
| 46 |
+
|
| 47 |
+
Use a LoRA multiplier between `1` and `3`.
|
| 48 |
+
|
| 49 |
+
Start around `1`. If music or score still appears, increase the multiplier
|
| 50 |
+
toward `2` or `3`.
|
| 51 |
+
|
| 52 |
+
Prompt with a short description of the visible action in the video, followed by:
|
| 53 |
+
|
| 54 |
+
```text
|
| 55 |
+
No speech is present. No music is present.
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
Example prompt:
|
| 59 |
+
|
| 60 |
+
```text
|
| 61 |
+
A barista uses an espresso machine to steam milk. No speech is present. No music is present
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
Recommended negative prompt:
|
| 65 |
+
|
| 66 |
+
```text
|
| 67 |
+
music, melody, song, singing, vocals, score, soundtrack, beat, rhythm bed, instrumental backing, tinny, thin, harsh, clipped, distorted, low bitrate
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
## Examples
|
| 71 |
+
|
| 72 |
+
The embedded gallery and `examples/` folder contain videos generated with the
|
| 73 |
+
LoRA.
|
| 74 |
+
|
| 75 |
+
The `comparisons/` folder contains comparisons showing LTX-2.3 outputs with and
|
| 76 |
+
without the Foley LoRA.
|
comparisons/barista-comparison.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:66f037280e3897a1ce8a2daca6214adf1dd5f1e5c8a944f1963b93f75177dac8
|
| 3 |
+
size 13747193
|
comparisons/sharpei-comparison.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b268120e86599f16331e9281d70a602540671a8c77233617156142a8b23841f5
|
| 3 |
+
size 12837778
|
comparisons/tennis-comparison.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:05a79327dfd69e348d88301138ebb07f50a3c9ed31293c11710f939e39e23db0
|
| 3 |
+
size 21544584
|
examples/door-close-modal-worked.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7c09a7e727cb90e1789c4aadc84af1c963d8f98d5cf2d15379c846f3448f4881
|
| 3 |
+
size 14330901
|
examples/man-diving-modal-worked.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b0b4479cc1a7aa3c43e9394f2f21904518186b2fdb705a099ad5fc4bfe72bed1
|
| 3 |
+
size 1001926
|
examples/pineapple-modal-worked.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:91831e55a7fc60f40ff540f1439bd617ba07e647b84b64306c6becc6a793030f
|
| 3 |
+
size 2048254
|
examples/race-car2-modal-1x-worked.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:33ee4291f469125a47305e4622fd9caf1446252b62e9144186a3c0e34cb9d63f
|
| 3 |
+
size 5184033
|
examples/squash-modal-1x-works.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:004333a1baeaf5f48f82326df810a05622f9e8ed34f58101ffaf2487d3baea6b
|
| 3 |
+
size 1951028
|
ltx-2.3-foley-400-steps.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:12b37ffff8444ac1348a8c9fb24843a6bd70e77cf4202ac39b1d056b81fa41c9
|
| 3 |
+
size 226687352
|