Instructions to use FuzzPuppy/LTX-2.3-Foley-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("FuzzPuppy/LTX-2.3-Foley-LoRA") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - LTX.io
How to use FuzzPuppy/LTX-2.3-Foley-LoRA with LTX.io:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download FuzzPuppy/LTX-2.3-Foley-LoRA --local-dir models/LTX-2.3-Foley-LoRA hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Text/image-to-video with the LoRA on the HQ two-stage base pipeline uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path path/to/checkpoint.safetensors \ --distilled-lora path/to/distilled_lora.safetensors 0.8 \ --spatial-upsampler-path path/to/spatial_upsampler.safetensors \ --gemma-root models/gemma-3-12b \ --lora models/LTX-2.3-Foley-LoRA/<weights>.safetensors 1.0 \ --prompt "your prompt here" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8 - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Upload V2A/Foley LoRA 400-step checkpoint
Browse files- .gitattributes +7 -0
- README.md +14 -14
- comparisons/barista-comparison-hf.mp4 +3 -0
- comparisons/sharpei-comparison-hf.mp4 +3 -0
- comparisons/tennis-comparison-hf.mp4 +3 -0
- door-close-hf.mp4 +3 -0
- pineapple-hf.mp4 +3 -0
- racecar-hf.mp4 +3 -0
- squash-hf.mp4 +3 -0
.gitattributes
CHANGED
|
@@ -41,3 +41,10 @@ examples/man-diving-modal-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
| 41 |
examples/pineapple-modal-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 42 |
examples/race-car2-modal-1x-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 43 |
examples/squash-modal-1x-works.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
examples/pineapple-modal-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 42 |
examples/race-car2-modal-1x-worked.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 43 |
examples/squash-modal-1x-works.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
comparisons/barista-comparison-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
comparisons/sharpei-comparison-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
comparisons/tennis-comparison-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
door-close-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
pineapple-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
racecar-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
squash-hf.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -26,8 +26,8 @@ effects.
|
|
| 26 |
## Demo with and without LoRA
|
| 27 |
|
| 28 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 960px; border-radius: 8px;">
|
| 29 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/sharpei-comparison.mp4" type="video/mp4">
|
| 30 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/sharpei-comparison.mp4">Download video</a>
|
| 31 |
</video>
|
| 32 |
|
| 33 |
## Compatibility
|
|
@@ -76,29 +76,29 @@ Example videos generated with the LoRA:
|
|
| 76 |
### Door close
|
| 77 |
|
| 78 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 79 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 80 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 81 |
</video>
|
| 82 |
|
| 83 |
### Pineapple slicing
|
| 84 |
|
| 85 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 86 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 87 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 88 |
</video>
|
| 89 |
|
| 90 |
### Race car
|
| 91 |
|
| 92 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 93 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 94 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 95 |
</video>
|
| 96 |
|
| 97 |
### Squash
|
| 98 |
|
| 99 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 100 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 101 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/
|
| 102 |
</video>
|
| 103 |
|
| 104 |
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
|
@@ -106,13 +106,13 @@ Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
|
| 106 |
### Barista comparison
|
| 107 |
|
| 108 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 109 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/barista-comparison.mp4" type="video/mp4">
|
| 110 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/barista-comparison.mp4">Download video</a>
|
| 111 |
</video>
|
| 112 |
|
| 113 |
### Tennis comparison
|
| 114 |
|
| 115 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 116 |
-
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/tennis-comparison.mp4" type="video/mp4">
|
| 117 |
-
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley/resolve/main/comparisons/tennis-comparison.mp4">Download video</a>
|
| 118 |
</video>
|
|
|
|
| 26 |
## Demo with and without LoRA
|
| 27 |
|
| 28 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 960px; border-radius: 8px;">
|
| 29 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/sharpei-comparison-hf.mp4" type="video/mp4">
|
| 30 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/sharpei-comparison-hf.mp4">Download video</a>
|
| 31 |
</video>
|
| 32 |
|
| 33 |
## Compatibility
|
|
|
|
| 76 |
### Door close
|
| 77 |
|
| 78 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 79 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/door-close-hf.mp4" type="video/mp4">
|
| 80 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/door-close-hf.mp4">Download video</a>
|
| 81 |
</video>
|
| 82 |
|
| 83 |
### Pineapple slicing
|
| 84 |
|
| 85 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 86 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/pineapple-hf.mp4" type="video/mp4">
|
| 87 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/pineapple-hf.mp4">Download video</a>
|
| 88 |
</video>
|
| 89 |
|
| 90 |
### Race car
|
| 91 |
|
| 92 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 93 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/racecar-hf.mp4" type="video/mp4">
|
| 94 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/racecar-hf.mp4">Download video</a>
|
| 95 |
</video>
|
| 96 |
|
| 97 |
### Squash
|
| 98 |
|
| 99 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 100 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/squash-hf.mp4" type="video/mp4">
|
| 101 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/squash-hf.mp4">Download video</a>
|
| 102 |
</video>
|
| 103 |
|
| 104 |
Comparisons showing LTX-2.3 outputs with and without the Foley LoRA:
|
|
|
|
| 106 |
### Barista comparison
|
| 107 |
|
| 108 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 109 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/barista-comparison-hf.mp4" type="video/mp4">
|
| 110 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/barista-comparison-hf.mp4">Download video</a>
|
| 111 |
</video>
|
| 112 |
|
| 113 |
### Tennis comparison
|
| 114 |
|
| 115 |
<video controls playsinline preload="metadata" style="width: 100%; max-width: 720px; border-radius: 8px;">
|
| 116 |
+
<source src="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/tennis-comparison-hf.mp4" type="video/mp4">
|
| 117 |
+
<a href="https://huggingface.co/FuzzPuppy/LTX-2.3-Foley-LoRA/resolve/main/comparisons/tennis-comparison-hf.mp4">Download video</a>
|
| 118 |
</video>
|
comparisons/barista-comparison-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:20d0c5c215daf2b7b690510ba148c8b9de222f13f3b264f445c8a3a93c21e79a
|
| 3 |
+
size 909309
|
comparisons/sharpei-comparison-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e2c88df05334a79de3da97c46ae9d0983b1916993c22601c15cfdde03b9cd19b
|
| 3 |
+
size 2318280
|
comparisons/tennis-comparison-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8188275ca310aa9b81a5583065cb09037a247b85c4014c66758502be7b435a06
|
| 3 |
+
size 1631786
|
door-close-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:61df4e73b48a2978c9bf086630f10623d2112b6233a67c68e3cc18911309ff23
|
| 3 |
+
size 546694
|
pineapple-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c4f83f11b126bfedad6a55e6c4f2d0a954055c92ce4c0893c5a6807b99a31e66
|
| 3 |
+
size 1237476
|
racecar-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f421160f74990d6da0c769b549327f5a48086d5101c4b6f82f341504317a4c61
|
| 3 |
+
size 2013187
|
squash-hf.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c92c520a019e5ebfd1edd34b1a067e97464d971d96784bf7be2a8c04fad6565f
|
| 3 |
+
size 1343496
|