---
license: cc-by-4.0
base_model:
  - Wan-AI/Wan2.1-FLF2V-14B-720P
datasets:
  - maxwelljones14/refVFX_dataset
tags:
  - lora
  - video
  - video-editing
  - visual-effects
  - vfx
  - temporal-transitions
  - wan2.1
  - image-to-video
  - video-to-video
pipeline_tag: image-to-video
library_name: peft
pretty_name: RefVFX LoRA (Wan2.1-FLF2V-14B-720P)
---

# RefVFX LoRA — Tuning-free Visual Effect Transfer across Videos

This is a **LoRA adapter** for [`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P),
trained for the **RefVFX** project on tuning-free visual effect transfer across videos.

> **Note:** This is an *unofficial* reimplementation produced at CMU. All code and training data were
> created from scratch using the publicly available arXiv paper and AI coding tools as the only resources.

## Links

- 📄 **Paper:** [Tuning-free Visual Effect Transfer across Videos](https://arxiv.org/abs/2601.07833) (arXiv:2601.07833)
- 🌐 **Project page:** https://snap-research.github.io/RefVFX/
- 💻 **Code (GitHub):** https://github.com/maxwelljones14/refVFX
- 🗂️ **Training dataset:** [`maxwelljones14/refVFX_dataset`](https://huggingface.co/datasets/maxwelljones14/refVFX_dataset)
- 🧱 **Base model:** [`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P)

## Model Details

- **Type:** LoRA adapter (fine-tune on top of Wan2.1-FLF2V-14B-720P)
- **Base model:** `Wan-AI/Wan2.1-FLF2V-14B-720P` (first-last-frame-to-video, 14B parameters, 720P)
- **Training data:** [`maxwelljones14/refVFX_dataset`](https://huggingface.co/datasets/maxwelljones14/refVFX_dataset)
- **Training steps:** 40K steps
- **Hardware:** 8× NVIDIA Blackwell GPUs
- **LoRA rank:** 1024
- **Precision:** bf16
- **Target modules:** all 40 transformer blocks — self-attention (`q`, `k`, `v`, `o`),
  cross-attention (`q`, `k`, `v`, `o`), and feed-forward (`ffn.0`, `ffn.2`)
- **Checkpoint file:** `step-10000.safetensors`

## What it does

RefVFX performs **tuning-free visual effect transfer across videos**: given a reference effect,
the model transfers that effect onto new input content while preserving the underlying motion and
structure. The adapter was trained on the multi-part RefVFX dataset, which combines:

- **Code-based edits** — deterministic spatial effects (posterize, pixelate, glitch, glow, halftone, …)
  composited with temporal transitions (wipes, circle/diamond reveals, checkerboard, dissolves, …).
- **Neural V2V edits** — diffusion-generated effect videos sharing motion with a base (no-effect) video.
- **I2V LoRA effects** — image-to-video effects generated with LoRA adapters.

## Usage

This is a LoRA adapter and must be applied on top of the base Wan2.1-FLF2V-14B-720P weights.
See the [GitHub repo](https://github.com/maxwelljones14/refVFX) for inference scripts and the full
pipeline. At a high level:

```python
from huggingface_hub import hf_hub_download

lora_path = hf_hub_download(
    repo_id="maxwelljones14/refVFX-LoRA",
    filename="step-10000.safetensors",
)
# Load Wan2.1-FLF2V-14B-720P, then apply the LoRA weights from `lora_path`.
# Refer to https://github.com/maxwelljones14/refVFX for the exact loading code.
```

The checkpoint stores LoRA `A`/`B` matrices keyed as
`blocks.{i}.<module>.lora_A.default.weight` / `...lora_B.default.weight`.

## Citation

```bibtex
@article{jones2026tuning,
  title={Tuning-free Visual Effect Transfer across Videos},
  author={Jones, Maxwell and Abdal, Rameen and Patashnik, Or and Salakhutdinov, Ruslan and Tulyakov, Sergey and Zhu, Jun-Yan and Wang, Kuan-Chieh Jackson},
  journal={arXiv preprint arXiv:2601.07833},
  year={2026}
}
```

## License

Released under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/). The adapter derives from
[`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P); please also
review the base model's license terms before use.