Add model card

README.md

@@ -1,43 +1,97 @@
 ---
-license: other
-license_name: wan-community
-base_model: Wan-AI/Wan2.1-FLF2V-14B-720P
-library_name: diffusers
+license: cc-by-4.0
+base_model:
+  - Wan-AI/Wan2.1-FLF2V-14B-720P
+datasets:
+  - maxwelljones14/refVFX_dataset
 tags:
   - lora
   - video
+  - video-editing
+  - visual-effects
+  - vfx
+  - temporal-transitions
+  - wan2.1
   - image-to-video
-  - wan
-  - refvfx
+  - video-to-video
+pipeline_tag: image-to-video
+library_name: peft
+pretty_name: RefVFX LoRA (Wan2.1-FLF2V-14B-720P)
 ---
 
-# refVFX LoRA
+# RefVFX LoRA — Tuning-free Visual Effect Transfer across Videos
 
-LoRA adapter for [`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P) trained to transfer a temporal visual effect from a *reference* video onto a separate input image or video.
+This is a **LoRA adapter** for [`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P),
+trained for the **RefVFX** project on tuning-free visual effect transfer across videos.
 
-## Files
+> **Note:** This is an *unofficial* reimplementation produced at CMU. All code and training data were
+> created from scratch using the publicly available arXiv paper and AI coding tools as the only resources.
 
-| File | Description |
-| --- | --- |
-| `epoch-0.safetensors` | LoRA model. |
+## Links
 
-## Training
+- 📄 **Paper:** [Tuning-free Visual Effect Transfer across Videos](https://arxiv.org/abs/2601.07833) (arXiv:2601.07833)
+- 🌐 **Project page:** https://snap-research.github.io/RefVFX/
+- 💻 **Code (GitHub):** https://github.com/maxwelljones14/refVFX
+- 🗂️ **Training dataset:** [`maxwelljones14/refVFX_dataset`](https://huggingface.co/datasets/maxwelljones14/refVFX_dataset)
+- 🧱 **Base model:** [`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P)
 
-- **Base model:** `Wan-AI/Wan2.1-FLF2V-14B-720P`
+## Model Details
+
+- **Type:** LoRA adapter (fine-tune on top of Wan2.1-FLF2V-14B-720P)
+- **Base model:** `Wan-AI/Wan2.1-FLF2V-14B-720P` (first-last-frame-to-video, 14B parameters, 720P)
+- **Training data:** [`maxwelljones14/refVFX_dataset`](https://huggingface.co/datasets/maxwelljones14/refVFX_dataset)
+- **Training steps:** 40K steps
+- **Hardware:** 8× NVIDIA Blackwell GPUs
 - **LoRA rank:** 1024
-- **Target modules:** `q, k, v, o, ffn.0, ffn.2` (applied to the DiT)
-- **Learning rate:** 4e-5, 200-step linear warmup
-- **Frames per clip:** 33
-- **Max pixels:** 399,360
-- **Optimizer parallelism:** DeepSpeed ZeRO-1, 8 ranks
-- **CFG dropout:** `p_drop_ref = 0.05`, `p_drop_control_video = 0.05`
+- **Precision:** bf16
+- **Target modules:** all 40 transformer blocks — self-attention (`q`, `k`, `v`, `o`),
+  cross-attention (`q`, `k`, `v`, `o`), and feed-forward (`ffn.0`, `ffn.2`)
+- **Checkpoint file:** `step-10000.safetensors`
+
+## What it does
 
-Trained on [`maxwelljones14/refVFX_dataset`](https://huggingface.co/datasets/maxwelljones14/refVFX_dataset) (code-based edits + neural V2V edits + I2V LoRA effects, sampled as triplets).
+RefVFX performs **tuning-free visual effect transfer across videos**: given a reference effect,
+the model transfers that effect onto new input content while preserving the underlying motion and
+structure. The adapter was trained on the multi-part RefVFX dataset, which combines:
+
+- **Code-based edits** — deterministic spatial effects (posterize, pixelate, glitch, glow, halftone, …)
+  composited with temporal transitions (wipes, circle/diamond reveals, checkerboard, dissolves, …).
+- **Neural V2V edits** — diffusion-generated effect videos sharing motion with a base (no-effect) video.
+- **I2V LoRA effects** — image-to-video effects generated with LoRA adapters.
 
 ## Usage
 
-Load the weights into a Wan2.1-FLF2V pipeline and inject them as a LoRA on the DiT (target modules above, `remove_prefix_in_ckpt="pipe.dit."`). See `infer_refvfx.py` in the [refVFX trainer repo](https://github.com/) for a reference implementation.
+This is a LoRA adapter and must be applied on top of the base Wan2.1-FLF2V-14B-720P weights.
+See the [GitHub repo](https://github.com/maxwelljones14/refVFX) for inference scripts and the full
+pipeline. At a high level:
+
+```python
+from huggingface_hub import hf_hub_download
+
+lora_path = hf_hub_download(
+    repo_id="maxwelljones14/refVFX-LoRA",
+    filename="step-10000.safetensors",
+)
+# Load Wan2.1-FLF2V-14B-720P, then apply the LoRA weights from `lora_path`.
+# Refer to https://github.com/maxwelljones14/refVFX for the exact loading code.
+```
+
+The checkpoint stores LoRA `A`/`B` matrices keyed as
+`blocks.{i}.<module>.lora_A.default.weight` / `...lora_B.default.weight`.
+
+## Citation
+
+```bibtex
+@article{jones2026tuning,
+  title={Tuning-free Visual Effect Transfer across Videos},
+  author={Jones, Maxwell and Abdal, Rameen and Patashnik, Or and Salakhutdinov, Ruslan and Tulyakov, Sergey and Zhu, Jun-Yan and Wang, Kuan-Chieh Jackson},
+  journal={arXiv preprint arXiv:2601.07833},
+  year={2026}
+}
+```
 
 ## License
 
-Inherits the base-model license from `Wan-AI/Wan2.1-FLF2V-14B-720P`. Use is subject to its terms.
+Released under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/). The adapter derives from
+[`Wan-AI/Wan2.1-FLF2V-14B-720P`](https://huggingface.co/Wan-AI/Wan2.1-FLF2V-14B-720P); please also
+review the base model's license terms before use.