Instructions to use kandinskylab/Kandinsky-WM-1.0-I2V-5s-RO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use kandinskylab/Kandinsky-WM-1.0-I2V-5s-RO with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("kandinskylab/Kandinsky-WM-1.0-I2V-5s-RO", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Commit Β·
e9346ed
1
Parent(s): 7989d31
docs: add RBench results and prompt enhancer
Browse files
README.md
CHANGED
|
@@ -29,7 +29,7 @@ tags: [kandinsky, image-to-video, physical-ai, world-model]
|
|
| 29 |
- [Overview](#overview)
|
| 30 |
- [Model Zoo](#model-zoo)
|
| 31 |
- [Examples](#examples) β [π Autonomous Driving](#π-autonomous-driving) Β· [π€ Robotics](#π€-robotics) Β· [π General Physics](#π-general-physics)
|
| 32 |
-
- [Results](#results) β [PAI-Bench-G](#pai-bench-g-physical-ai-bench--generation) Β· [Physics-IQ Verified](#physics-iq-verified)
|
| 33 |
- [Quickstart](#quickstart) β [Path A: GitHub code](#path-a--github-code) Β· [Path B: Diffusers](#path-b--diffusers)
|
| 34 |
- [Acknowledgements](#acknowledgements)
|
| 35 |
|
|
@@ -113,9 +113,9 @@ First frames + prompts: [`assets/general/`](https://github.com/kandinskylab/kand
|
|
| 113 |
|
| 114 |
## Results
|
| 115 |
|
| 116 |
-
**Kandinsky WM 1.0** on
|
| 117 |
-
|
| 118 |
-
|
| 119 |
counts where publicly disclosed (MoE models note active params; proprietary/undisclosed left as β).
|
| 120 |
|
| 121 |
### PAI-Bench-G (Physical AI Bench β Generation)
|
|
@@ -141,6 +141,36 @@ Physical-AI post-training lifts the base **K5 Lite** by **+1.2 Overall** (80.5
|
|
| 141 |
**+3.0 on the Domain axis** (83.0 β 86.0) β a 2B model landing between Veo-3 and the
|
| 142 |
Cosmos-Predict2.5 family.
|
| 143 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 144 |
### Physics-IQ Verified
|
| 145 |
|
| 146 |
> **Prompt enhancer used:** [`prompt_enhancers/physics_iq/temporal_expansion.py`](https://github.com/kandinskylab/kandinsky-wm/blob/main/prompt_enhancers/physics_iq/temporal_expansion.py) β Qwen3-VL temporal caption expansion.
|
|
@@ -231,5 +261,6 @@ Built on [Kandinsky 5.0](https://github.com/kandinskylab/kandinsky-5)
|
|
| 231 |
([HunyuanVideo VAE](https://huggingface.co/hunyuanvideo-community/HunyuanVideo),
|
| 232 |
[Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct),
|
| 233 |
[CLIP](https://huggingface.co/openai/clip-vit-large-patch14)). Evaluated on
|
| 234 |
-
[PAI-Bench-G](https://huggingface.co/spaces/shi-labs/physical-ai-bench-leaderboard)
|
| 235 |
-
[Physics-IQ](https://github.com/google-deepmind/physics-IQ-benchmark)
|
|
|
|
|
|
| 29 |
- [Overview](#overview)
|
| 30 |
- [Model Zoo](#model-zoo)
|
| 31 |
- [Examples](#examples) β [π Autonomous Driving](#π-autonomous-driving) Β· [π€ Robotics](#π€-robotics) Β· [π General Physics](#π-general-physics)
|
| 32 |
+
- [Results](#results) β [PAI-Bench-G](#pai-bench-g-physical-ai-bench--generation) Β· [RBench](#rbench) Β· [Physics-IQ Verified](#physics-iq-verified)
|
| 33 |
- [Quickstart](#quickstart) β [Path A: GitHub code](#path-a--github-code) Β· [Path B: Diffusers](#path-b--diffusers)
|
| 34 |
- [Acknowledgements](#acknowledgements)
|
| 35 |
|
|
|
|
| 113 |
|
| 114 |
## Results
|
| 115 |
|
| 116 |
+
**Kandinsky WM 1.0** on three physical-AI video benchmarks (our row in **bold**). RBench and
|
| 117 |
+
Physics-IQ were run with Qwen3-VL prompt enhancers β noted above each table, with the scripts under
|
| 118 |
+
[`prompt_enhancers/`](https://github.com/kandinskylab/kandinsky-wm/tree/main/prompt_enhancers). Sizes are total parameter
|
| 119 |
counts where publicly disclosed (MoE models note active params; proprietary/undisclosed left as β).
|
| 120 |
|
| 121 |
### PAI-Bench-G (Physical AI Bench β Generation)
|
|
|
|
| 141 |
**+3.0 on the Domain axis** (83.0 β 86.0) β a 2B model landing between Veo-3 and the
|
| 142 |
Cosmos-Predict2.5 family.
|
| 143 |
|
| 144 |
+
### RBench
|
| 145 |
+
|
| 146 |
+
[Leaderboard](https://huggingface.co/spaces/DAGroup-PKU/RBench-Leaderboard), **Qwen** evaluator tab.
|
| 147 |
+
RBench evaluates robot-oriented image-to-video generation across five task categories and four
|
| 148 |
+
robot embodiments.
|
| 149 |
+
|
| 150 |
+
> **Prompt enhancer used:** [`prompt_enhancers/rbench/enhance_rbench_prompts_qwen_v3.py`](https://github.com/kandinskylab/kandinsky-wm/blob/main/prompt_enhancers/rbench/enhance_rbench_prompts_qwen_v3.py) β conservative Qwen3-VL image-grounded action canonicalization.
|
| 151 |
+
|
| 152 |
+
| Rank | Model | Size | Avg. | Common Manipulation | Spatial Relationship | Multi-entity Collaboration | Long-horizon Planning | Visual Reasoning | Single Arm | Dual Arm | Quadruped Robot | Humanoid Robot |
|
| 153 |
+
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| 154 |
+
| 1 | Veo 3 | β | 0.784 | 0.897 | 0.740 | 0.924 | 0.854 | 0.750 | 0.742 | 0.708 | 0.726 | 0.716 |
|
| 155 |
+
| 2 | Wan 2.5 | β | 0.781 | 0.888 | 0.825 | 0.916 | 0.719 | 0.738 | 0.742 | 0.744 | 0.711 | 0.742 |
|
| 156 |
+
| 3 | Hailuo v2 | β | 0.762 | 0.843 | 0.840 | 0.892 | 0.705 | 0.820 | 0.696 | 0.706 | 0.632 | 0.720 |
|
| 157 |
+
| 4 | Seedance 1.0 | β | 0.755 | 0.856 | 0.665 | 0.899 | 0.716 | 0.790 | 0.708 | 0.730 | 0.674 | 0.759 |
|
| 158 |
+
| 5 | Wan2.2_A14B | 27B (14B active) | 0.698 | 0.709 | 0.660 | 0.921 | 0.681 | 0.550 | 0.688 | 0.678 | 0.670 | 0.728 |
|
| 159 |
+
| 6 | **Kandinsky-WM-1.0** | **2B** | **0.651** | **0.836** | **0.685** | **0.846** | **0.657** | **0.524** | **0.509** | **0.568** | **0.578** | **0.656** |
|
| 160 |
+
| 7 | Cosmos 2.5 | 14B | 0.632 | 0.688 | 0.512 | 0.768 | 0.597 | 0.507 | 0.646 | 0.647 | 0.639 | 0.688 |
|
| 161 |
+
| 8 | LongCat-Video | 13.6B | 0.609 | 0.678 | 0.465 | 0.814 | 0.490 | 0.354 | 0.698 | 0.602 | 0.666 | 0.710 |
|
| 162 |
+
| 9 | DreamGen(gr1) | 14B | 0.575 | 0.507 | 0.500 | 0.848 | 0.353 | 0.405 | 0.660 | 0.632 | 0.611 | 0.656 |
|
| 163 |
+
| 10 | Wan2.2_5B | 5B | 0.551 | 0.598 | 0.402 | 0.722 | 0.450 | 0.420 | 0.511 | 0.534 | 0.638 | 0.682 |
|
| 164 |
+
| 11 | Wan2.1_14B | 14B | 0.542 | 0.688 | 0.400 | 0.702 | 0.465 | 0.270 | 0.519 | 0.562 | 0.604 | 0.664 |
|
| 165 |
+
| 12 | SkyReels | 13B | 0.531 | 0.546 | 0.400 | 0.687 | 0.324 | 0.358 | 0.612 | 0.574 | 0.654 | 0.628 |
|
| 166 |
+
| 13 | DreamGen(droid) | 14B | 0.514 | 0.465 | 0.505 | 0.591 | 0.302 | 0.386 | 0.589 | 0.570 | 0.584 | 0.633 |
|
| 167 |
+
| 14 | LTX-Video | 2B | 0.486 | 0.450 | 0.382 | 0.734 | 0.358 | 0.286 | 0.487 | 0.487 | 0.588 | 0.603 |
|
| 168 |
+
| 15 | FramePack | 13B | 0.453 | 0.455 | 0.240 | 0.630 | 0.197 | 0.345 | 0.403 | 0.500 | 0.670 | 0.635 |
|
| 169 |
+
| 16 | CogVideoX_5B | 5B | 0.322 | 0.290 | 0.240 | 0.426 | 0.096 | 0.030 | 0.374 | 0.422 | 0.494 | 0.524 |
|
| 170 |
+
| 17 | Vidar | β | 0.207 | 0.118 | 0.140 | 0.082 | 0.019 | 0.030 | 0.344 | 0.390 | 0.380 | 0.364 |
|
| 171 |
+
| 18 | UnifoLM-WMA-0 | β | 0.104 | 0.029 | 0.065 | 0.025 | 0.000 | 0.000 | 0.290 | 0.106 | 0.251 | 0.170 |
|
| 172 |
+
|
| 173 |
+
### Physics-IQ Verified
|
| 174 |
### Physics-IQ Verified
|
| 175 |
|
| 176 |
> **Prompt enhancer used:** [`prompt_enhancers/physics_iq/temporal_expansion.py`](https://github.com/kandinskylab/kandinsky-wm/blob/main/prompt_enhancers/physics_iq/temporal_expansion.py) β Qwen3-VL temporal caption expansion.
|
|
|
|
| 261 |
([HunyuanVideo VAE](https://huggingface.co/hunyuanvideo-community/HunyuanVideo),
|
| 262 |
[Qwen2.5-VL](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct),
|
| 263 |
[CLIP](https://huggingface.co/openai/clip-vit-large-patch14)). Evaluated on
|
| 264 |
+
[PAI-Bench-G](https://huggingface.co/spaces/shi-labs/physical-ai-bench-leaderboard),
|
| 265 |
+
[Physics-IQ](https://github.com/google-deepmind/physics-IQ-benchmark), and
|
| 266 |
+
[RBench](https://huggingface.co/spaces/DAGroup-PKU/RBench-Leaderboard).
|