kandinskylab/Kandinsky-WM-1.0-I2V-5s-RO

Kandinsky WM 1.0: A family of models for Physical AI

Image-to-Video generation for Physical AI: autonomous driving · robotics · general physics

🤗 Checkpoints collection · 🧩 Base model (Kandinsky 5.0) · 💻 GitHub · 📝 Habr article

This repo holds the 🤖 Robotics checkpoint of the Kandinsky WM 1.0 family — DiT robot_rl, published here in two layouts: a diffusers pipeline (model_index.json, transformer/, vae/, text_encoder*/) and the GitHub-code DiT at model/robot_rl.safetensors (same weights). The other two variants are in the Kandinsky WM 1.0 collection.

---

Contents

• Overview
• Model Zoo
• Examples — 🚗 Autonomous Driving · 🤖 Robotics · 🌍 General Physics
• Results — PAI-Bench-G · RBench · Physics-IQ Verified
• Quickstart — Path A: GitHub code · Path B: Diffusers
• Acknowledgements

---

Overview

Kandinsky WM (World Model) 1.0 is a family of Image-to-Video models that adapt Kandinsky 5.0 Video Lite — a 2B-parameter latent video diffusion model (a Kandinsky5Transformer3DModel DiT paired with a HunyuanVideo VAE and Qwen2.5-VL + CLIP text encoders, trained with flow matching) — to Physical AI: video generation that is not only visually convincing but physically plausible — consistent scene geometry, object dynamics, interactions and cause-and-effect — so the model can serve as a source of synthetic training data and as a building block for world models and simulators.

The base model was domain-adapted on large corpora of 5-second scenes (first text-to-video, then a mixed text/image-to-video regime that preserves first-frame continuation), spanning autonomous driving, robotics, and a general domain (industrial processes, physical phenomena, and human–object / human–human interaction). Each checkpoint is then reinforcement-learning post-trained with GRPO against a reward model that scores the physical plausibility of the generated scene, steering the generator toward more faithful geometry, object behaviour and interactions than domain fine-tuning alone.

Each checkpoint generates 5-second, 121-frame clips at 768×512.

Model Zoo

Three domain checkpoints, same architecture — only the DiT weights differ (the VAE / text encoders are byte-identical across all three). Each is published as a one-line diffusers pipeline; a GitHub-code (DiT-only) copy lives in the same Hub repo used purely as file storage.

Domain	Checkpoint
🚗 Autonomous Driving	🤗 Kandinsky-WM-1.0-I2V-5s-AV
🤖 Robotics	🤗 Kandinsky-WM-1.0-I2V-5s-RO	← this repo
🌍 General Physics	🤗 Kandinsky-WM-1.0-I2V-5s-PH

Examples

Four Kandinsky WM 1.0 image-to-video generations per domain (top-ranked by an internal visual review). Each grid shows the first frame of a clip — click it to play the generated video (hosted on the static_videos dataset). The first frame and prompt for every clip also live in assets/: under assets/<domain>/, frame_N.jpg and prompt_N.txt correspond to grid cell N (left→right, top→bottom).

🚗 Autonomous Driving

First frames + prompts: assets/av/.

🤖 Robotics

First frames + prompts: assets/robotics/.

🌍 General Physics

First frames + prompts: assets/general/.

Results

Kandinsky WM 1.0 on three physical-AI video benchmarks (our row in bold). RBench and Physics-IQ were run with Qwen3-VL prompt enhancers — noted above each table, with the scripts under prompt_enhancers/. Sizes are total parameter counts where publicly disclosed (MoE models note active params; proprietary/undisclosed left as —).

PAI-Bench-G (Physical AI Bench — Generation)

Leaderboard. Column abbreviations follow the PAI-Bench-G dimensions — see the leaderboard for exact definitions. Run without prompt enhancement.

Rank	Model	Size	Overall	Domain	Quality	SC	BC	MS	AQ	IQ	OC	IS	IB	CS	AV	RO	IN	PH	HU
1	Cosmos3-Super	64B	83.9	89.5	78.2	92.7	94.1	99.2	52.7	70.8	20.5	97.7	98.1	94.4	77.5	90.0	90.7	95.0	87.6
2	Cosmos3-Nano	16B	83.7	89.4	78.1	92.3	93.8	99.2	52.7	70.1	20.4	97.9	98.3	95.0	75.4	90.2	89.7	94.5	88.0
3	Veo-3	—	82.1	86.7	77.6	91.4	93.1	99.2	51.9	69.8	21.7	97.0	96.9	94.4	68.7	86.9	89.7	91.6	84.4
4	Kandinsky WM 1.0	2B	81.7	86.0	77.4	91.8	94.1	99.0	53.2	64.9	21.6	97.2	97.7	93.6	72.6	82.1	88.0	90.7	86.0
5	k5 Lite FT	2B	81.4	85.8	77.1	91.3	93.9	98.7	52.7	64.3	21.7	96.6	97.3	92.9	72.6	82.1	87.7	88.8	86.4
6	Cosmos-Predict2.5-14B	14B	81.0	83.8	78.1	93.4	94.8	99.1	52.5	70.0	20.1	97.2	97.9	94.2	67.8	79.9	87.7	93.5	80.0
7	Cosmos-Predict2.5-2B	2B	81.0	84.0	77.9	92.5	94.2	99.1	52.4	70.8	20.1	96.6	97.4	94.1	66.1	80.8	87.8	93.9	81.4
8	Wan2.2-I2V-A14B	27B (14B active)	80.6	84.1	77.2	91.6	93.7	98.3	51.2	69.6	20.4	96.0	96.6	93.2	66.3	81.7	89.2	91.8	82.1
9	K5 Lite	2B	80.5	83.0	77.9	91.7	94.4	99.3	54.1	65.6	21.7	98.1	98.6	89.2	66.3	77.3	86.3	87.2	84.6
10	Wan2.2-TI2V-5B	5B	80.4	83.4	77.4	91.8	93.7	98.8	51.9	69.9	20.3	95.9	96.7	93.1	65.2	79.3	88.4	91.5	83.0

Physical-AI post-training lifts the base K5 Lite by +1.2 Overall (80.5 → 81.7) and +3.0 on the Domain axis (83.0 → 86.0) — a 2B model landing between Veo-3 and the Cosmos-Predict2.5 family.

RBench

Leaderboard, Qwen evaluator tab. RBench evaluates robot-oriented image-to-video generation across five task categories and four robot embodiments.

Prompt enhancer used: prompt_enhancers/rbench/enhance_rbench_prompts_qwen_v3.py — conservative Qwen3-VL image-grounded action canonicalization.

Rank	Model	Size	Avg.	Common Manipulation	Spatial Relationship	Multi-entity Collaboration	Long-horizon Planning	Visual Reasoning	Single Arm	Dual Arm	Quadruped Robot	Humanoid Robot
1	Veo 3	—	0.784	0.897	0.740	0.924	0.854	0.750	0.742	0.708	0.726	0.716
2	Wan 2.5	—	0.781	0.888	0.825	0.916	0.719	0.738	0.742	0.744	0.711	0.742
3	Hailuo v2	—	0.762	0.843	0.840	0.892	0.705	0.820	0.696	0.706	0.632	0.720
4	Seedance 1.0	—	0.755	0.856	0.665	0.899	0.716	0.790	0.708	0.730	0.674	0.759
5	Wan2.2_A14B	27B (14B active)	0.698	0.709	0.660	0.921	0.681	0.550	0.688	0.678	0.670	0.728
6	Kandinsky-WM-1.0	2B	0.651	0.836	0.685	0.846	0.657	0.524	0.509	0.568	0.578	0.656
7	Cosmos 2.5	14B	0.632	0.688	0.512	0.768	0.597	0.507	0.646	0.647	0.639	0.688
8	LongCat-Video	13.6B	0.609	0.678	0.465	0.814	0.490	0.354	0.698	0.602	0.666	0.710
9	DreamGen(gr1)	14B	0.575	0.507	0.500	0.848	0.353	0.405	0.660	0.632	0.611	0.656
10	Wan2.2_5B	5B	0.551	0.598	0.402	0.722	0.450	0.420	0.511	0.534	0.638	0.682
11	Wan2.1_14B	14B	0.542	0.688	0.400	0.702	0.465	0.270	0.519	0.562	0.604	0.664
12	SkyReels	13B	0.531	0.546	0.400	0.687	0.324	0.358	0.612	0.574	0.654	0.628
13	DreamGen(droid)	14B	0.514	0.465	0.505	0.591	0.302	0.386	0.589	0.570	0.584	0.633
14	LTX-Video	2B	0.486	0.450	0.382	0.734	0.358	0.286	0.487	0.487	0.588	0.603
15	FramePack	13B	0.453	0.455	0.240	0.630	0.197	0.345	0.403	0.500	0.670	0.635
16	CogVideoX_5B	5B	0.322	0.290	0.240	0.426	0.096	0.030	0.374	0.422	0.494	0.524
17	Vidar	—	0.207	0.118	0.140	0.082	0.019	0.030	0.344	0.390	0.380	0.364
18	UnifoLM-WMA-0	—	0.104	0.029	0.065	0.025	0.000	0.000	0.290	0.106	0.251	0.170

Physics-IQ Verified

Physics-IQ Verified

Prompt enhancer used: prompt_enhancers/physics_iq/temporal_expansion.py — Qwen3-VL temporal caption expansion.

#	Model	Size	Input type	Score	Date added
1	Magi-1 24B + GeoPhys (BoN) (op)	24B	multiframe (v2v)	58.2 ± 1.8	2026-06-19
2	Magi-1 24B (op)	24B	multiframe (v2v)	48.4 ± 1.1	2026-06-19
3	Cosmos3-Super-Image2Video	64B	i2v	39.5 ± 0.8	2026-06-18
4	Grok Imagine Video	—	i2v	34.8 ± 0.6	2026-06-17
5	Magi-1 24B + GeoPhys (BoN) (op)	24B	i2v	33.7 ± 1.4	2026-06-19
6	Hunyuan Video 1.5	8.3B	i2v	33.4 ± 0.8	2026-06-17
7	Wan 2.2	27B (14B active)	i2v	32.2 ± 0.6	2026-06-17
8	Kandinsky WM 1.0	2B	i2v	30.8 ± 0.5	internal
9	Cosmos3-Nano	16B	i2v	30.3 ± 0.6	2026-06-18
10	Magi-1 24B (op)	24B	i2v	30.2 ± 1.1	2026-06-19
11	Sora 2	—	i2v	26.5 ± 0.8	2026-06-17
12	K5 Video Lite FT	2B	i2v	25.8 ± 1.7	internal
13	P-Video	—	i2v	25.3 ± 1.8	2026-06-17
14	K5 Video Lite	2B	i2v	16.0 ± 1.2	internal

Quickstart

Two ways to run — the checkpoints ship in both formats. The GitHub-code path is primary; the diffusers path is a one-liner that already works off the Hub.

Path A — GitHub code

The base inference code is vendored as a git submodule at kandinsky-5/ (from kandinskylab/kandinsky-5).

# 0. get this repo with the submodule
git clone --recurse-submodules https://github.com/kandinskylab/kandinsky-wm.git
cd kandinsky-wm
# (already cloned without it? run: git submodule update --init --recursive)

# 1. install the base code
pip install -r kandinsky-5/requirements.txt

# 2. pull a checkpoint (repo-as-file-storage; DiT + shared VAE/text encoders)
python examples/download_checkpoint.py --domain robotics

# 3. generate (CLI) — run from inside the submodule
cd kandinsky-5
python test.py \
  --config ./configs/k5_lite_i2v_5s_sft_sd.yaml \
  --image  ../assets/robotics/frame_1.jpg \
  --prompt "$(cat ../assets/robotics/prompt_1.txt)" \
  --video_duration 5

Python variant: examples/run_i2v_github.py.

Path B — Diffusers

pip install -U diffusers transformers accelerate imageio-ffmpeg

import torch
from diffusers import Kandinsky5I2VPipeline
from diffusers.utils import export_to_video, load_image

pipe = Kandinsky5I2VPipeline.from_pretrained(
    "kandinskylab/Kandinsky-WM-1.0-I2V-5s-RO",
    torch_dtype=torch.bfloat16,
).to("cuda")

frames = pipe(
    image=load_image("assets/robotics/frame_1.jpg"),
    prompt=open("assets/robotics/prompt_1.txt").read().strip(),
    negative_prompt=("Static, 2D cartoon, cartoon, 2d animation, paintings, images, "
                     "worst quality, low quality, ugly, deformed, walking backwards"),
    height=512, width=768, num_frames=121,
    num_inference_steps=50, guidance_scale=5.0,
).frames[0]
export_to_video(frames, "robotics_out.mp4", fps=24, quality=9)

Script: examples/run_i2v_diffusers.py.

Acknowledgements

Built on Kandinsky 5.0 (HunyuanVideo VAE, Qwen2.5-VL, CLIP). Evaluated on PAI-Bench-G, Physics-IQ, and RBench.