SyncWorld zero-shot robot simulator (ZeroGPU)

.gitattributes

@@ -33,3 +33,22 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-black-bowl/calibration/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-black-bowl/expert/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-bowl-plate/calibration/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-drawer/calibration/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-drawer/expert/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-soup/calibration/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-soup/expert/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-stove/calibration/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/libero-stove/expert/agentview_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/peginsertion/calibration/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/peginsertion/expert/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/plugcharger/calibration/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/plugcharger/expert/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/pushcube/calibration/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/pushcube/expert/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/stackcube/calibration/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/stackcube/expert/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/stackpyramid/calibration/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/scenes/stackpyramid/expert/render_camera_rgb/video.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,13 +1,79 @@
 ---
-title: Syncworld Simulator
-emoji: ⚡
-colorFrom: red
-colorTo: blue
+title: SyncWorld Simulator
+emoji: 🦾
+colorFrom: pink
+colorTo: red
 sdk: gradio
 sdk_version: 6.26.0
-python_version: '3.12'
 app_file: app.py
-pinned: false
+short_description: Visually-calibrated zero-shot robot world simulator
+python_version: "3.12"
+startup_duration_timeout: 1h
+models:
+  - yyuncong/SyncWorld
+datasets:
+  - yyuncong/SyncWorld-Evaluation
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# 🦾 SyncWorld — visual calibration turns a world model into a zero-shot simulator
+
+Demo for **[SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators](https://huggingface.co/papers/2609.09155)**
+(UMass Embodied AGI) — [code](https://github.com/UMass-Embodied-AGI/SyncWorld) ·
+[weights](https://huggingface.co/yyuncong/SyncWorld).
+
+A pretrained action-conditioned video world model cannot be dropped into a new scene, because it
+does not know how *this* camera sees *this* robot's action space. SyncWorld's fix is **visual
+calibration**: prepend a short clip of the arm sweeping each degree of freedom, paired with the
+actions that produced it, as extra fully-conditioning context. The model infers the unseen
+camera↔robot mapping in-context and becomes a usable forward-dynamics simulator — with no
+per-scene finetuning.
+
+## What this Space does
+
+Pick one of the authors' calibrated evaluation episodes, then either
+
+- **replay the robot's own actions** — re-simulate the expert trajectory and compare against ground
+  truth, or
+- **drive the arm yourself** — command a constant per-step end-effector twist
+  (Δx/Δy/Δz in cm/step, Δroll/Δpitch/Δyaw in °/step, plus the gripper) and watch the model render
+  a counterfactual rollout.
+
+Each press generates one or more autoregressive 16-frame windows at 512 px, closed-loop (each
+window conditions on the previously *generated* frames).
+
+## Inference path
+
+Ported 1:1 from the authors' reference script `examples/eval_gripperhead_fdm_rollout.py`, keeping
+the trained recipe exactly:
+
+| | |
+|---|---|
+| Conditioning items | `[6 per-DoF calibration segments × 5 frames] + [25 sparse history frames @ stride 3] + [current frame + 16 future frames]` |
+| Actions | 7-D `[Δpos_cm(3), Δeuler_xyz_deg(3), gripper(1)]`, `backward_framewise` convention, translation scale 100.0, rotation scale 57.2958 |
+| Resolution / fps | 512 px square, conditioning fps 15 |
+| Sampler | rectified flow, UniPC, 20 steps (adjustable) |
+| Caption | the FDM training constant `"A robot arm interacts with the scene."` |
+| Backbone | 16B Cosmos-3 mixture-of-transformers (Qwen3-VL-8B understanding expert + diffusion expert), Wan2.2 4×16×16 video VAE |
+
+Deviations forced by the target hardware: single-process inference (`parallel_dims=None`),
+`torch.compile` disabled (ZeroGPU forks a fresh worker per call), the audio expert disabled
+(unused by this checkpoint), and the VLM processor served from a local copy of the seven tokenizer
+files in `nvidia/Cosmos3-Nano` instead of downloading that repo's 35 GB of weights.
+
+## Bundled scenes
+
+Ten episodes (5 ManiSkill + 5 LIBERO) copied verbatim from the authors' own evaluation set,
+[`yyuncong/SyncWorld-Evaluation`](https://huggingface.co/datasets/yyuncong/SyncWorld-Evaluation),
+each with its paired calibration sweep and pose track. Both the code and the data are released
+under **OpenMDW-1.1**; see `ATTRIBUTIONS.md` / `NOTICE` in the upstream repo.
+
+## Citation
+
+```bibtex
+@article{syncworld2026,
+  title  = {SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators},
+  author = {UMass Embodied AGI},
+  year   = {2026},
+  url    = {https://huggingface.co/papers/2609.09155}
+}
+```

app.py

@@ -0,0 +1,656 @@
+"""SyncWorld — visual calibration turns a video world model into a zero-shot robot simulator.
+
+Gradio / ZeroGPU demo for `yyuncong/SyncWorld` (paper 2609.09155, UMass Embodied AGI).
+
+The inference path is ported 1:1 from the authors' reference script
+`examples/eval_gripperhead_fdm_rollout.py` in
+https://github.com/UMass-Embodied-AGI/SyncWorld — same config surgery, same
+multi-item `[calib x 6] + [history] + [current+future]` batch, same sampler
+call, same conditioning horizon (25 sparse history frames @ stride 3, 16
+predicted frames, 512px, conditioning fps 15, 7-D `[dpos_cm, deuler_deg,
+gripper]` actions in the `backward_framewise` convention).
+"""
+import os
+import sys
+
+os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
+os.environ.setdefault("COSMOS_DEVICE", "cuda")
+
+import spaces  # noqa: F401  # MUST precede torch / any CUDA-touching import
+
+from cosmos_framework.inference.common.init import init_script
+
+init_script()  # sets grad off, seeds, wires the cosmos logger (matches the reference script)
+sys.excepthook = sys.__excepthook__  # undo init_script's distributed excepthook (Gradio needs the default)
+
+import functools
+import json
+import math
+import pickle
+import shutil
+import tempfile
+import time
+import types
+from pathlib import Path
+
+import gradio as gr
+import imageio
+import numpy as np
+import torch
+import torch.nn.functional as F
+from huggingface_hub import hf_hub_download, snapshot_download
+
+from cosmos_framework.configs.base.defaults.compile import CompileConfig
+from cosmos_framework.data.vfm.action.calib_segments import build_calib_segment_indices
+from cosmos_framework.data.vfm.action.datasets.gripperhead_fdm_dataset import NEUTRAL_CAPTION_SEED
+from cosmos_framework.data.vfm.action.domain_utils import get_domain_id
+from cosmos_framework.data.vfm.action.pose_utils import pose_abs_to_rel
+from cosmos_framework.data.vfm.action.transforms import build_sequence_plan_from_mode
+from cosmos_framework.inference.model import Cosmos3OmniConfig, Cosmos3OmniModel
+from cosmos_framework.model.vfm.vlm.qwen3_vl.utils import tokenize_caption
+
+HERE = Path(__file__).parent
+DEVICE = "cuda"
+OUT_DIR = Path(tempfile.gettempdir()) / "syncworld_out"
+OUT_DIR.mkdir(parents=True, exist_ok=True)
+
+CKPT_REPO = "yyuncong/SyncWorld"
+VAE_REPO = "Wan-AI/Wan2.2-TI2V-5B"
+VAE_FILE = "Wan2.2_VAE.pth"
+QWEN_REPO = "nvidia/Cosmos3-Nano"
+# Only the processor / tokenizer artifacts at the repo root — NOT the 35GB of weights.
+QWEN_FILES = [
+    "chat_template.json",
+    "merges.txt",
+    "preprocessor_config.json",
+    "tokenizer.json",
+    "tokenizer_config.json",
+    "video_preprocessor_config.json",
+    "vocab.json",
+]
+
+# ---------------------------------------------------------------- shipped recipe
+# These are the trained checkpoint's conditioning + sampling defaults, i.e. the
+# argparse defaults of the authors' eval script. They MUST match the checkpoint.
+def _recipe(**over):
+    a = types.SimpleNamespace(
+        num_history_frames=25,
+        num_pred_frames=16,
+        history_frame_stride=3,
+        resolution=512,
+        fps=15.0,
+        use_calibration=True,
+        calib_null=False,
+        calib_segments=6,
+        calib_seg_len=5,
+        calib_frame_interval=3,
+        calib_positive_actions=False,
+        action_convention="backward_framewise",
+        action_rot_format="euler_xyz",
+        action_trans_scale=100.0,
+        action_rot_scale=57.2958,
+        action_cfg_scale=1.0,
+        num_steps=20,
+        seed=0,
+    )
+    for k, v in over.items():
+        setattr(a, k, v)
+    return a
+
+
+# ================================================================ model loading
+# (ported from the reference script: _REWRITES / _config_from_dir /
+#  _patch_offline_vlm_processor / load_model)
+_REWRITES = [
+    ("cosmos3._src.vfm.configs.base.", "cosmos_framework.configs.base."),
+    ("cosmos3._src.vfm.models.", "cosmos_framework.model.vfm."),
+    ("cosmos3._src.vfm.tokenizers.", "cosmos_framework.model.vfm.tokenizers."),
+    ("cosmos3._src.imaginaire.", "cosmos_framework."),
+]
+
+
+def _config_from_dir(d: Path, vae_path: str = "") -> Cosmos3OmniConfig:
+    cfg = d / "config.json" if (d / "config.json").exists() else d / "model" / "config.json"
+    if not cfg.exists():
+        raise FileNotFoundError(f"no config.json under {d}")
+    text = cfg.read_text()
+    for a, b in _REWRITES:
+        text = text.replace(a, b)
+    model_cfg = json.loads(text)["model"]
+    # The shipped config resolves the Wan2.2 video VAE through an object store
+    # (vae_path="pretrained/..." + bucket_name="bucket"). Point it at the local
+    # .pth and clear the bucket so loading is self-contained.
+    wan = (vae_path or "").strip()
+    if wan:
+        def _override_vae(o):
+            if isinstance(o, dict):
+                if "vae_path" in o:  # the VIDEO tokenizer dict ("avae_path" is the audio one)
+                    o["vae_path"] = wan
+                    o["bucket_name"] = ""
+                for v in o.values():
+                    _override_vae(v)
+            elif isinstance(o, list):
+                for v in o:
+                    _override_vae(v)
+        _override_vae(model_cfg)
+    # Sound is DISABLED in the gripperhead FDM recipe, so the checkpoint has no
+    # sound expert; leaving sound_gen=True would make the build fetch an audio
+    # tokenizer that does not exist for this release.
+    if isinstance(model_cfg.get("config"), dict):
+        model_cfg["config"]["sound_gen"] = False
+        model_cfg["config"]["sound_tokenizer"] = None
+    return Cosmos3OmniConfig(model=model_cfg)
+
+
+def _patch_offline_vlm_processor(qwen_assets: str):
+    """Serve the VLM processor from a local dir instead of shelling out to `uvx hf download`
+    (which would pull the whole 35GB nvidia/Cosmos3-Nano repo)."""
+    import cosmos_framework.utils.checkpoint_db as _ckdb
+
+    _orig = _ckdb._hf_download
+
+    def _patched(cmd_args):
+        repo = str(cmd_args[0]) if cmd_args else ""
+        if "Cosmos3-Nano" in repo:
+            print(f"[boot] offline VLM processor: _hf_download({repo}) -> {qwen_assets}", flush=True)
+            return qwen_assets
+        return _orig(cmd_args)
+
+    _ckdb._hf_download = _patched
+
+
+def _load_model(checkpoint: str, vae_path: str, qwen_assets: str):
+    _patch_offline_vlm_processor(qwen_assets)
+    ckpt = Path(checkpoint)
+    wrapper = Cosmos3OmniModel.from_pretrained_dcp(
+        ckpt,
+        config=_config_from_dir(ckpt, vae_path),
+        compile_config=CompileConfig(enabled=False),  # torch.compile is off: ZeroGPU forks a fresh worker
+    )
+    m = wrapper.model
+    # Single-process inference: null parallel_dims so every collective site takes
+    # its local no-op branch (the reference does the same for single-GPU eval).
+    if getattr(m, "parallel_dims", None) is not None:
+        m.parallel_dims = None
+    m.eval()
+    return m
+
+
+# ================================================================ episode reading
+# (ported: _read_video / _center_square_crop_np / _load_pose / _build_action /
+#  read_episode / _to_u8 / build_fdm_batch_multiitem / build_calib_eval_items /
+#  _gen_window)
+def _read_video(path: str) -> np.ndarray:
+    import imageio.v3 as iio
+
+    try:
+        return iio.imread(path, plugin="pyav")  # (T,H,W,C) uint8
+    except Exception:
+        return iio.imread(path)
+
+
+@functools.lru_cache(maxsize=4)
+def _read_video_cached(path: str) -> np.ndarray:
+    return _read_video(path)
+
+
+def _center_square_crop_np(a: np.ndarray) -> np.ndarray:
+    h, w = a.shape[1], a.shape[2]
+    if h == w:
+        return a
+    m = min(h, w)
+    top, left = (h - m) // 2, (w - m) // 2
+    return a[:, top:top + m, left:left + m]
+
+
+def _load_pose(leaf: str, thr: float = 0.6):
+    with open(os.path.join(leaf, "pose.pkl"), "rb") as f:
+        d = pickle.load(f)
+    mats = np.asarray(d["gripper_matrix"], dtype=np.float32)  # (T,4,4)
+    gopen = np.asarray(d.get("gripper_open", np.ones(len(mats))), dtype=np.float32)
+    return mats, (gopen > thr).astype(np.float32)
+
+
+def _build_action(mats, gopen, convention="backward_framewise", rot_format="euler_xyz",
+                  trans_scale=100.0, rot_scale=57.2958) -> torch.Tensor:
+    """abs poses -> per-step delta actions [dpos_cm(3), deuler_deg(3), gripper(1)]."""
+    poses_rel = pose_abs_to_rel(mats, rotation_format=rot_format, pose_convention=convention,
+                               translation_scale=trans_scale, rotation_scale=rot_scale)
+    grip = gopen[1:].reshape(-1, 1)
+    return torch.from_numpy(np.concatenate([poses_rel, grip], axis=-1).astype(np.float32))
+
+
+def _frames_to_pm1(frames: np.ndarray, res: int) -> torch.Tensor:
+    v = torch.from_numpy(_center_square_crop_np(frames)).float().permute(0, 3, 1, 2) / 255.0
+    _, _, hh, ww = v.shape
+    th = res
+    tw = max(16, int(round((res * ww / hh) / 16) * 16))  # aspect preserving, /16 for the VAE
+    if (hh, ww) != (th, tw):
+        v = F.interpolate(v, size=(th, tw), mode="bilinear", align_corners=False)
+    return v * 2 - 1
+
+
+def read_episode(leaf: str, view: str, res: int):
+    frames = _read_video_cached(os.path.join(leaf, view, "video.mp4"))
+    v = _frames_to_pm1(frames, res)
+    mats, gopen = _load_pose(leaf)
+    n = min(len(v), len(mats))
+    # FDM training always emits NEUTRAL_CAPTION_SEED, so anything else is out of distribution.
+    return v[:n], mats[:n], gopen[:n], NEUTRAL_CAPTION_SEED
+
+
+def _to_u8(x: torch.Tensor) -> torch.Tensor:
+    return ((x.clamp(-1, 1) + 1) / 2 * 255.0).round().clamp(0, 255).to(torch.uint8)
+
+
+def build_fdm_batch_multiitem(model, video_u8_list, action_list, caption, device, fps=15.0,
+                              mode: str = "forward_dynamics"):
+    """N-item FDM batch: `[calib x K] + [history] + [current+future]`. Every item but the
+    last is fully conditioning; the last conditions on its current-frame latent and
+    generates the P future latents."""
+    maxD = model.config.max_action_dim
+
+    def pad(a):
+        p = torch.zeros(a.shape[0], maxD, device=device)
+        p[:, : a.shape[1]] = a.to(device)
+        return p
+
+    vids = [v.to(device) for v in video_u8_list]
+    acts = [pad(a) for a in action_list]
+    dims = [torch.tensor(int(a.shape[1]), dtype=torch.long, device=device) for a in action_list]
+    sizes = [torch.tensor([[v.shape[-2], v.shape[-1], v.shape[-2], v.shape[-1]]],
+                          dtype=torch.float32, device=device) for v in vids]
+    cf = vids[-1]
+    sp = build_sequence_plan_from_mode(mode, video_length=cf.shape[1],
+                                       action_length=action_list[-1].shape[0], has_text=True,
+                                       num_condition_latent_frames=1)
+    sp.share_vision_temporal_positions = False  # distinct time states per item — MUST match training
+    ids = tokenize_caption(caption, model.vlm_tokenizer, is_video=False,
+                           use_system_prompt=model.vlm_config.use_system_prompt)
+    return {
+        model.input_video_key: [vids],
+        "action": [acts],
+        "raw_action_dim": [dims],
+        "image_size": [sizes],
+        "mode": [mode],
+        model.input_caption_key: [caption],
+        "text_token_ids": [torch.tensor(ids, dtype=torch.long, device=device).unsqueeze(0)],
+        "fps": torch.tensor([float(fps)], device=device),
+        "conditioning_fps": torch.tensor([float(fps)], device=device),
+        "domain_id": [torch.tensor(get_domain_id("gripperhead"), dtype=torch.long, device=device)],
+        "sequence_plan": [sp],
+        # NO is_preprocessed -> the uint8 items get normalized + re-stacked for the VAE
+    }
+
+
+def build_calib_items(calib_dir: str, args, view: str):
+    """Load + per-DoF segment the calibration sweep into (K uint8 video items, K action blocks)."""
+    frames = _read_video_cached(os.path.join(calib_dir, view, "video.mp4"))
+    v = _frames_to_pm1(frames, args.resolution)
+    mats, gopen = _load_pose(calib_dir)
+    n = min(len(mats), len(v))
+    move_order = None
+    mrp = os.path.join(calib_dir, "move_range.pkl")
+    if os.path.isfile(mrp):
+        try:
+            with open(mrp, "rb") as f:
+                move_order = pickle.load(f).get("movement_order")
+        except Exception:
+            move_order = None
+    sub = list(range(0, n, args.calib_frame_interval)) or [0]
+    efficient = (args.calib_segments == 6)  # 6 -> one segment per DoF; 12 -> both signs
+    seg_lists = build_calib_segment_indices(mats[sub], args.calib_seg_len, efficient, move_order,
+                                            positive_body_actions=bool(args.calib_positive_actions))
+    vids, acts = [], []
+    for seg in seg_lists:
+        real = [sub[i] for i in seg]
+        vids.append(_to_u8(v[real].permute(1, 0, 2, 3)))  # (C,seg_len,H,W) uint8
+        acts.append(_build_action(mats[real], gopen[real], args.action_convention,
+                                  args.action_rot_format, args.action_trans_scale, args.action_rot_scale))
+    return vids, acts
+
+
+def _gen_window(model, ep_frames, gen_frames, mats, gopen, caption, calib_v, calib_a,
+                start, args, device, custom_act=None):
+    """Generate ONE 1+P frame window at frame `start`. Returns (pred_u8, gt_u8) as (P+1,H,W,C)."""
+    T = ep_frames.shape[0]
+    H, P, S = args.num_history_frames, args.num_pred_frames, args.history_frame_stride
+    clamp = lambda x: max(0, min(int(x), T - 1))  # noqa: E731
+    use_history, use_calib = H > 1, args.use_calibration
+    cf_idx = [clamp(start)] + [clamp(start + 1 + j) for j in range(P)]
+    cf_frames = torch.stack([gen_frames[clamp(start)]]
+                            + [ep_frames[clamp(start + 1 + j)] for j in range(P)], dim=1)
+    # custom-action mode: drive the window with a synthesized cm/deg action. The HISTORY
+    # action block stays real — it is observed context, not the commanded motion.
+    cf_act = (custom_act.to(torch.float32) if custom_act is not None else
+              _build_action(mats[cf_idx], gopen[cf_idx], args.action_convention,
+                            args.action_rot_format, args.action_trans_scale, args.action_rot_scale))
+    items_v = list(calib_v) if use_calib else []
+    items_a = list(calib_a) if use_calib else []
+    if use_history:
+        hist_idx = [clamp(start - S * (H - i)) for i in range(H)]
+        hist_frames = torch.stack([gen_frames[i] for i in hist_idx], dim=1)
+        hist_act = _build_action(mats[hist_idx], gopen[hist_idx], args.action_convention,
+                                 args.action_rot_format, args.action_trans_scale, args.action_rot_scale)
+        items_v.append(_to_u8(hist_frames))
+        items_a.append(hist_act)
+    items_v.append(_to_u8(cf_frames))
+    items_a.append(cf_act)
+    batch = build_fdm_batch_multiitem(model, items_v, items_a, caption, device, fps=args.fps)
+    with torch.no_grad():
+        outputs = model.generate_samples_from_batch(
+            batch, guidance=1.0, action_guidance=args.action_cfg_scale,
+            seed=[args.seed], num_steps=args.num_steps)
+        dec = model.decode(outputs["vision"][0])[0].clamp(-1, 1)  # (C,P+1,h,w)
+    pred_u8 = ((dec.float().permute(1, 2, 3, 0) + 1) / 2 * 255).round().clamp(0, 255).byte().cpu().numpy()
+    gt_win = torch.stack([ep_frames[clamp(start + i)] for i in range(P + 1)], dim=0)
+    gt_u8 = ((gt_win.float().permute(0, 2, 3, 1) + 1) / 2 * 255).round().clamp(0, 255).byte().cpu().numpy()
+    return pred_u8, gt_u8
+
+
+# ================================================================ boot
+_t = time.perf_counter()
+print("[boot] fetching processor assets ...", flush=True)
+QWEN_DIR = None
+for _f in QWEN_FILES:
+    QWEN_DIR = os.path.dirname(hf_hub_download(QWEN_REPO, _f))
+print(f"[boot] processor assets at {QWEN_DIR}", flush=True)
+
+print("[boot] fetching Wan2.2 video VAE ...", flush=True)
+VAE_PATH = hf_hub_download(VAE_REPO, VAE_FILE)
+
+print(f"[boot] fetching {CKPT_REPO} (~31 GB) ...", flush=True)
+CKPT_DIR = snapshot_download(CKPT_REPO)
+print(f"[boot] checkpoint at {CKPT_DIR} ({time.perf_counter() - _t:.0f}s)", flush=True)
+
+print("[boot] building + loading SyncWorld ...", flush=True)
+MODEL = _load_model(CKPT_DIR, VAE_PATH, QWEN_DIR)
+print(f"[boot] model ready ({time.perf_counter() - _t:.0f}s)", flush=True)
+
+if not os.environ.get("SYNCWORLD_KEEP_CKPT"):
+    # The weights now live in (fake-)CUDA tensors that ZeroGPU packs to its own
+    # on-disk store; the 31 GB download is dead weight and would blow the Space's
+    # ephemeral disk once the pack copy lands.
+    try:
+        repo_root = Path(CKPT_DIR).parent.parent
+        if repo_root.name.startswith("models--"):
+            shutil.rmtree(repo_root, ignore_errors=True)
+            print(f"[boot] freed {repo_root}", flush=True)
+    except Exception as e:  # pragma: no cover
+        print(f"[boot] WARN could not free checkpoint dir: {e!r}", flush=True)
+
+# ================================================================ scenes
+with open(HERE / "assets" / "scenes" / "scenes.json") as f:
+    SCENE_META = json.load(f)
+SCENES = {}
+for _slug, _m in SCENE_META.items():
+    _base = HERE / "assets" / "scenes" / _slug
+    SCENES[_m["label"]] = dict(
+        slug=_slug,
+        leaf=str(_base / "expert"),
+        calib=str(_base / "calibration"),
+        view=_m["view"],
+        num_frames=int(_m["num_frames"]),
+        thumb=str(_base / "thumb.jpg"),
+        expert_video=str(_base / "expert" / _m["view"] / "video.mp4"),
+        calib_video=str(_base / "calibration" / _m["view"] / "video.mp4"),
+        suite="ManiSkill" if _m["suite"].endswith("maniskill") else "LIBERO",
+        task=_m["task"],
+    )
+SCENE_NAMES = sorted(SCENES)
+MAX_FRAMES = max(v["num_frames"] for v in SCENES.values())
+
+MODE_REPLAY = "Replay the robot's own actions"
+MODE_DRIVE = "Drive the arm myself"
+GRIP_KEEP, GRIP_OPEN, GRIP_CLOSE = "keep current", "open", "close"
+
+
+def _write_mp4(frames_u8: np.ndarray, fps: int = 15) -> str:
+    d = tempfile.mkdtemp(dir=str(OUT_DIR))
+    p = os.path.join(d, "rollout.mp4")
+    imageio.mimwrite(p, list(frames_u8), fps=fps, macro_block_size=1, quality=9)
+    return p
+
+
+def _estimate_duration(*a, **k):
+    """ZeroGPU reservation: scales with rollout rounds x sampler steps."""
+    rounds = int(a[10]) if len(a) > 10 else 2
+    steps = int(a[11]) if len(a) > 11 else 20
+    return int(min(280, 45 + max(1, rounds) * (8 + max(1, steps) * 2.2)))
+
+
+@spaces.GPU(duration=_estimate_duration)
+def simulate(
+    scene: str,
+    action_mode: str = MODE_REPLAY,
+    start_frame: int = 0,
+    delta_x_cm: float = 0.0,
+    delta_y_cm: float = 0.0,
+    delta_z_cm: float = 0.0,
+    delta_rot_x_deg: float = 0.0,
+    delta_rot_y_deg: float = 0.0,
+    delta_rot_z_deg: float = 0.0,
+    gripper: str = GRIP_KEEP,
+    rollout_rounds: int = 2,
+    num_steps: int = 20,
+    action_guidance: float = 1.0,
+    seed: int = 0,
+    progress=gr.Progress(track_tqdm=True),
+):
+    """Roll out the SyncWorld world model on a visually-calibrated robot episode.
+
+    Args:
+        scene: name of a bundled calibrated episode (a ManiSkill or LIBERO tabletop scene).
+        action_mode: "Replay the robot's own actions" to re-simulate the expert trajectory,
+            or "Drive the arm myself" to command a constant per-step end-effector twist.
+        start_frame: index of the episode frame used as the current observation.
+        delta_x_cm: commanded per-step end-effector translation along body X, in centimetres.
+        delta_y_cm: commanded per-step end-effector translation along body Y, in centimetres.
+        delta_z_cm: commanded per-step end-effector translation along body Z, in centimetres.
+        delta_rot_x_deg: commanded per-step end-effector rotation about X, in degrees.
+        delta_rot_y_deg: commanded per-step end-effector rotation about Y, in degrees.
+        delta_rot_z_deg: commanded per-step end-effector rotation about Z, in degrees.
+        gripper: commanded gripper state ("keep current", "open" or "close").
+        rollout_rounds: number of autoregressive 16-frame windows to chain.
+        num_steps: rectified-flow sampler steps per window.
+        action_guidance: action classifier-free-guidance scale.
+        seed: sampler seed.
+
+    Returns:
+        A tuple of (predicted rollout mp4, ground-truth window mp4, markdown run report).
+    """
+    t0 = time.perf_counter()
+    sc = SCENES.get(scene) or SCENES[SCENE_NAMES[0]]
+    args = _recipe(num_steps=int(num_steps), action_cfg_scale=float(action_guidance), seed=int(seed))
+
+    ep_frames, mats, gopen, caption = read_episode(sc["leaf"], sc["view"], args.resolution)
+    calib_v, calib_a = build_calib_items(sc["calib"], args, sc["view"])
+    t_prep = time.perf_counter() - t0
+
+    T = int(ep_frames.shape[0])
+    P = args.num_pred_frames
+    start = max(0, min(int(start_frame), T - 1))
+    rounds = max(1, min(int(rollout_rounds), 3))
+    drive = str(action_mode) == MODE_DRIVE
+
+    custom_row = None
+    if drive:
+        if gripper == GRIP_OPEN:
+            g = 1.0
+        elif gripper == GRIP_CLOSE:
+            g = 0.0
+        else:
+            g = float(gopen[start])
+        custom_row = torch.tensor(
+            [float(delta_x_cm), float(delta_y_cm), float(delta_z_cm),
+             float(delta_rot_x_deg), float(delta_rot_y_deg), float(delta_rot_z_deg), g],
+            dtype=torch.float32)
+
+    gen_frames = ep_frames.clone()
+    pred_acc, gt_acc = [], []
+    for r in range(rounds):
+        s = start + r * P
+        cact = custom_row.unsqueeze(0).repeat(P, 1) if custom_row is not None else None
+        pred_u8, gt_u8 = _gen_window(MODEL, ep_frames, gen_frames, mats, gopen, caption,
+                                     calib_v, calib_a, s, args, DEVICE, custom_act=cact)
+        if rounds > 1:  # closed-loop write-back: the next round conditions on generated frames
+            for i in range(min(pred_u8.shape[0], gen_frames.shape[0] - s)):
+                fr = pred_u8[i].astype(np.float32) / 255.0 * 2.0 - 1.0
+                gen_frames[s + i] = torch.from_numpy(fr).permute(2, 0, 1).to(gen_frames)
+        skip = 0 if r == 0 else 1  # drop the 1-frame overlap between rounds
+        m = min(pred_u8.shape[0], gt_u8.shape[0])
+        pred_acc.extend(pred_u8[i] for i in range(skip, m))
+        gt_acc.extend(gt_u8[i] for i in range(skip, m))
+
+    pred = np.stack(pred_acc, 0)
+    gt = np.stack(gt_acc, 0)
+    pred_path = _write_mp4(pred)
+    gt_path = _write_mp4(gt)
+    dt = time.perf_counter() - t0
+
+    if drive:
+        act_desc = (f"commanded twist  `Δpos = ({delta_x_cm:+.2f}, {delta_y_cm:+.2f}, "
+                    f"{delta_z_cm:+.2f}) cm/step`,  `Δrot = ({delta_rot_x_deg:+.2f}, "
+                    f"{delta_rot_y_deg:+.2f}, {delta_rot_z_deg:+.2f}) deg/step`,  gripper "
+                    f"**{gripper}** — held for {rounds * P} steps")
+    else:
+        act_desc = f"the episode's own recorded actions for frames {start}…{min(start + rounds * P, T - 1)}"
+    report = (
+        f"**{sc['suite']} · `{sc['task']}`**  ·  camera `{sc['view']}`  ·  "
+        f"episode has {T} frames\n\n"
+        f"Conditioned on **6 per-DoF calibration segments** + **25 sparse history frames** "
+        f"(stride 3, from frame {max(0, start - 3 * 25)}) + current frame **{start}**.\n\n"
+        f"Action: {act_desc}.\n\n"
+        f"Generated **{len(pred)} frames** in {rounds} autoregressive window(s) × {args.num_steps} "
+        f"sampler steps  ·  **{dt:.1f}s** total (scene prep {t_prep:.1f}s)."
+    )
+    return pred_path, gt_path, report
+
+
+# ================================================================ UI
+def _scene_preview(scene: str, start_frame: int):
+    sc = SCENES.get(scene) or SCENES[SCENE_NAMES[0]]
+    T = sc["num_frames"]
+    s = max(0, min(int(start_frame), T - 1))
+    try:
+        frame = _read_video_cached(sc["expert_video"])[s]
+    except Exception:
+        frame = None
+    info = (f"`{sc['suite']}` · `{sc['task']}` · camera `{sc['view']}` · **{T} frames** "
+            f"· current frame **{s}**")
+    return frame, sc["calib_video"], sc["expert_video"], info
+
+
+THEME = gr.themes.Citrus()
+
+DESC = """\
+# 🦾 SyncWorld — a zero-shot robot simulator
+
+[**SyncWorld**](https://huggingface.co/papers/2609.09155) shows that a pretrained video world model
+becomes a usable robot simulator once you *visually calibrate* it: prepend a short clip of the arm
+sweeping each degree of freedom, and the model infers the unseen camera↔robot mapping on the fly —
+no per-scene finetuning.
+
+Pick a calibrated scene, then either **replay the robot's own actions** or **drive the arm yourself**
+with a per-step end-effector twist, and watch the model render what happens next.
+
+*Model: [`yyuncong/SyncWorld`](https://huggingface.co/yyuncong/SyncWorld) (16B Cosmos-3 mixture-of-transformers,
+Wan2.2 video VAE). Scenes and calibration clips are the authors' own evaluation episodes from
+[`yyuncong/SyncWorld-Evaluation`](https://huggingface.co/datasets/yyuncong/SyncWorld-Evaluation) (OpenMDW-1.1).*
+"""
+
+with gr.Blocks(theme=THEME, title="SyncWorld — zero-shot robot simulator") as demo:
+    gr.Markdown(DESC)
+
+    with gr.Row():
+        with gr.Column(scale=5):
+            scene = gr.Dropdown(SCENE_NAMES, value=SCENE_NAMES[0], label="Calibrated scene")
+            info_md = gr.Markdown()
+            with gr.Row():
+                cur_img = gr.Image(label="Current frame (the model's last observation)",
+                                   height=232, interactive=False)
+                calib_vid = gr.Video(label="Calibration sweep (6 DoF)", height=232,
+                                     interactive=False, autoplay=True, loop=True)
+            start_frame = gr.Slider(0, MAX_FRAMES - 1, value=0, step=1,
+                                    label="Current frame index (clamped to the episode length)")
+            action_mode = gr.Radio([MODE_REPLAY, MODE_DRIVE], value=MODE_REPLAY, label="Actions")
+            with gr.Group():
+                gr.Markdown("**Commanded end-effector twist** — used in *Drive the arm myself* mode. "
+                            "Held constant for every predicted step (the calibration sweeps cover "
+                            "roughly ±3 cm/step and ±3 °/step).")
+                with gr.Row():
+                    dx = gr.Slider(-3.0, 3.0, value=0.0, step=0.05, label="Δ X (cm/step)")
+                    dy = gr.Slider(-3.0, 3.0, value=0.0, step=0.05, label="Δ Y (cm/step)")
+                    dz = gr.Slider(-3.0, 3.0, value=0.0, step=0.05, label="Δ Z (cm/step)")
+                with gr.Row():
+                    rx = gr.Slider(-3.0, 3.0, value=0.0, step=0.05, label="Δ rot X (°/step)")
+                    ry = gr.Slider(-3.0, 3.0, value=0.0, step=0.05, label="Δ rot Y (°/step)")
+                    rz = gr.Slider(-3.0, 3.0, value=0.0, step=0.05, label="Δ rot Z (°/step)")
+                gripper = gr.Radio([GRIP_KEEP, GRIP_OPEN, GRIP_CLOSE], value=GRIP_KEEP,
+                                   label="Gripper")
+            rollout_rounds = gr.Slider(1, 3, value=2, step=1,
+                                       label="Rollout windows (16 generated frames each)")
+            with gr.Accordion("Advanced", open=False):
+                num_steps = gr.Slider(4, 40, value=20, step=1, label="Sampler steps per window")
+                action_guidance = gr.Slider(1.0, 5.0, value=1.0, step=0.1,
+                                            label="Action guidance (CFG)")
+                seed = gr.Slider(0, 2**31 - 1, value=0, step=1, label="Seed")
+            run = gr.Button("Simulate", variant="primary")
+
+        with gr.Column(scale=5):
+            out_pred = gr.Video(label="SyncWorld rollout", autoplay=True, loop=True,
+                                interactive=False, height=360)
+            out_gt = gr.Video(label="Ground truth for the same window (reference)",
+                              autoplay=True, loop=True, interactive=False, height=240)
+            report_md = gr.Markdown()
+
+    with gr.Accordion("Full expert episode (reference)", open=False):
+        expert_vid = gr.Video(label="Expert demonstration", interactive=False, height=360)
+
+    INPUTS = [scene, action_mode, start_frame, dx, dy, dz, rx, ry, rz, gripper,
+              rollout_rounds, num_steps, action_guidance, seed]
+    OUTPUTS = [out_pred, out_gt, report_md]
+
+    # Every example row supplies the FULL input list. Gradio inserts its Progress
+    # object at the positional index of the `progress` parameter, so a short row
+    # would silently shift values into the wrong arguments.
+    EXAMPLES = [
+        # scene, mode, start, dx, dy, dz, rx, ry, rz, gripper, rounds, steps, cfg, seed
+        ["ManiSkill · PushCube", MODE_REPLAY, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, GRIP_KEEP, 2, 20, 1.0, 0],
+        ["LIBERO · put the bowl on the plate", MODE_REPLAY, 16, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, GRIP_KEEP, 2, 20, 1.0, 0],
+        ["ManiSkill · StackCube", MODE_DRIVE, 32, 0.0, 1.2, 0.0, 0.0, 0.0, 0.0, GRIP_KEEP, 2, 20, 1.0, 0],
+        ["ManiSkill · PushCube", MODE_DRIVE, 0, 1.5, 0.0, 0.0, 0.0, 0.0, 0.0, GRIP_KEEP, 2, 20, 1.0, 0],
+        ["LIBERO · open the middle drawer", MODE_DRIVE, 24, 0.0, 0.0, -1.2, 0.0, 0.0, 0.0, GRIP_KEEP, 2, 20, 1.0, 0],
+        ["LIBERO · alphabet soup into basket", MODE_DRIVE, 48, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, GRIP_CLOSE, 2, 20, 1.0, 0],
+    ]
+    EXAMPLE_LABELS = [
+        "PushCube · replay the expert",
+        "LIBERO bowl→plate · replay the expert",
+        "StackCube · push +Y at 1.2 cm/step",
+        "PushCube · push +X at 1.5 cm/step",
+        "LIBERO drawer · pull −Z at 1.2 cm/step",
+        "LIBERO soup · close the gripper, hold still",
+    ]
+
+    run.click(fn=simulate, inputs=INPUTS, outputs=OUTPUTS)
+
+    for _ev in (scene.change, start_frame.release):
+        _ev(fn=_scene_preview, inputs=[scene, start_frame],
+            outputs=[cur_img, calib_vid, expert_vid, info_md], show_progress="minimal")
+
+    gr.Examples(
+        examples=EXAMPLES,
+        example_labels=EXAMPLE_LABELS,
+        inputs=INPUTS,
+        outputs=OUTPUTS,
+        fn=simulate,
+        cache_examples=True,
+        cache_mode="lazy",
+        label="Examples",
+    )
+
+    demo.load(fn=_scene_preview, inputs=[scene, start_frame],
+              outputs=[cur_img, calib_vid, expert_vid, info_md])
+
+if __name__ == "__main__":
+    demo.queue(max_size=12).launch(mcp_server=True, allowed_paths=[str(OUT_DIR), str(HERE / "assets")])

assets/scenes/libero-black-bowl/calibration/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4df8106071d26afbad4430d65b4a4ad87f036d25a19b9acd7c033f6c9db94b9b
+size 130801

assets/scenes/libero-black-bowl/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f36ffe48457db0d8e3c8a7f98164b86e6ff04ddd09566b323967fd09dcc4d38c
+size 33351

assets/scenes/libero-black-bowl/expert/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fd11adb011caf6ecd364f3908fbedde20fac0973a66470ffda335dd39830dd7
+size 147392

assets/scenes/libero-black-bowl/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c099ddae68aff7e093b72f7a2c8f31eeeb1e3fda88a176d50b4e2d7e8b880321
+size 20164

assets/scenes/libero-bowl-plate/calibration/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df76bf0a3a6ea9c66884e93059338a39697d01b0381ff203bcefd267e2325f33
+size 171469

assets/scenes/libero-bowl-plate/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a68fc4ce6960d7fb65192626ac2be4ad9426bf14ee188806c19057a43133c502
+size 42567

assets/scenes/libero-bowl-plate/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:031a8f596b385be63492ac350721145051067df2d797333062ee9bd1bfc4cb35
+size 11844

assets/scenes/libero-drawer/calibration/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c86ccd71e4cf1a410d7cd8e0defadc3b907ef64b42d60c4d40a7c3ca19a4756a
+size 142168

assets/scenes/libero-drawer/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df632260a5b759cf03f74c7f9604d869fb42627aa3ae010ccb67ff5aaf6f7ca5
+size 34887

assets/scenes/libero-drawer/expert/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d192f2e9079070ee653110ea023b74f8ccb2817b2ec65b334191fdbe5ececa5
+size 142231

assets/scenes/libero-drawer/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa11f6bdcceb3e83d6367738c1c08b123ffddccc9d1102fe5990ef08091af8b4
+size 17988

assets/scenes/libero-soup/calibration/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99056f78f6dd0237eac3a61928dc7cc7538c64d010376c6740b9543a5d175231
+size 199757

assets/scenes/libero-soup/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e082d106c2b953cf904dff7909e11d6637f3dfcb7d87892f3d0927baac527a6
+size 37959

assets/scenes/libero-soup/expert/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6afa6a3984a1ecc4f2f409cc590d2e4b045f5bbc3f6f9c18602907eabc7d50e
+size 158901

assets/scenes/libero-soup/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a29645a254feda994d340368327851da72e0db55725803bd1f8031ce7028503
+size 19268

assets/scenes/libero-stove/calibration/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e05506712a8c19a320c05d2b663e49a375f9f84bdeb35404130022b789cfd10b
+size 178356

assets/scenes/libero-stove/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:551906131f26cb71feeeb75ac9caac4b0e5d6eb915228e61b65d6175dc41eb9b
+size 35655

assets/scenes/libero-stove/expert/agentview_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebd84ff43c9fd0a2c6968444e25dc1e6f21daf87d65a66ef6dc4fd446b90d041
+size 299642

assets/scenes/libero-stove/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:432f2235bb96fc43306c83426be6d4a72fcce9a5273212f5a13044926104aea1
+size 35143

assets/scenes/peginsertion/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b821226f7d54ce35d9c65e34ee3733e3b558c6d0b33ad3d820a84441b44eccc2
+size 40263

assets/scenes/peginsertion/calibration/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce685583a51a6ed10388767caa1194578ab91c8ba7564f59c66cd9f8a38643c7
+size 401985

assets/scenes/peginsertion/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f3dfb8b8fc36322257195fd800abd768cf954448ac9a961995cb3f87f3a3c35
+size 23270

assets/scenes/peginsertion/expert/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:868e664dfdbb75df9d438c7eb9e0286c7e2b007b29224928b38a437f3c488d12
+size 290856

assets/scenes/plugcharger/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9b7a6ac349bff83a695e315bc02e662747d3fd96e2f65085396a0d1342754be
+size 34887

assets/scenes/plugcharger/calibration/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94cc01fd066e02b89376a7f71fc5fc25c5c09ce21116547610ed66740ab0967f
+size 273092

assets/scenes/plugcharger/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d35f4694d2270210e30323191372f10c3df808c4f5454ae63d796e850122b23
+size 20454

assets/scenes/plugcharger/expert/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94b64706f8606a3d31bcd7e34f55347a86c2a0e81c492d202224b8037bb7edf0
+size 206705

assets/scenes/pushcube/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e150c2689746def363d24924317313239b155528148c5d212d475535d7a93b82
+size 37959

assets/scenes/pushcube/calibration/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:74a428f128b46fadedd1327ea6c1882267f766520e7cdcdeb6fc00da0e9f9ce0
+size 385927

assets/scenes/pushcube/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1feb82f1bf0301033fc5748f36a04023bea4119f4221165da87771dc11a19157
+size 9574

assets/scenes/pushcube/expert/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:535f820fb0f05cc360e09bf46c8983868f28992521e4f4279ba7684f3aed88b7
+size 168226

assets/scenes/scenes.json

@@ -0,0 +1,72 @@
+{
+ "pushcube": {
+  "label": "ManiSkill \u00b7 PushCube",
+  "view": "render_camera_rgb",
+  "suite": "evaluation_maniskill",
+  "task": "PushCube-v1",
+  "num_frames": 72
+ },
+ "stackcube": {
+  "label": "ManiSkill \u00b7 StackCube",
+  "view": "render_camera_rgb",
+  "suite": "evaluation_maniskill",
+  "task": "StackCube-v1",
+  "num_frames": 108
+ },
+ "stackpyramid": {
+  "label": "ManiSkill \u00b7 StackPyramid",
+  "view": "render_camera_rgb",
+  "suite": "evaluation_maniskill",
+  "task": "StackPyramid-v1",
+  "num_frames": 177
+ },
+ "peginsertion": {
+  "label": "ManiSkill \u00b7 PegInsertionSide",
+  "view": "render_camera_rgb",
+  "suite": "evaluation_maniskill",
+  "task": "PegInsertionSide-v1",
+  "num_frames": 179
+ },
+ "plugcharger": {
+  "label": "ManiSkill \u00b7 PlugCharger",
+  "view": "render_camera_rgb",
+  "suite": "evaluation_maniskill",
+  "task": "PlugCharger-v1",
+  "num_frames": 157
+ },
+ "libero-bowl-plate": {
+  "label": "LIBERO \u00b7 put the bowl on the plate",
+  "view": "agentview_rgb",
+  "suite": "evaluation_libero",
+  "task": "put_the_bowl_on_the_plate",
+  "num_frames": 90
+ },
+ "libero-drawer": {
+  "label": "LIBERO \u00b7 open the middle drawer",
+  "view": "agentview_rgb",
+  "suite": "evaluation_libero",
+  "task": "open_the_middle_drawer_of_the_cabinet",
+  "num_frames": 138
+ },
+ "libero-soup": {
+  "label": "LIBERO \u00b7 alphabet soup into basket",
+  "view": "agentview_rgb",
+  "suite": "evaluation_libero",
+  "task": "pick_up_the_alphabet_soup_and_place_it_in_the_basket",
+  "num_frames": 148
+ },
+ "libero-black-bowl": {
+  "label": "LIBERO \u00b7 black bowl onto plate",
+  "view": "agentview_rgb",
+  "suite": "evaluation_libero",
+  "task": "pick_up_the_black_bowl_on_the_stove_and_place_it_on_the_plate",
+  "num_frames": 155
+ },
+ "libero-stove": {
+  "label": "LIBERO \u00b7 stove + moka pot",
+  "view": "agentview_rgb",
+  "suite": "evaluation_libero",
+  "task": "KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it",
+  "num_frames": 272
+ }
+}

assets/scenes/stackcube/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7788b705707d9566bed30098bc22c7ae9c59301da54bce0a5270b35fb12a8864
+size 46407

assets/scenes/stackcube/calibration/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcabdaf8a0af444fc1dbca7ee60a435fcdb23202636962db976ca8ed92a42440
+size 468760

assets/scenes/stackcube/expert/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0dc2b5f9e732b1f090e6a815f86582dcf7cba27feaa8c9124f0ac598cae69ab1
+size 14182

assets/scenes/stackcube/expert/render_camera_rgb/video.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1ebf6d49490eb59a8d161be2f0c12f36202297cfee95b7f9f97da45e2ad19107
+size 192343

assets/scenes/stackpyramid/calibration/pose.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:393a570213981f6377fe153917d39b3ee3db73b7208a4c5bdcdf23d527b34359
+size 37959