| --- |
| license: other |
| license_name: gemma-terms-of-use |
| license_link: https://ai.google.dev/gemma/terms |
| pipeline_tag: robotics |
| tags: [robotics, vla, openpi, manipulation, maniguard, franka] |
| --- |
| |
| # pi05-real-cab-higher-firsthalf-60-droid-refined-lora |
|
|
| LoRA fine-tune of **pi0.5** (`pi05_droid` warm-start) on |
| `IDEAS-Lab-Northwestern/real-cab-higher-firsthalf-60-droid-refined` — 60 real |
| Franka teleop trajectories of the **first half of the cabinet task** (open the |
| cabinet, keep the door area clear), DROID schema. |
|
|
| - **Task prompt:** "Open the cabinet. Keep the area around the door clear." |
| - **Warm-start:** `gs://openpi-assets/checkpoints/pi05_droid/params` (clean, fresh) |
| - **LoRA:** `gemma_2b_lora` (rank 16) + `gemma_300m_lora` (rank 32), batch 4, EMA off. |
| - **Norm-stats:** reuses `pi05_droid`'s bundled DROID assets (no recompute). |
|
|
| ## Checkpoint ladder |
|
|
| | Dir | Cumulative steps | Train loss | |
| |---|---|---| |
| | `10000/` | 10k | 0.0046 | |
| | `20000/` | 20k | 0.0032 | |
| | `30000/` | 30k | **0.0029** | |
| | `40000/` | 40k | 0.0038 | |
| | `50000/` | 50k (saved at step 49999) | 0.0031 | |
|
|
| Train loss bottoms at ~30k; eval-sweep **20k–40k**. |
|
|
| ## Refined real setup |
|
|
| Refined real setup (raised-Z cabinet) — refined wrist & main-camera poses and |
| gripper. Wrist = raw `cam1`, exterior = raw `cam0`. **Collector:** yypeng666. |
|
|
| ## Schema (DROID) / inference |
|
|
| `exterior_image_1_left` ← exterior cam0, `wrist_image_left` ← wrist cam1, |
| `exterior_image_2_left` zero-pad; `joint_position`(7)+`gripper_position`(1) state; |
| `actions`(8)=joint_velocity(7)+next gripper target(1). Serve with openpi |
| `pi05_droid_finetune_lora`. `train_state/` omitted (inference-only). Eval on a |
| real Franka, not the SFT server. |
| |
| ## Paper & Citation |
| |
| Part of **ManiGuard**: [paper (arXiv:2608.17386)](https://arxiv.org/abs/2608.17386) · |
| [code](https://github.com/NU-IDEAS-Lab/ManiGuard) · |
| [docs](https://nu-ideas-lab.github.io/ManiGuard/) |
| |
| ```bibtex |
| @misc{peng2026maniguard, |
| title = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded |
| Safety Evaluation and Improvement of Robotic Manipulation}, |
| author = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi |
| and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen |
| and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling |
| and Zhang, Ruohan and Zhu, Qi}, |
| year = {2026}, |
| eprint = {2608.17386}, |
| archivePrefix = {arXiv}, |
| primaryClass = {cs.RO}, |
| url = {https://arxiv.org/abs/2608.17386}, |
| } |
| ``` |
| |
| ## License |
| |
| The fine-tuned weights derive from a Physical Intelligence openpi base model whose |
| VLM backbone is PaliGemma; use of these weights is therefore subject to the |
| [Gemma Terms of Use](https://ai.google.dev/gemma/terms) (including the |
| [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy)), |
| which downstream users must pass on. The openpi training code and ManiGuard's own |
| contributions are Apache-2.0. |
| |
