model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: lerobot/smolvla_base
|
| 4 |
+
library_name: lerobot
|
| 5 |
+
pipeline_tag: robotics
|
| 6 |
+
tags: [robotics, vla, smolvla, lerobot, manipulation, maniguard, franka]
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# SmolVLA - Clutter (joint, 2-cam)
|
| 10 |
+
|
| 11 |
+
HuggingFace LeRobot **SmolVLA** fine-tuned on the ManiGuard **clutter** base task (sim
|
| 12 |
+
Franka Panda). Part of the ManiGuard VLA benchmark - SmolVLA vs pi0.5 vs GR00T on the
|
| 13 |
+
same task families with identical data, cameras, and controller.
|
| 14 |
+
|
| 15 |
+
## Model
|
| 16 |
+
- **Base:** [lerobot/smolvla_base](https://huggingface.co/lerobot/smolvla_base) - SmolVLM2 vision-language backbone + flow-matching action expert
|
| 17 |
+
- **Embodiment:** Franka Panda, **8-D joint** state/action (7 arm joints + 1 gripper), padded to SmolVLA's 32-D
|
| 18 |
+
- **Cameras (2):** `observation.images.top` (overview = `left` view) + `observation.images.wrist` (256x256)
|
| 19 |
+
- **Action:** absolute joint targets (fed straight to a JointController at eval); 50-step chunk
|
| 20 |
+
- **Tuning:** SmolVLA default - vision encoder **frozen**, train the action expert (**no LoRA**)
|
| 21 |
+
|
| 22 |
+
## Training
|
| 23 |
+
- 8-GPU accelerate DDP, global batch 512, 3522 steps (~2 epochs over 901,520 frames)
|
| 24 |
+
- Data: [IDEAS-Lab-Northwestern/datagen-clutter-v1-joint-5cam](https://huggingface.co/datasets/IDEAS-Lab-Northwestern/datagen-clutter-v1-joint-5cam) - the 5-cam datagen dataset, prepared to a 2-cam standard-keyed copy (H.264 videos)
|
| 25 |
+
|
| 26 |
+
## Usage
|
| 27 |
+
Load with `SmolVLAPolicy.from_pretrained("IDEAS-Lab-Northwestern/smolvla-base-datagen-v1-clutter-joint-2cam-yanZ")` from [LeRobot](https://github.com/huggingface/lerobot). The checkpoint carries the normalization stats.
|
| 28 |
+
|
| 29 |
+
> WARNING - Convention (must match at eval): joint-space JointController (absolute joint targets) + 2 cameras (`observation.images.top` = the `left` overview, `observation.images.wrist`). A mismatched controller, camera set, or overview view silently feeds an out-of-distribution input.
|