README.md

---
tags: [robotics, lerobot, smolvla, xlerobot, jetson]
library_name: lerobot
pipeline_tag: robotics
license: apache-2.0
base_model: lerobot/smolvla_base
datasets:
- Suyang99/xlerobot-cup-grasp-20260820-0230
---

# SmolVLA — right-arm cup grasp (XLeRobot)

Finetuned from [`lerobot/smolvla_base`](https://huggingface.co/lerobot/smolvla_base) on
[`Suyang99/xlerobot-cup-grasp-20260820-0230`](https://huggingface.co/datasets/Suyang99/xlerobot-cup-grasp-20260820-0230).

| | |
|---|---|
| Task | `Pick up the cup and place it down` — **right arm only** (`arms: 1.0`) |
| Policy | **SmolVLA**, 450 M total / **100 M trainable** (VLM frozen: `freeze_vision_encoder=True`, `train_expert_only=True`) |
| VLM backbone | `HuggingFaceTB/SmolVLM2-500M-Video-Instruct`, 16 layers |
| **Uploaded checkpoint** | **step 16 000** |
| **Training reached** | **step 18 000 of a planned 20 000 — stopped early, see below** |
| Final loss | **0.053** at step 16 000 (0.054 at 18 000) |
| Trained on | Jetson Orin Nano Super, 8 GB unified memory, torch 2.8.0 + CUDA 12.6 |
| Batch size / throughput | 2 · ~1.05 step/s · 4 h 54 m wall clock |
| Cameras | `head` → `camera1`, `right_arm_wrist` → `camera2`, `left_arm_wrist` → `camera3` |
| Action / state | 17-dim (left arm 6 · right arm 6 · head 2 · base 3) |
| `chunk_size` / `n_action_steps` | 50 / 50 |

## Training data: 40 of the 50 episodes

`dataset.eval_split=0.2` held out the last 10 episodes. Because the dataset has a single task,
LeRobot's per-task split (`datasets/factory.py:156`) took the **last** 10 episodes — which, since
cup positions were recorded in order, is **exactly position P5**.

So this model trained on **P1–P4 (40 episodes / 15 866 frames)** and never saw P5.

⚠️ `eval_steps` was left at its default of `0`, and `lerobot_train.py:748` gates evaluation on
`cfg.eval_steps > 0`. **No validation ran during training.** It was measured afterwards instead —
see below.

## Held-out result: no position memorisation detected

Measured after training with `05-training/eval_heldout.py`, 120 frames per group, using the same
loss path as training (uint8→float32/255 → preprocessor → `policy(batch)`):

| | frames | loss |
|---|---|---|
| Train positions P1–P4 | 120 | **0.2348** |
| **Held-out P5 (never trained on)** | 120 | **0.2300** |
| | | **ratio 0.98×** |

The model does as well on a position it never saw as on the ones it trained on. Had it merely
memorised "P3 → this trajectory", the held-out loss would be several times higher. This is the
pretrained prior doing its job.

**This does not replace a real-robot test.** P5 is still the same table, cup and lighting; a low
loss says the predicted actions match the human demonstrations, not that the arm actually grasps.

## Why training stopped at 18 000 instead of 20 000

Not divergence, not a manual interrupt — **the disk filled up** while writing the step-18 000
checkpoint:

```
safetensors._safetensors_rust.SafetensorError: Error while serializing:
    I/O error: No space left on device (os error 28)
```

Each SmolVLA checkpoint is 1.26 GB and one was saved every 2 000 steps. The step-18 000 checkpoint
was left half-written, so `checkpoints/last` still points at **016000** — which is what is uploaded
here. Loss had already flattened (0.053 at 16 000, 0.054 at 18 000), so the missing 2 000 steps
make no material difference.

## Loss curve

`training_log.csv` in this repo carries all 27 logged points. Summary:

| step | epoch | loss | grad norm |
|---|---|---|---|
| 100 | 0.01 | 2.366 | 18.683 |
| 200 | 0.03 | 0.936 | 8.605 |
| 16 000 | 2.08 | **0.053** | 1.178 |
| 18 000 | 2.27 | 0.054 | 1.223 |

Loss fell 98% and the gradient norm 93%; the last 2 000 steps vary by less than 0.02. Converged.

## Limitations

1. **Cannot drive the left arm.** The recording ran with `arms: 1.0`, so action dimensions 0–5 hold
   one constant across all frames. This policy will not transfer to a left-arm version of the task.
2. **Trained on data from the 68 mm gripper.** The Fin-Ray blade is being replaced with a 90 mm
   version; lever arm and compliance both change, so real-world behaviour must be re-measured.
3. **Zero physical trials so far.** Every number here is offline.

## Companion model

[`Suyang99/xlerobot-act-cup-grasp-right`](https://huggingface.co/Suyang99/xlerobot-act-cup-grasp-right)
— ACT trained from scratch on all 50 episodes, for comparison. Loss values are **not** comparable
between the two: ACT's is `l1 + 10 × kld`, SmolVLA's is a different objective. Only real-robot
success rate compares them.