Suyang99/xlerobot-smolvla-cup-grasp-right

SmolVLA — right-arm cup grasp (XLeRobot)

Finetuned from lerobot/smolvla_base on Suyang99/xlerobot-cup-grasp-20260820-0230.

Task	Pick up the cup and place it down — right arm only (arms: 1.0)
Policy	SmolVLA, 450 M total / 100 M trainable (VLM frozen: freeze_vision_encoder=True, train_expert_only=True)
VLM backbone	HuggingFaceTB/SmolVLM2-500M-Video-Instruct, 16 layers
Uploaded checkpoint	step 16 000
Training reached	step 18 000 of a planned 20 000 — stopped early, see below
Final loss	0.053 at step 16 000 (0.054 at 18 000)
Trained on	Jetson Orin Nano Super, 8 GB unified memory, torch 2.8.0 + CUDA 12.6
Batch size / throughput	2 · ~1.05 step/s · 4 h 54 m wall clock
Cameras	head → camera1, right_arm_wrist → camera2, left_arm_wrist → camera3
Action / state	17-dim (left arm 6 · right arm 6 · head 2 · base 3)
chunk_size / n_action_steps	50 / 50

Training data: 40 of the 50 episodes

dataset.eval_split=0.2 held out the last 10 episodes. Because the dataset has a single task, LeRobot's per-task split (datasets/factory.py:156) took the last 10 episodes — which, since cup positions were recorded in order, is exactly position P5.

So this model trained on P1–P4 (40 episodes / 15 866 frames) and never saw P5.

⚠️ eval_steps was left at its default of 0, and lerobot_train.py:748 gates evaluation on cfg.eval_steps > 0. No validation ran during training. It was measured afterwards instead — see below.

Held-out result: no position memorisation detected

Measured after training with 05-training/eval_heldout.py, 120 frames per group, using the same loss path as training (uint8→float32/255 → preprocessor → policy(batch)):

	frames	loss
Train positions P1–P4	120	0.2348
Held-out P5 (never trained on)	120	0.2300
		ratio 0.98×

The model does as well on a position it never saw as on the ones it trained on. Had it merely memorised "P3 → this trajectory", the held-out loss would be several times higher. This is the pretrained prior doing its job.

This does not replace a real-robot test. P5 is still the same table, cup and lighting; a low loss says the predicted actions match the human demonstrations, not that the arm actually grasps.

Why training stopped at 18 000 instead of 20 000

Not divergence, not a manual interrupt — the disk filled up while writing the step-18 000 checkpoint:

safetensors._safetensors_rust.SafetensorError: Error while serializing:
    I/O error: No space left on device (os error 28)

Each SmolVLA checkpoint is 1.26 GB and one was saved every 2 000 steps. The step-18 000 checkpoint was left half-written, so checkpoints/last still points at 016000 — which is what is uploaded here. Loss had already flattened (0.053 at 16 000, 0.054 at 18 000), so the missing 2 000 steps make no material difference.

Loss curve

training_log.csv in this repo carries all 27 logged points. Summary:

step	epoch	loss	grad norm
100	0.01	2.366	18.683
200	0.03	0.936	8.605
16 000	2.08	0.053	1.178
18 000	2.27	0.054	1.223

Loss fell 98% and the gradient norm 93%; the last 2 000 steps vary by less than 0.02. Converged.

Limitations

1. Cannot drive the left arm. The recording ran with arms: 1.0, so action dimensions 0–5 hold one constant across all frames. This policy will not transfer to a left-arm version of the task.
2. Trained on data from the 68 mm gripper. The Fin-Ray blade is being replaced with a 90 mm version; lever arm and compliance both change, so real-world behaviour must be re-measured.
3. Zero physical trials so far. Every number here is offline.

Companion model

Suyang99/xlerobot-act-cup-grasp-right — ACT trained from scratch on all 50 episodes, for comparison. Loss values are not comparable between the two: ACT's is l1 + 10 × kld, SmolVLA's is a different objective. Only real-robot success rate compares them.