Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: gemma-terms-of-use
|
| 4 |
+
license_link: https://ai.google.dev/gemma/terms
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- pi0.5
|
| 8 |
+
- openpi
|
| 9 |
+
- lora
|
| 10 |
+
- tactile
|
| 11 |
+
- xarm
|
| 12 |
+
base_model: physical-intelligence/pi05_droid
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# pi05_xarm_charger_points9_arrow_len0_lora
|
| 16 |
+
|
| 17 |
+
pi0.5 (openpi) LoRA fine-tune for the xArm **charger** task (`"pick up the charger and plug it into the nearest plug"`), trained on
|
| 18 |
+
camera frames with the **points9_arrow_len0** tactile overlay: 9 tactile-pad dots per
|
| 19 |
+
finger at FK-projected positions, force arrows at **zero length** — the force-information
|
| 20 |
+
ablation of points9_arrow.
|
| 21 |
+
|
| 22 |
+
## Training
|
| 23 |
+
|
| 24 |
+
| | |
|
| 25 |
+
|---|---|
|
| 26 |
+
| Base | `pi05_droid` (knowledge-insulated pi0.5, DROID pre-training) |
|
| 27 |
+
| Recipe | LoRA r16/a16 on Gemma-2B backbone + r32/a32 on 300M action expert, attn+ffn |
|
| 28 |
+
| Batch / LR | 8 / cosine 1e-4 -> 1e-5, 500 warmup, AdamW, grad-clip 1.0 |
|
| 29 |
+
| Data | [EdwardoSunny/xarm_charger_points9_arrow_len0](https://huggingface.co/datasets/EdwardoSunny/xarm_charger_points9_arrow_len0) (100 eps, 23808 frames) |
|
| 30 |
+
| Early stop | step 5100 (rolling-1k-window <0.5% rel. improvement, 2 consecutive checks) |
|
| 31 |
+
| Final train loss | **0.0402** (baseline no-overlay: 0.0413; full points9_arrow: 0.04) |
|
| 32 |
+
|
| 33 |
+
Training-loss comparison across this task's variants (identical hyperparameters/protocol,
|
| 34 |
+
only the burned-in overlay differs) — see the collection for all four tasks.
|
| 35 |
+
|
| 36 |
+
## Contents
|
| 37 |
+
|
| 38 |
+
- `params/` — merged base+LoRA weights (orbax), self-contained for inference
|
| 39 |
+
- `assets/local/xarm_charger_points9_arrow_len0/norm_stats.json` — state/action normalization
|
| 40 |
+
- Optimizer state stripped (inference-ready). Config name: `pi05_xarm_charger_points9_arrow_len0_lora`
|
| 41 |
+
|
| 42 |
+
## Deployment note
|
| 43 |
+
|
| 44 |
+
At inference, render the live camera frames with the SAME overlay
|
| 45 |
+
(`SensorDrawer` mode `points9_arrow`, `arrow_length_scale=0`, thickness 8, dot size 22)
|
| 46 |
+
or the policy will be out-of-distribution. See `tactile-data-collection/scripts/render_arrowlen0.py`.
|