Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: gemma-terms-of-use
|
| 4 |
+
license_link: https://ai.google.dev/gemma/terms
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- pi0.5
|
| 8 |
+
- openpi
|
| 9 |
+
- lora
|
| 10 |
+
- tactile
|
| 11 |
+
- xarm
|
| 12 |
+
base_model: physical-intelligence/pi05_droid
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# pi05_xarm_cube_binmarker_lora
|
| 16 |
+
|
| 17 |
+
pi0.5 (openpi) LoRA fine-tune for the xArm **cube** task (`"pick up the cube"`),
|
| 18 |
+
trained on frames with the **binmarker** overlay: a fixed-length,
|
| 19 |
+
fixed-direction arrow per finger (sensor-frame pad normal), shown iff that
|
| 20 |
+
finger is in binary contact. NO force magnitude and NO force direction — the
|
| 21 |
+
overlay carries only the per-finger contact bit. Ported from
|
| 22 |
+
LIBERO_contact_overlay's binbars style.
|
| 23 |
+
|
| 24 |
+
## Training
|
| 25 |
+
|
| 26 |
+
| | |
|
| 27 |
+
|---|---|
|
| 28 |
+
| Base | `pi05_droid` |
|
| 29 |
+
| Recipe | LoRA r16/a16 backbone + r32/a32 action expert, attn+ffn |
|
| 30 |
+
| Batch / LR | 8 / cosine 1e-4 -> 1e-5, 500 warmup, AdamW, grad-clip 1.0 |
|
| 31 |
+
| Data | [EdwardoSunny/xarm_cube_binmarker](https://huggingface.co/datasets/EdwardoSunny/xarm_cube_binmarker) (100 eps, 13103 frames) |
|
| 32 |
+
| Early stop | step 7800 |
|
| 33 |
+
| Final train loss | **0.0429** |
|
| 34 |
+
|
| 35 |
+
Variant comparison for this task (identical protocol, only overlay pixels differ):
|
| 36 |
+
baseline 0.0437 / points9_arrow_len0 0.0435 / **binmarker 0.0429** / points9_arrow 0.0425.
|
| 37 |
+
|
| 38 |
+
## Contact detection (how the threshold was tuned)
|
| 39 |
+
|
| 40 |
+
Per-finger binary contact from the 3x3x3-axis tactile array:
|
| 41 |
+
|
| 42 |
+
- **Normalization offset** = median over known-idle frames (first 10 frames of
|
| 43 |
+
each episode, post-homing, gripper open). The naive whole-dataset median is
|
| 44 |
+
contact-polluted on tasks that spend most frames in contact (charger: 60%
|
| 45 |
+
closed frames made idle read as a fake magnitude of ~1.2).
|
| 46 |
+
- **Metric** = max over the 9 cells of the normalized cell-vector L2 norm
|
| 47 |
+
("max-cell"). The cell-sum alternative cannot separate charger idle noise
|
| 48 |
+
(p99.5 = 0.45) from real light holds (0.5 on tube/dishwasher's weaker finger).
|
| 49 |
+
Max-cell: idle p99.5 <= 0.24 on every task, real contacts >= ~0.3.
|
| 50 |
+
- **Hysteresis + debounce** (Schmitt trigger): ON after 2 consecutive frames
|
| 51 |
+
>= 0.25, OFF after 3 consecutive frames < 0.18, state reset at episode
|
| 52 |
+
boundaries. Light-but-real holds (dips to 0.23) stay latched; single-frame
|
| 53 |
+
noise spikes never trigger.
|
| 54 |
+
|
| 55 |
+
Validated on all 400 episodes (zero idle false positives in every post-homing
|
| 56 |
+
window) and **visually verified on 5 episodes per task** (timelines + frame
|
| 57 |
+
montages at approach / contact onset / mid-hold / post-release), plus a final
|
| 58 |
+
spot check on the converted LeRobot frames: no arrows during approach, arrows
|
| 59 |
+
at contact onset, latched through holds, cleared within <=3 frames of release,
|
| 60 |
+
and honest negatives (gripper closed around nothing draws nothing).
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
## Contents
|
| 64 |
+
|
| 65 |
+
- `params/` — merged weights (orbax), self-contained for inference
|
| 66 |
+
- `assets/local/xarm_cube_binmarker/norm_stats.json`
|
| 67 |
+
- Config name: `pi05_xarm_cube_binmarker_lora`
|
| 68 |
+
|
| 69 |
+
## Deployment note
|
| 70 |
+
|
| 71 |
+
At inference, reproduce the SAME overlay on live frames: run the tuned contact
|
| 72 |
+
detector (max-cell metric, ON>=0.25 / OFF<0.18, 2-on/3-off debounce, idle-window
|
| 73 |
+
offset) and draw the fixed-direction marker (FIXED_DIRECTION, arrow_length_scale=0.12,
|
| 74 |
+
thickness 8, dot 22) — see `tactile-data-collection/scripts/render_binmarker.py`
|
| 75 |
+
at commit c64c562 or later (earlier revisions used the measured force direction).
|