EdwardoSunny commited on
Commit
5dbb4cf
·
verified ·
1 Parent(s): ad585f0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +75 -0
README.md ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: gemma-terms-of-use
4
+ license_link: https://ai.google.dev/gemma/terms
5
+ tags:
6
+ - robotics
7
+ - pi0.5
8
+ - openpi
9
+ - lora
10
+ - tactile
11
+ - xarm
12
+ base_model: physical-intelligence/pi05_droid
13
+ ---
14
+
15
+ # pi05_xarm_cube_binmarker_lora
16
+
17
+ pi0.5 (openpi) LoRA fine-tune for the xArm **cube** task (`"pick up the cube"`),
18
+ trained on frames with the **binmarker** overlay: a fixed-length,
19
+ fixed-direction arrow per finger (sensor-frame pad normal), shown iff that
20
+ finger is in binary contact. NO force magnitude and NO force direction — the
21
+ overlay carries only the per-finger contact bit. Ported from
22
+ LIBERO_contact_overlay's binbars style.
23
+
24
+ ## Training
25
+
26
+ | | |
27
+ |---|---|
28
+ | Base | `pi05_droid` |
29
+ | Recipe | LoRA r16/a16 backbone + r32/a32 action expert, attn+ffn |
30
+ | Batch / LR | 8 / cosine 1e-4 -> 1e-5, 500 warmup, AdamW, grad-clip 1.0 |
31
+ | Data | [EdwardoSunny/xarm_cube_binmarker](https://huggingface.co/datasets/EdwardoSunny/xarm_cube_binmarker) (100 eps, 13103 frames) |
32
+ | Early stop | step 7800 |
33
+ | Final train loss | **0.0429** |
34
+
35
+ Variant comparison for this task (identical protocol, only overlay pixels differ):
36
+ baseline 0.0437 / points9_arrow_len0 0.0435 / **binmarker 0.0429** / points9_arrow 0.0425.
37
+
38
+ ## Contact detection (how the threshold was tuned)
39
+
40
+ Per-finger binary contact from the 3x3x3-axis tactile array:
41
+
42
+ - **Normalization offset** = median over known-idle frames (first 10 frames of
43
+ each episode, post-homing, gripper open). The naive whole-dataset median is
44
+ contact-polluted on tasks that spend most frames in contact (charger: 60%
45
+ closed frames made idle read as a fake magnitude of ~1.2).
46
+ - **Metric** = max over the 9 cells of the normalized cell-vector L2 norm
47
+ ("max-cell"). The cell-sum alternative cannot separate charger idle noise
48
+ (p99.5 = 0.45) from real light holds (0.5 on tube/dishwasher's weaker finger).
49
+ Max-cell: idle p99.5 <= 0.24 on every task, real contacts >= ~0.3.
50
+ - **Hysteresis + debounce** (Schmitt trigger): ON after 2 consecutive frames
51
+ >= 0.25, OFF after 3 consecutive frames < 0.18, state reset at episode
52
+ boundaries. Light-but-real holds (dips to 0.23) stay latched; single-frame
53
+ noise spikes never trigger.
54
+
55
+ Validated on all 400 episodes (zero idle false positives in every post-homing
56
+ window) and **visually verified on 5 episodes per task** (timelines + frame
57
+ montages at approach / contact onset / mid-hold / post-release), plus a final
58
+ spot check on the converted LeRobot frames: no arrows during approach, arrows
59
+ at contact onset, latched through holds, cleared within <=3 frames of release,
60
+ and honest negatives (gripper closed around nothing draws nothing).
61
+
62
+
63
+ ## Contents
64
+
65
+ - `params/` — merged weights (orbax), self-contained for inference
66
+ - `assets/local/xarm_cube_binmarker/norm_stats.json`
67
+ - Config name: `pi05_xarm_cube_binmarker_lora`
68
+
69
+ ## Deployment note
70
+
71
+ At inference, reproduce the SAME overlay on live frames: run the tuned contact
72
+ detector (max-cell metric, ON>=0.25 / OFF<0.18, 2-on/3-off debounce, idle-window
73
+ offset) and draw the fixed-direction marker (FIXED_DIRECTION, arrow_length_scale=0.12,
74
+ thickness 8, dot 22) — see `tactile-data-collection/scripts/render_binmarker.py`
75
+ at commit c64c562 or later (earlier revisions used the measured force direction).