model card

README.md

@@ -0,0 +1,53 @@
+---
+license: cc-by-nc-4.0
+library_name: lerobot
+pipeline_tag: robotics
+tags:
+  - robotics
+  - lerobot
+  - flow-matching
+  - diffusion-transformer
+  - imitation-learning
+  - ur7e
+  - end-effector
+---
+
+# Flow-Matching DiT · carrot-in-pot · EEF-delta (state 16 / action 7) — checkpoint 60k
+
+Text-conditioned flow-matching policy (`multi_task_dit`, CLIP text encoder, Euler ODE) trained on
+the **real** UR7e *"Put carrot in pot"* demonstrations (54 GELLO-teleop takes, 30 fps) in the
+**EEF-delta action space** (`eef_delta_v1`). This is the **60k-step checkpoint** (best open-loop
+chunk-30 error of 10k..100k; kanu kept only 100k).
+
+## Observation / action space (`eef_delta_v1`)
+- `observation.state` **16-D** = `[q1..q6 (rad, UR order), tcp_x, tcp_y, tcp_z (m, base_link),
+  r11, r21, r31, r12, r22, r32 (first two columns of the TCP rotation), grip_pos (0=open..1=closed)]`.
+  TCP = `ur_kin.fk(q)` + 0.174 m along flange +Z.
+- `action` **7-D** = `[dx, dy, dz, drx, dry, drz, grip_cmd]` — achieved per-frame TCP motion
+  (`dp = p_{t+1}-p_t`, `drot = so3_log(R_{t+1} R_t^T)`, base frame), gripper absolute 0..1.
+  Deploy: `p_target = p_live + dp`, `R_target = so3_exp(drot) R_live`, analytic IK, branch locking.
+- Cameras: `cam1` (scene), `cam2` (wrist), native 720×1280 fed in; the policy resizes internally
+  to `image_resize_shape = [224, 224]` — **do not pre-resize** (double-resize silently degrades).
+- Task string: `"Put carrot in pot"` (CLIP-conditioned; send it every tick).
+- `chunk_size 32`, `n_action_steps 24`, `n_obs_steps 2`, `num_integration_steps` 100 at train
+  time (10 is the usual serving override), DiT hidden 512 × 6 layers.
+
+## Training
+- Dataset: `carrot_in_pot_eef_lerobot_v3` — local LeRobot v3 re-export of
+  [`Bigenlight/carrot_in_pot_lerobot_v3`](https://huggingface.co/datasets/Bigenlight/carrot_in_pot_lerobot_v3)
+  (54 ep / 17,085 frames; joints de-lagged by per-take τ≈0.90 s + linear re-interpolation).
+  Not yet on the Hub.
+- `lerobot-train`, batch 8, seed 1000, 100k steps (10 h 15 m on one RTX A4000, kanu), job `fm_carrot_eef`.
+
+## Held-out results (open-loop, chunk-30 cumulative error, episodes 48–53)
+| 10k | 20k | 30k | **60k (this)** | 100k | zero-motion |
+|---|---|---|---|---|---|
+| 34.8 mm | 32.8 mm | ~33 mm | **32.0 mm** | 32.6 mm | 65.6 mm |
+
+lerobot's `eval_loss` rose monotonically 0.096→0.233 over the run while the open-loop metric
+kept improving — it was ignored for checkpoint selection.
+
+## Status
+Real-robot closed-loop evaluation: **not yet run**. The shipped ZMQ servers
+(`gello_policy/policy_server/fm_server.py`) are joint-space 7/7 and refuse this checkpoint's
+16-D state; serving needs the EEF-mode deploy node from branch `feat/carrot-eef-il`.