Add droid-toys absjoint checkpoint

README.md

@@ -0,0 +1,63 @@
+---
+license: apache-2.0
+tags:
+  - robotics
+  - vla
+  - pi0
+  - droid
+  - molmobot
+  - lora
+---
+
+# MolmoBot-Pi0 — DROID-toys, absolute joint, r64 LoRA (merged)
+
+Whole-model rank-64 LoRA fine-tune of
+[MolmoBot-Pi0-DROID](https://huggingface.co/shrg7/MolmoBot-Pi0-DROID-equad) on
+[`shrg7/droid-toys`](https://huggingface.co/datasets/shrg7/droid-toys), with
+**absolute joint-position** action targets.
+
+**The LoRA has already been merged into the base weights.** The uploaded
+`model.safetensors` is flat (777 tensors, identical key set to the full-FT
+counterpart) and loads directly into a plain `PI0Pytorch` — no PEFT, no adapter
+folding needed at load time.
+
+## Checkpoint
+
+- **Step 3500**, `val_action_loss = 0.0274`
+- Merged with `scripts/merge_pi0_lora.py` (folds `lora_B @ lora_A * alpha/r`, α/r = 128/64 = 2.0; 333 modules folded)
+- `assets/` holds the norm stats; `metadata.pt` the run metadata
+- Optimizer state is **not** included (inference/eval only, no resume)
+
+## Training
+
+| | |
+|---|---|
+| Config | `molmobot_pi0_lerobot_droid_absjoint` |
+| Dataset | `shrg7/droid-toys` (LeRobot v3.0) — 31 episodes / 6,952 frames @15fps |
+| Action repr | `joint_absolute` — targets are future absolute joint positions `q[t+1]`, 8-dim (7 arm + gripper) |
+| Action horizon | 16 |
+| Cameras | `exterior_1_left` (exo) + `wrist_left`, 224×224 resize-with-pad |
+| LoRA | r=64, α=128, dropout 0.05, targets `q/k/v/o/gate/up/down_proj` across the VLM + action expert |
+| Trainable | 118,112,256 params (3.26% of 3.62B); `vision_tower` frozen |
+| Batch | 32, 1× NVIDIA L40 |
+| LR | 5e-5 constant (300-step warmup; `peak_lr == decay_lr`, so the cosine is flat) |
+| Steps | 3,500 of a planned 8,000 |
+
+### Validation curve
+
+| step | val_action_loss |
+|---|---|
+| 2000 | 0.0331 |
+| 2500 | 0.0313 |
+| 3000 | 0.0273 |
+| 3500 | **0.0274** |
+
+## Caveats
+
+This checkpoint is **under-trained**: the run was cut short by a wall-clock limit
+at 3,500 of 8,000 steps, with the loss curve still descending steeply.
+
+Do **not** read the gap against the full-FT checkpoint
+([`MolmoBot-Pi0-DROID-toys-absjoint-fullft`](https://huggingface.co/shrg7/MolmoBot-Pi0-DROID-toys-absjoint-fullft),
+val 0.0079) as a LoRA-vs-full-FT result. That run completed its full schedule at
+4× the effective batch; the two are not comparable at this point in training.

assets/droid_equad/norm_stats.json

@@ -0,0 +1,88 @@
+{
+  "norm_stats": {
+    "state": {
+      "mean": [
+        0.14381547272205353,
+        -0.01525520347058773,
+        0.22665268182754517,
+        -2.375757932662964,
+        -0.19893218576908112,
+        2.159259557723999,
+        0.7774521112442017,
+        0.24431027472019196
+      ],
+      "std": [
+        0.3617416024208069,
+        0.4120100140571594,
+        0.344979465007782,
+        0.2962265908718109,
+        0.31048065423965454,
+        0.3002902567386627,
+        0.7728461623191833,
+        0.3485460877418518
+      ],
+      "q01": [
+        -0.1332843154668808,
+        -0.5958264470100403,
+        -0.05540205165743828,
+        -2.6645798683166504,
+        -0.5420994758605957,
+        1.7278691530227661,
+        -0.16293670237064362,
+        0.031214039772748947
+      ],
+      "q99": [
+        0.3712266683578491,
+        0.41510656476020813,
+        0.46062949299812317,
+        -2.086469888687134,
+        0.2004101276397705,
+        2.4869449138641357,
+        1.5071415901184082,
+        0.7801601886749268
+      ]
+    },
+    "actions": {
+      "mean": [
+        0.14381547272205353,
+        -0.01525520347058773,
+        0.22665268182754517,
+        -2.375757932662964,
+        -0.19893218576908112,
+        2.159259557723999,
+        0.7774521112442017,
+        0.2847833037376404
+      ],
+      "std": [
+        0.3617416024208069,
+        0.4120100140571594,
+        0.344979465007782,
+        0.2962265908718109,
+        0.31048065423965454,
+        0.3002902567386627,
+        0.7728461623191833,
+        0.41377538442611694
+      ],
+      "q01": [
+        -0.1332843154668808,
+        -0.5958264470100403,
+        -0.05540205165743828,
+        -2.6645798683166504,
+        -0.5420994758605957,
+        1.7278691530227661,
+        -0.16293670237064362,
+        0.023410528898239136
+      ],
+      "q99": [
+        0.3712266683578491,
+        0.41510656476020813,
+        0.46062949299812317,
+        -2.086469888687134,
+        0.2004101276397705,
+        2.4869449138641357,
+        1.5071415901184082,
+        0.9907714128494263
+      ]
+    }
+  }
+}

assets/train_config.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfa4b08627b3e62308730752454bbfd6ae3622b110795e97b4be1933ab299665
+size 1680

metadata.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1623b107268e76bc115df6c917e9d6dd8e914245bc5faf7f017908ff0a94d40d
+size 2931

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a39ba5eef2f8fb40e17de8241b126e1f451c70c8be8ed46f13ca4864bffdde94
+size 7011248800