Upload folder using huggingface_hub

README.md

@@ -0,0 +1,255 @@
+---
+license: apache-2.0
+library_name: lerobot
+pipeline_tag: robotics
+tags:
+  - robotics
+  - lerobot
+  - imitation-learning
+  - act
+  - b-spline
+  - so101
+  - bspline_act
+datasets:
+  - atharva-pantheon/pick_block_bowl
+model_name: pickblock_bspline_act
+---
+
+# pickblock_bspline_act
+
+An [ACT](https://huggingface.co/papers/2304.13705) policy that predicts **B-spline
+trajectory segments** instead of discrete action chunks, trained on SO-101 black-cube
+picking. This is the "Reg.+BSP" variant from
+[B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations](https://arxiv.org/abs/2607.09648).
+
+The policy outputs a curve, so executing `a(n·t)` replays the same trajectory geometry
+n times faster without retraining. Speed-up is an inference flag, verified exact to
+0.000e+00 deviation at 2× and 4×.
+
+> [!WARNING]
+> **This model is trained on 13 episodes and does not generalize well.** Held-out error
+> is 22.8° against 6.2° on training data — a 3.7× gap that never closed across 40,000
+> steps. It has never been run on hardware. Treat it as a pipeline validation artifact,
+> not a working policy. See [Evaluation](#evaluation) and [Limitations](#limitations).
+
+## Download
+
+```bash
+hf download Dimios45/pickblock_bspline_act --local-dir ./pickblock_bspline_act
+```
+
+The repository root holds the recommended checkpoint, so `--policy.path` also accepts
+the repo id directly. Individual checkpoints live under `checkpoints/<step>/`.
+
+## Run it
+
+```bash
+lerobot-rollout \
+    --strategy.type=base \
+    --policy.path=./pickblock_bspline_act/checkpoints/010000/pretrained_model \
+    --policy.speed_up=1.0 \
+    --device=cuda \
+    --robot.type=so101_follower \
+    --robot.port=/dev/ttyACM0 --robot.id=FOLLOWER \
+    --robot.cameras='{front: {type: opencv, index_or_path: 0, width: 640, height: 360, fps: 30}, ee: {type: opencv, index_or_path: 4, width: 640, height: 360, fps: 30}}' \
+    --task="Grab the black cube" \
+    --fps=30 --duration=60
+```
+
+`--policy.path` takes either a Hub repo id (`namespace/name`) or a **local directory
+containing `config.json`** — which is the `pretrained_model/` subdirectory, not the
+step directory above it. There is no `--policy.subfolder` flag.
+
+### Two things differ from the cup-stacking model — check both
+
+**Resolution is 640×360, not 640×480.** The source demonstrations were recorded at
+1920×1080; they were downscaled to 640×360 for training, preserving the 16:9 aspect
+ratio. Your cameras must be configured to match.
+
+**The cameras are `front` and `ee`**, not `front` and `gripper`:
+
+| Key | Shape |
+|---|---|
+| `observation.images.front` | `(3, 360, 640)` |
+| `observation.images.ee` | `(3, 360, 640)` |
+| `observation.state` | `(6,)` |
+
+Joint order is `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper`,
+in degrees. Swapping the two cameras produces confident but wrong behaviour rather than
+an error.
+
+To try the speed-up, raise `--policy.speed_up` to 2.0 or 4.0. The paper reaches 4× on
+cube picking specifically, so this task should tolerate it better than precise-placement
+tasks — but validate at 1× first, and note this policy has not been shown to complete
+the task at any speed.
+
+### Requirements
+
+This policy type is not in upstream LeRobot. You need a checkout containing the
+`bspline_act` policy, installed with the scipy extra:
+
+```bash
+uv pip install -e ".[feetech,bspline]"
+```
+
+`scipy>=1.15` is required for `interpolate.generate_knots`.
+
+Unnormalization statistics are stored as buffers inside `model.safetensors`, so each
+checkpoint is self-contained — you do not need the training dataset on the robot machine.
+
+## Checkpoints
+
+| Step | Epoch | Train error | **Held-out error** | Eval loss | |
+|---|---|---|---|---|---|
+| **10000** | 58 | 6.19° | **22.81°** | **0.3903** | **recommended — best** |
+| 20000 | 116 | 4.67° | 24.15° | 0.3986 | |
+| 30000 | 174 | 3.90° | 24.77° | 0.3996 | |
+| 40000 | 234 | 3.44° | 26.22° | 0.4030 | final, worst held-out |
+
+Held-out error gets **monotonically worse** after step 10,000 while training error keeps
+falling. Use 10000. Later checkpoints are included only for reproducibility — they are
+strictly worse policies.
+
+Error is mean absolute joint error in degrees, worst joint, over each predicted
+segment's horizon (~0.76 s), measured by decoding the spline and comparing against
+ground-truth actions.
+
+## Training
+
+| | |
+|---|---|
+| Steps | 40,000 |
+| Batch size | 32 |
+| Epochs | 234 |
+| Dataset | [`atharva-pantheon/pick_block_bowl`](https://huggingface.co/datasets/atharva-pantheon/pick_block_bowl) → downscaled → idle-trimmed → B-spline converted |
+| Episodes | **13 train / 3 held out** |
+| Frames | 6,422 (from 10,342; 37.9% boundary idle removed) |
+| Optimizer | AdamW, lr 2e-5 (backbone 1e-5), wd 1e-3 |
+| Precision | bf16 autocast + `channels_last` |
+| Augmentation | ColorJitter (brightness/contrast/saturation/hue/sharpness), up to 3 per frame |
+| Hardware | 1× RTX 4090, ~2 h 45 min total |
+| Seed | 1000 |
+
+Augmentation and 10× weight decay were added specifically to combat overfitting. They
+slowed the fit to training data and **moved held-out loss by 0.0003**. An unaugmented
+control run is available on request; the two are indistinguishable on unseen data.
+
+### Preprocessing
+
+Source video is 1920×1080 AV1, which is unusable for ACT directly — a ResNet18
+stride-32 feature map is 34×60, putting over 4,000 tokens into the transformer encoder
+across two cameras. Transcoded to 640×360 H.264, which also sped up random-access
+decode from 40 to 183 frames/s per worker.
+
+Leading and trailing motionless frames were then removed: 46 s of lead-in and 27 s of
+trailing idle. This dataset is unusually idle-heavy — 37.9% of frames — including
+20 s and 32 s dead stretches at the ends of two episodes where recording continued
+after the task finished.
+
+Motion is measured as peak-to-peak joint range over a ±0.5 s window rather than
+frame-to-frame delta, because isolated jitter samples inside a dead stretch defeat a
+delta test and silently leave the idle in.
+
+## Architecture
+
+Standard ACT with one change: the decoder emits a B-spline **parameter matrix** rather
+than an action chunk.
+
+```
+(n_knots, 1 + action_dim) = (16, 7) = 112 values per prediction
+   column 0   knot vector, in source-frame units, 0 = "now"
+   columns 1: control points, one per joint
+```
+
+| | |
+|---|---|
+| Params | 51.6 M |
+| Vision backbone | ResNet18 (ImageNet init, fine-tuned) |
+| dim_model / chunk_size | 512 / 16 |
+| VAE | enabled, kl_weight 10.0 |
+| B-spline degree | 3 (cubic, C² continuous) |
+| bspline_chunk_size | 10 |
+| Fitting tolerance ε | 0.2 (degrees) |
+| Max knot span | 6 frames |
+
+The paper uses a DINOv2 encoder for its regression variant; this port uses LeRobot's
+ResNet18, which is what LeRobot's ACT supports. The paper reports no backbone ablation,
+and holds the encoder fixed between its baseline and its B-spline variant, so none of
+its claims depend on that choice.
+
+Segments span 0.50–1.10 s (0.76 s mean). The network runs only when a segment is
+exhausted, which decouples policy rate from control rate.
+
+### On `max_knot_span`
+
+This dataset is why the flag exists. Adaptive knot fitting covers a motionless stretch
+with one enormous knot span, and uncapped fitting here produced segments with a **p95
+of 18.4 seconds** — the policy would commit to eighteen seconds of open-loop motion
+without looking at a camera. Capping span width at 6 frames bounds that at
+`chunk_size × 6` = 2 s, costing 0.04× compression.
+
+After correct idle-trimming the uncapped p95 drops to 1.17 s on its own, so the dead
+recording time was the real cause. The cap remains as a cheap safety rail.
+
+## Evaluation
+
+**Offline only. No hardware evaluation has been performed.**
+
+| | Train (13 eps) | **Held-out (3 eps)** |
+|---|---|---|
+| Best checkpoint (10k) | 6.19° | **22.81°** |
+| Final (40k) | 3.44° | 26.22° |
+
+Reference points:
+
+- **Inter-demonstration spread: 54.83°** — how far apart two human demonstrations of
+  this task are, phase-aligned. At 22.8° the policy is well inside that, so it is
+  extracting real per-episode information from the cameras rather than replaying an
+  average trajectory. This is the one genuinely positive result.
+- **Hold-still baseline: 16.23°.** The policy scores worse than this. Over a 0.76 s
+  horizon on a slow task, "don't move" is a decent short-horizon *predictor* while
+  being a useless *policy*, so this is not the damning comparison it looks like — but
+  combined with the widening train/eval gap it is not encouraging either.
+
+Held-out loss plateaued at ~0.39 by step 2,000 and never improved across the remaining
+38,000 steps.
+
+### Why the held-out number is a pessimistic bound
+
+The split holds out the last 3 of 16 episodes. Inspecting the raw recording:
+
+- 8 of 15 episode boundaries are contiguous to under 1° (several to exactly 0.00°), so
+  episodes were largely cut from a continuous teleoperation stream. Adjacent episodes
+  are correlated, meaning an interleaved split would leak.
+- There is a 97.67° discontinuity between episodes 9 and 10, and episodes 10, 13, 14
+  and 15 all end with the gripper open (8–16°) while the other twelve end closed (~2°).
+- **All three held-out episodes sit after that change point**, and two of them *start*
+  with the gripper open — a state the model never sees at t=0 in training.
+
+So the tail split is biased pessimistic and an interleaved split would be biased
+optimistic. With 16 semi-continuous episodes there is no clean holdout. 22.8° is a
+lower bound on quality, not a verdict.
+
+## Limitations
+
+- **13 training episodes is too few for this task.** This is the dominant limitation and
+  no hyperparameter fixed it. 40–50 demonstrations with varied cube placement and an
+  **independent reset between takes** would address both the training and the
+  measurement problem.
+- **No hardware evaluation.** Task success rate is unknown, and given the above it
+  should not be assumed to be non-zero.
+- **Held-out split is confounded** — see above.
+- **Resolution and camera names differ** from the cup-stacking model. Check both.
+- **Camera assignment is silent when wrong.**
+
+## Citation
+
+```bibtex
+@article{han2026b,
+  title={B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations},
+  author={Han, Xiaoshen and Xiong, Haoyu and Chen, Haonan and Liu, Chaoqi and
+          Torralba, Antonio and Zhu, Yuke and Du, Yilun},
+  journal={arXiv preprint arXiv:2607.09648},
+  year={2026}
+}
+```

checkpoints/010000/pretrained_model/config.json

@@ -0,0 +1,83 @@
+{
+    "type": "bspline_act",
+    "n_obs_steps": 1,
+    "input_features": {
+        "observation.state": {
+            "type": "STATE",
+            "shape": [
+                6
+            ]
+        },
+        "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        },
+        "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        }
+    },
+    "output_features": {
+        "action": {
+            "type": "ACTION",
+            "shape": [
+                112
+            ]
+        }
+    },
+    "device": "cuda",
+    "use_amp": false,
+    "use_peft": false,
+    "push_to_hub": false,
+    "repo_id": null,
+    "private": null,
+    "tags": null,
+    "license": null,
+    "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+    "pretrained_revision": null,
+    "chunk_size": 16,
+    "n_action_steps": 16,
+    "normalization_mapping": {
+        "VISUAL": "MEAN_STD",
+        "STATE": "MEAN_STD",
+        "ACTION": "MEAN_STD"
+    },
+    "vision_backbone": "resnet18",
+    "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+    "replace_final_stride_with_dilation": 0,
+    "pre_norm": false,
+    "dim_model": 512,
+    "n_heads": 8,
+    "dim_feedforward": 3200,
+    "feedforward_activation": "relu",
+    "n_encoder_layers": 4,
+    "n_decoder_layers": 1,
+    "use_vae": true,
+    "latent_dim": 32,
+    "n_vae_encoder_layers": 4,
+    "temporal_ensemble_coeff": null,
+    "dropout": 0.1,
+    "kl_weight": 10.0,
+    "optimizer_lr": 2e-05,
+    "optimizer_weight_decay": 0.001,
+    "optimizer_lr_backbone": 1e-05,
+    "bspline_degree": 3,
+    "bspline_chunk_size": 10,
+    "relative_knots": false,
+    "speed_up": 1.0,
+    "n_decoded_steps": 32,
+    "align_segments": true,
+    "align_error_threshold": 0.5,
+    "align_max_fraction": 0.2,
+    "align_exclude_last_dims": 1,
+    "replan_margin": 1.0,
+    "channels_last": true
+}

checkpoints/010000/pretrained_model/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e16d5b72180ba0ecf13a77f9a81b69b3f252d9ca457941ff211c19b65859d85
+size 206360820

checkpoints/010000/pretrained_model/policy_postprocessor.json

@@ -0,0 +1,12 @@
+{
+  "name": "policy_postprocessor",
+  "steps": [
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cpu",
+        "float_dtype": null
+      }
+    }
+  ]
+}

checkpoints/010000/pretrained_model/policy_preprocessor.json

@@ -0,0 +1,64 @@
+{
+  "name": "policy_preprocessor",
+  "steps": [
+    {
+      "registry_name": "rename_observations_processor",
+      "config": {
+        "rename_map": {}
+      }
+    },
+    {
+      "registry_name": "to_batch_processor",
+      "config": {}
+    },
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cuda",
+        "float_dtype": null
+      }
+    },
+    {
+      "registry_name": "normalizer_processor",
+      "config": {
+        "eps": 1e-08,
+        "features": {
+          "observation.state": {
+            "type": "STATE",
+            "shape": [
+              6
+            ]
+          },
+          "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "action": {
+            "type": "ACTION",
+            "shape": [
+              112
+            ]
+          }
+        },
+        "norm_map": {
+          "VISUAL": "MEAN_STD",
+          "STATE": "MEAN_STD",
+          "ACTION": "MEAN_STD"
+        }
+      },
+      "state_file": "policy_preprocessor_step_3_normalizer_processor.safetensors"
+    }
+  ]
+}

checkpoints/010000/pretrained_model/policy_preprocessor_step_3_normalizer_processor.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b1863e4f75643077d48e73b9a90b50bec2e8a27fb3cb5d60195ac41d06ef48e
+size 11544

checkpoints/010000/pretrained_model/train_config.json

@@ -0,0 +1,230 @@
+{
+    "dataset": {
+        "repo_id": "aryankakad/pick_block_bowl_bspline",
+        "root": null,
+        "episodes": null,
+        "image_transforms": {
+            "enable": true,
+            "max_num_transforms": 3,
+            "random_order": false,
+            "tfs": {
+                "brightness": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "brightness": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "contrast": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "contrast": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "saturation": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "saturation": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "hue": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "hue": [
+                            -0.05,
+                            0.05
+                        ]
+                    }
+                },
+                "sharpness": {
+                    "weight": 1.0,
+                    "type": "SharpnessJitter",
+                    "kwargs": {
+                        "sharpness": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "affine": {
+                    "weight": 1.0,
+                    "type": "RandomAffine",
+                    "kwargs": {
+                        "degrees": [
+                            -5.0,
+                            5.0
+                        ],
+                        "translate": [
+                            0.05,
+                            0.05
+                        ]
+                    }
+                }
+            }
+        },
+        "revision": null,
+        "use_imagenet_stats": true,
+        "video_backend": "torchcodec",
+        "return_uint8": false,
+        "depth_output_unit": "mm",
+        "streaming": false,
+        "eval_split": 0.15
+    },
+    "env": null,
+    "policy": {
+        "type": "bspline_act",
+        "n_obs_steps": 1,
+        "input_features": {
+            "observation.state": {
+                "type": "STATE",
+                "shape": [
+                    6
+                ]
+            },
+            "observation.images.ee": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            },
+            "observation.images.front": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            }
+        },
+        "output_features": {
+            "action": {
+                "type": "ACTION",
+                "shape": [
+                    112
+                ]
+            }
+        },
+        "device": "cuda",
+        "use_amp": false,
+        "use_peft": false,
+        "push_to_hub": false,
+        "repo_id": null,
+        "private": null,
+        "tags": null,
+        "license": null,
+        "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+        "pretrained_revision": null,
+        "chunk_size": 16,
+        "n_action_steps": 16,
+        "normalization_mapping": {
+            "VISUAL": "MEAN_STD",
+            "STATE": "MEAN_STD",
+            "ACTION": "MEAN_STD"
+        },
+        "vision_backbone": "resnet18",
+        "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+        "replace_final_stride_with_dilation": 0,
+        "pre_norm": false,
+        "dim_model": 512,
+        "n_heads": 8,
+        "dim_feedforward": 3200,
+        "feedforward_activation": "relu",
+        "n_encoder_layers": 4,
+        "n_decoder_layers": 1,
+        "use_vae": true,
+        "latent_dim": 32,
+        "n_vae_encoder_layers": 4,
+        "temporal_ensemble_coeff": null,
+        "dropout": 0.1,
+        "kl_weight": 10.0,
+        "optimizer_lr": 2e-05,
+        "optimizer_weight_decay": 0.001,
+        "optimizer_lr_backbone": 1e-05,
+        "bspline_degree": 3,
+        "bspline_chunk_size": 10,
+        "relative_knots": false,
+        "speed_up": 1.0,
+        "n_decoded_steps": 32,
+        "align_segments": true,
+        "align_error_threshold": 0.5,
+        "align_max_fraction": 0.2,
+        "align_exclude_last_dims": 1,
+        "replan_margin": 1.0,
+        "channels_last": true
+    },
+    "reward_model": null,
+    "output_dir": "outputs/pickblock_bspline_act_aug",
+    "job_name": "bspline_act",
+    "resume": true,
+    "seed": 1000,
+    "cudnn_deterministic": false,
+    "num_workers": 8,
+    "batch_size": 32,
+    "prefetch_factor": 4,
+    "persistent_workers": true,
+    "steps": 40000,
+    "env_eval_freq": 20000,
+    "log_freq": 250,
+    "eval_steps": 1000,
+    "max_eval_samples": 512,
+    "tolerance_s": 0.0001,
+    "save_checkpoint": true,
+    "save_freq": 5000,
+    "use_policy_training_preset": true,
+    "optimizer": {
+        "type": "adamw",
+        "lr": 2e-05,
+        "weight_decay": 0.001,
+        "grad_clip_norm": 10.0,
+        "betas": [
+            0.9,
+            0.999
+        ],
+        "eps": 1e-08
+    },
+    "scheduler": null,
+    "eval": {
+        "n_episodes": 50,
+        "batch_size": 22,
+        "use_async_envs": true,
+        "recording": false,
+        "recording_repo_id": null,
+        "recording_private": false
+    },
+    "wandb": {
+        "enable": false,
+        "disable_artifact": false,
+        "project": "lerobot",
+        "entity": null,
+        "notes": null,
+        "run_id": null,
+        "mode": null,
+        "add_tags": true
+    },
+    "peft": null,
+    "job": {
+        "target": null,
+        "image": "huggingface/lerobot-gpu:latest",
+        "timeout": "2d",
+        "detach": false,
+        "tags": []
+    },
+    "save_checkpoint_to_hub": false,
+    "sample_weighting": null,
+    "rename_map": {},
+    "checkpoint_path": "outputs/pickblock_bspline_act_aug/checkpoints/last"
+}

checkpoints/020000/pretrained_model/config.json

@@ -0,0 +1,83 @@
+{
+    "type": "bspline_act",
+    "n_obs_steps": 1,
+    "input_features": {
+        "observation.state": {
+            "type": "STATE",
+            "shape": [
+                6
+            ]
+        },
+        "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        },
+        "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        }
+    },
+    "output_features": {
+        "action": {
+            "type": "ACTION",
+            "shape": [
+                112
+            ]
+        }
+    },
+    "device": "cuda",
+    "use_amp": false,
+    "use_peft": false,
+    "push_to_hub": false,
+    "repo_id": null,
+    "private": null,
+    "tags": null,
+    "license": null,
+    "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+    "pretrained_revision": null,
+    "chunk_size": 16,
+    "n_action_steps": 16,
+    "normalization_mapping": {
+        "VISUAL": "MEAN_STD",
+        "STATE": "MEAN_STD",
+        "ACTION": "MEAN_STD"
+    },
+    "vision_backbone": "resnet18",
+    "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+    "replace_final_stride_with_dilation": 0,
+    "pre_norm": false,
+    "dim_model": 512,
+    "n_heads": 8,
+    "dim_feedforward": 3200,
+    "feedforward_activation": "relu",
+    "n_encoder_layers": 4,
+    "n_decoder_layers": 1,
+    "use_vae": true,
+    "latent_dim": 32,
+    "n_vae_encoder_layers": 4,
+    "temporal_ensemble_coeff": null,
+    "dropout": 0.1,
+    "kl_weight": 10.0,
+    "optimizer_lr": 2e-05,
+    "optimizer_weight_decay": 0.001,
+    "optimizer_lr_backbone": 1e-05,
+    "bspline_degree": 3,
+    "bspline_chunk_size": 10,
+    "relative_knots": false,
+    "speed_up": 1.0,
+    "n_decoded_steps": 32,
+    "align_segments": true,
+    "align_error_threshold": 0.5,
+    "align_max_fraction": 0.2,
+    "align_exclude_last_dims": 1,
+    "replan_margin": 1.0,
+    "channels_last": true
+}

checkpoints/020000/pretrained_model/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c062e889da46ea6a81ef9a03d9cbddbe6a3d29f0d6ba9ab60538b9a7f9cc1e73
+size 206360820

checkpoints/020000/pretrained_model/policy_postprocessor.json

@@ -0,0 +1,12 @@
+{
+  "name": "policy_postprocessor",
+  "steps": [
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cpu",
+        "float_dtype": null
+      }
+    }
+  ]
+}

checkpoints/020000/pretrained_model/policy_preprocessor.json

@@ -0,0 +1,64 @@
+{
+  "name": "policy_preprocessor",
+  "steps": [
+    {
+      "registry_name": "rename_observations_processor",
+      "config": {
+        "rename_map": {}
+      }
+    },
+    {
+      "registry_name": "to_batch_processor",
+      "config": {}
+    },
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cuda",
+        "float_dtype": null
+      }
+    },
+    {
+      "registry_name": "normalizer_processor",
+      "config": {
+        "eps": 1e-08,
+        "features": {
+          "observation.state": {
+            "type": "STATE",
+            "shape": [
+              6
+            ]
+          },
+          "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "action": {
+            "type": "ACTION",
+            "shape": [
+              112
+            ]
+          }
+        },
+        "norm_map": {
+          "VISUAL": "MEAN_STD",
+          "STATE": "MEAN_STD",
+          "ACTION": "MEAN_STD"
+        }
+      },
+      "state_file": "policy_preprocessor_step_3_normalizer_processor.safetensors"
+    }
+  ]
+}

checkpoints/020000/pretrained_model/policy_preprocessor_step_3_normalizer_processor.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b1863e4f75643077d48e73b9a90b50bec2e8a27fb3cb5d60195ac41d06ef48e
+size 11544

checkpoints/020000/pretrained_model/train_config.json

@@ -0,0 +1,230 @@
+{
+    "dataset": {
+        "repo_id": "aryankakad/pick_block_bowl_bspline",
+        "root": null,
+        "episodes": null,
+        "image_transforms": {
+            "enable": true,
+            "max_num_transforms": 3,
+            "random_order": false,
+            "tfs": {
+                "brightness": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "brightness": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "contrast": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "contrast": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "saturation": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "saturation": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "hue": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "hue": [
+                            -0.05,
+                            0.05
+                        ]
+                    }
+                },
+                "sharpness": {
+                    "weight": 1.0,
+                    "type": "SharpnessJitter",
+                    "kwargs": {
+                        "sharpness": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "affine": {
+                    "weight": 1.0,
+                    "type": "RandomAffine",
+                    "kwargs": {
+                        "degrees": [
+                            -5.0,
+                            5.0
+                        ],
+                        "translate": [
+                            0.05,
+                            0.05
+                        ]
+                    }
+                }
+            }
+        },
+        "revision": null,
+        "use_imagenet_stats": true,
+        "video_backend": "torchcodec",
+        "return_uint8": false,
+        "depth_output_unit": "mm",
+        "streaming": false,
+        "eval_split": 0.15
+    },
+    "env": null,
+    "policy": {
+        "type": "bspline_act",
+        "n_obs_steps": 1,
+        "input_features": {
+            "observation.state": {
+                "type": "STATE",
+                "shape": [
+                    6
+                ]
+            },
+            "observation.images.ee": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            },
+            "observation.images.front": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            }
+        },
+        "output_features": {
+            "action": {
+                "type": "ACTION",
+                "shape": [
+                    112
+                ]
+            }
+        },
+        "device": "cuda",
+        "use_amp": false,
+        "use_peft": false,
+        "push_to_hub": false,
+        "repo_id": null,
+        "private": null,
+        "tags": null,
+        "license": null,
+        "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+        "pretrained_revision": null,
+        "chunk_size": 16,
+        "n_action_steps": 16,
+        "normalization_mapping": {
+            "VISUAL": "MEAN_STD",
+            "STATE": "MEAN_STD",
+            "ACTION": "MEAN_STD"
+        },
+        "vision_backbone": "resnet18",
+        "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+        "replace_final_stride_with_dilation": 0,
+        "pre_norm": false,
+        "dim_model": 512,
+        "n_heads": 8,
+        "dim_feedforward": 3200,
+        "feedforward_activation": "relu",
+        "n_encoder_layers": 4,
+        "n_decoder_layers": 1,
+        "use_vae": true,
+        "latent_dim": 32,
+        "n_vae_encoder_layers": 4,
+        "temporal_ensemble_coeff": null,
+        "dropout": 0.1,
+        "kl_weight": 10.0,
+        "optimizer_lr": 2e-05,
+        "optimizer_weight_decay": 0.001,
+        "optimizer_lr_backbone": 1e-05,
+        "bspline_degree": 3,
+        "bspline_chunk_size": 10,
+        "relative_knots": false,
+        "speed_up": 1.0,
+        "n_decoded_steps": 32,
+        "align_segments": true,
+        "align_error_threshold": 0.5,
+        "align_max_fraction": 0.2,
+        "align_exclude_last_dims": 1,
+        "replan_margin": 1.0,
+        "channels_last": true
+    },
+    "reward_model": null,
+    "output_dir": "outputs/pickblock_bspline_act_aug",
+    "job_name": "bspline_act",
+    "resume": true,
+    "seed": 1000,
+    "cudnn_deterministic": false,
+    "num_workers": 8,
+    "batch_size": 32,
+    "prefetch_factor": 4,
+    "persistent_workers": true,
+    "steps": 40000,
+    "env_eval_freq": 20000,
+    "log_freq": 250,
+    "eval_steps": 1000,
+    "max_eval_samples": 512,
+    "tolerance_s": 0.0001,
+    "save_checkpoint": true,
+    "save_freq": 5000,
+    "use_policy_training_preset": true,
+    "optimizer": {
+        "type": "adamw",
+        "lr": 2e-05,
+        "weight_decay": 0.001,
+        "grad_clip_norm": 10.0,
+        "betas": [
+            0.9,
+            0.999
+        ],
+        "eps": 1e-08
+    },
+    "scheduler": null,
+    "eval": {
+        "n_episodes": 50,
+        "batch_size": 22,
+        "use_async_envs": true,
+        "recording": false,
+        "recording_repo_id": null,
+        "recording_private": false
+    },
+    "wandb": {
+        "enable": false,
+        "disable_artifact": false,
+        "project": "lerobot",
+        "entity": null,
+        "notes": null,
+        "run_id": null,
+        "mode": null,
+        "add_tags": true
+    },
+    "peft": null,
+    "job": {
+        "target": null,
+        "image": "huggingface/lerobot-gpu:latest",
+        "timeout": "2d",
+        "detach": false,
+        "tags": []
+    },
+    "save_checkpoint_to_hub": false,
+    "sample_weighting": null,
+    "rename_map": {},
+    "checkpoint_path": "outputs/pickblock_bspline_act_aug/checkpoints/last"
+}

checkpoints/030000/pretrained_model/config.json

@@ -0,0 +1,83 @@
+{
+    "type": "bspline_act",
+    "n_obs_steps": 1,
+    "input_features": {
+        "observation.state": {
+            "type": "STATE",
+            "shape": [
+                6
+            ]
+        },
+        "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        },
+        "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        }
+    },
+    "output_features": {
+        "action": {
+            "type": "ACTION",
+            "shape": [
+                112
+            ]
+        }
+    },
+    "device": "cuda",
+    "use_amp": false,
+    "use_peft": false,
+    "push_to_hub": false,
+    "repo_id": null,
+    "private": null,
+    "tags": null,
+    "license": null,
+    "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+    "pretrained_revision": null,
+    "chunk_size": 16,
+    "n_action_steps": 16,
+    "normalization_mapping": {
+        "VISUAL": "MEAN_STD",
+        "STATE": "MEAN_STD",
+        "ACTION": "MEAN_STD"
+    },
+    "vision_backbone": "resnet18",
+    "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+    "replace_final_stride_with_dilation": 0,
+    "pre_norm": false,
+    "dim_model": 512,
+    "n_heads": 8,
+    "dim_feedforward": 3200,
+    "feedforward_activation": "relu",
+    "n_encoder_layers": 4,
+    "n_decoder_layers": 1,
+    "use_vae": true,
+    "latent_dim": 32,
+    "n_vae_encoder_layers": 4,
+    "temporal_ensemble_coeff": null,
+    "dropout": 0.1,
+    "kl_weight": 10.0,
+    "optimizer_lr": 2e-05,
+    "optimizer_weight_decay": 0.001,
+    "optimizer_lr_backbone": 1e-05,
+    "bspline_degree": 3,
+    "bspline_chunk_size": 10,
+    "relative_knots": false,
+    "speed_up": 1.0,
+    "n_decoded_steps": 32,
+    "align_segments": true,
+    "align_error_threshold": 0.5,
+    "align_max_fraction": 0.2,
+    "align_exclude_last_dims": 1,
+    "replan_margin": 1.0,
+    "channels_last": true
+}

checkpoints/030000/pretrained_model/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad487ea70d1fe034680f950378c0453f4cf9a2cccfefb7631816054297363f35
+size 206360820

checkpoints/030000/pretrained_model/policy_postprocessor.json

@@ -0,0 +1,12 @@
+{
+  "name": "policy_postprocessor",
+  "steps": [
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cpu",
+        "float_dtype": null
+      }
+    }
+  ]
+}

checkpoints/030000/pretrained_model/policy_preprocessor.json

@@ -0,0 +1,64 @@
+{
+  "name": "policy_preprocessor",
+  "steps": [
+    {
+      "registry_name": "rename_observations_processor",
+      "config": {
+        "rename_map": {}
+      }
+    },
+    {
+      "registry_name": "to_batch_processor",
+      "config": {}
+    },
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cuda",
+        "float_dtype": null
+      }
+    },
+    {
+      "registry_name": "normalizer_processor",
+      "config": {
+        "eps": 1e-08,
+        "features": {
+          "observation.state": {
+            "type": "STATE",
+            "shape": [
+              6
+            ]
+          },
+          "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "action": {
+            "type": "ACTION",
+            "shape": [
+              112
+            ]
+          }
+        },
+        "norm_map": {
+          "VISUAL": "MEAN_STD",
+          "STATE": "MEAN_STD",
+          "ACTION": "MEAN_STD"
+        }
+      },
+      "state_file": "policy_preprocessor_step_3_normalizer_processor.safetensors"
+    }
+  ]
+}

checkpoints/030000/pretrained_model/policy_preprocessor_step_3_normalizer_processor.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b1863e4f75643077d48e73b9a90b50bec2e8a27fb3cb5d60195ac41d06ef48e
+size 11544

checkpoints/030000/pretrained_model/train_config.json

@@ -0,0 +1,230 @@
+{
+    "dataset": {
+        "repo_id": "aryankakad/pick_block_bowl_bspline",
+        "root": null,
+        "episodes": null,
+        "image_transforms": {
+            "enable": true,
+            "max_num_transforms": 3,
+            "random_order": false,
+            "tfs": {
+                "brightness": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "brightness": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "contrast": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "contrast": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "saturation": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "saturation": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "hue": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "hue": [
+                            -0.05,
+                            0.05
+                        ]
+                    }
+                },
+                "sharpness": {
+                    "weight": 1.0,
+                    "type": "SharpnessJitter",
+                    "kwargs": {
+                        "sharpness": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "affine": {
+                    "weight": 1.0,
+                    "type": "RandomAffine",
+                    "kwargs": {
+                        "degrees": [
+                            -5.0,
+                            5.0
+                        ],
+                        "translate": [
+                            0.05,
+                            0.05
+                        ]
+                    }
+                }
+            }
+        },
+        "revision": null,
+        "use_imagenet_stats": true,
+        "video_backend": "torchcodec",
+        "return_uint8": false,
+        "depth_output_unit": "mm",
+        "streaming": false,
+        "eval_split": 0.15
+    },
+    "env": null,
+    "policy": {
+        "type": "bspline_act",
+        "n_obs_steps": 1,
+        "input_features": {
+            "observation.state": {
+                "type": "STATE",
+                "shape": [
+                    6
+                ]
+            },
+            "observation.images.ee": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            },
+            "observation.images.front": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            }
+        },
+        "output_features": {
+            "action": {
+                "type": "ACTION",
+                "shape": [
+                    112
+                ]
+            }
+        },
+        "device": "cuda",
+        "use_amp": false,
+        "use_peft": false,
+        "push_to_hub": false,
+        "repo_id": null,
+        "private": null,
+        "tags": null,
+        "license": null,
+        "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+        "pretrained_revision": null,
+        "chunk_size": 16,
+        "n_action_steps": 16,
+        "normalization_mapping": {
+            "VISUAL": "MEAN_STD",
+            "STATE": "MEAN_STD",
+            "ACTION": "MEAN_STD"
+        },
+        "vision_backbone": "resnet18",
+        "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+        "replace_final_stride_with_dilation": 0,
+        "pre_norm": false,
+        "dim_model": 512,
+        "n_heads": 8,
+        "dim_feedforward": 3200,
+        "feedforward_activation": "relu",
+        "n_encoder_layers": 4,
+        "n_decoder_layers": 1,
+        "use_vae": true,
+        "latent_dim": 32,
+        "n_vae_encoder_layers": 4,
+        "temporal_ensemble_coeff": null,
+        "dropout": 0.1,
+        "kl_weight": 10.0,
+        "optimizer_lr": 2e-05,
+        "optimizer_weight_decay": 0.001,
+        "optimizer_lr_backbone": 1e-05,
+        "bspline_degree": 3,
+        "bspline_chunk_size": 10,
+        "relative_knots": false,
+        "speed_up": 1.0,
+        "n_decoded_steps": 32,
+        "align_segments": true,
+        "align_error_threshold": 0.5,
+        "align_max_fraction": 0.2,
+        "align_exclude_last_dims": 1,
+        "replan_margin": 1.0,
+        "channels_last": true
+    },
+    "reward_model": null,
+    "output_dir": "outputs/pickblock_bspline_act_aug",
+    "job_name": "bspline_act",
+    "resume": true,
+    "seed": 1000,
+    "cudnn_deterministic": false,
+    "num_workers": 8,
+    "batch_size": 32,
+    "prefetch_factor": 4,
+    "persistent_workers": true,
+    "steps": 40000,
+    "env_eval_freq": 20000,
+    "log_freq": 250,
+    "eval_steps": 1000,
+    "max_eval_samples": 512,
+    "tolerance_s": 0.0001,
+    "save_checkpoint": true,
+    "save_freq": 5000,
+    "use_policy_training_preset": true,
+    "optimizer": {
+        "type": "adamw",
+        "lr": 2e-05,
+        "weight_decay": 0.001,
+        "grad_clip_norm": 10.0,
+        "betas": [
+            0.9,
+            0.999
+        ],
+        "eps": 1e-08
+    },
+    "scheduler": null,
+    "eval": {
+        "n_episodes": 50,
+        "batch_size": 22,
+        "use_async_envs": true,
+        "recording": false,
+        "recording_repo_id": null,
+        "recording_private": false
+    },
+    "wandb": {
+        "enable": false,
+        "disable_artifact": false,
+        "project": "lerobot",
+        "entity": null,
+        "notes": null,
+        "run_id": null,
+        "mode": null,
+        "add_tags": true
+    },
+    "peft": null,
+    "job": {
+        "target": null,
+        "image": "huggingface/lerobot-gpu:latest",
+        "timeout": "2d",
+        "detach": false,
+        "tags": []
+    },
+    "save_checkpoint_to_hub": false,
+    "sample_weighting": null,
+    "rename_map": {},
+    "checkpoint_path": "outputs/pickblock_bspline_act_aug/checkpoints/last"
+}

checkpoints/040000/pretrained_model/config.json

@@ -0,0 +1,83 @@
+{
+    "type": "bspline_act",
+    "n_obs_steps": 1,
+    "input_features": {
+        "observation.state": {
+            "type": "STATE",
+            "shape": [
+                6
+            ]
+        },
+        "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        },
+        "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        }
+    },
+    "output_features": {
+        "action": {
+            "type": "ACTION",
+            "shape": [
+                112
+            ]
+        }
+    },
+    "device": "cuda",
+    "use_amp": false,
+    "use_peft": false,
+    "push_to_hub": false,
+    "repo_id": null,
+    "private": null,
+    "tags": null,
+    "license": null,
+    "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+    "pretrained_revision": null,
+    "chunk_size": 16,
+    "n_action_steps": 16,
+    "normalization_mapping": {
+        "VISUAL": "MEAN_STD",
+        "STATE": "MEAN_STD",
+        "ACTION": "MEAN_STD"
+    },
+    "vision_backbone": "resnet18",
+    "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+    "replace_final_stride_with_dilation": 0,
+    "pre_norm": false,
+    "dim_model": 512,
+    "n_heads": 8,
+    "dim_feedforward": 3200,
+    "feedforward_activation": "relu",
+    "n_encoder_layers": 4,
+    "n_decoder_layers": 1,
+    "use_vae": true,
+    "latent_dim": 32,
+    "n_vae_encoder_layers": 4,
+    "temporal_ensemble_coeff": null,
+    "dropout": 0.1,
+    "kl_weight": 10.0,
+    "optimizer_lr": 2e-05,
+    "optimizer_weight_decay": 0.001,
+    "optimizer_lr_backbone": 1e-05,
+    "bspline_degree": 3,
+    "bspline_chunk_size": 10,
+    "relative_knots": false,
+    "speed_up": 1.0,
+    "n_decoded_steps": 32,
+    "align_segments": true,
+    "align_error_threshold": 0.5,
+    "align_max_fraction": 0.2,
+    "align_exclude_last_dims": 1,
+    "replan_margin": 1.0,
+    "channels_last": true
+}

checkpoints/040000/pretrained_model/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3205af457d14f2629e276a3371abc925ea5304c1e74eeb9444eadcca01dcb27d
+size 206360820

checkpoints/040000/pretrained_model/policy_postprocessor.json

@@ -0,0 +1,12 @@
+{
+  "name": "policy_postprocessor",
+  "steps": [
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cpu",
+        "float_dtype": null
+      }
+    }
+  ]
+}

checkpoints/040000/pretrained_model/policy_preprocessor.json

@@ -0,0 +1,64 @@
+{
+  "name": "policy_preprocessor",
+  "steps": [
+    {
+      "registry_name": "rename_observations_processor",
+      "config": {
+        "rename_map": {}
+      }
+    },
+    {
+      "registry_name": "to_batch_processor",
+      "config": {}
+    },
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cuda",
+        "float_dtype": null
+      }
+    },
+    {
+      "registry_name": "normalizer_processor",
+      "config": {
+        "eps": 1e-08,
+        "features": {
+          "observation.state": {
+            "type": "STATE",
+            "shape": [
+              6
+            ]
+          },
+          "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "action": {
+            "type": "ACTION",
+            "shape": [
+              112
+            ]
+          }
+        },
+        "norm_map": {
+          "VISUAL": "MEAN_STD",
+          "STATE": "MEAN_STD",
+          "ACTION": "MEAN_STD"
+        }
+      },
+      "state_file": "policy_preprocessor_step_3_normalizer_processor.safetensors"
+    }
+  ]
+}

checkpoints/040000/pretrained_model/policy_preprocessor_step_3_normalizer_processor.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b1863e4f75643077d48e73b9a90b50bec2e8a27fb3cb5d60195ac41d06ef48e
+size 11544

checkpoints/040000/pretrained_model/train_config.json

@@ -0,0 +1,230 @@
+{
+    "dataset": {
+        "repo_id": "aryankakad/pick_block_bowl_bspline",
+        "root": null,
+        "episodes": null,
+        "image_transforms": {
+            "enable": true,
+            "max_num_transforms": 3,
+            "random_order": false,
+            "tfs": {
+                "brightness": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "brightness": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "contrast": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "contrast": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "saturation": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "saturation": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "hue": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "hue": [
+                            -0.05,
+                            0.05
+                        ]
+                    }
+                },
+                "sharpness": {
+                    "weight": 1.0,
+                    "type": "SharpnessJitter",
+                    "kwargs": {
+                        "sharpness": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "affine": {
+                    "weight": 1.0,
+                    "type": "RandomAffine",
+                    "kwargs": {
+                        "degrees": [
+                            -5.0,
+                            5.0
+                        ],
+                        "translate": [
+                            0.05,
+                            0.05
+                        ]
+                    }
+                }
+            }
+        },
+        "revision": null,
+        "use_imagenet_stats": true,
+        "video_backend": "torchcodec",
+        "return_uint8": false,
+        "depth_output_unit": "mm",
+        "streaming": false,
+        "eval_split": 0.15
+    },
+    "env": null,
+    "policy": {
+        "type": "bspline_act",
+        "n_obs_steps": 1,
+        "input_features": {
+            "observation.state": {
+                "type": "STATE",
+                "shape": [
+                    6
+                ]
+            },
+            "observation.images.ee": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            },
+            "observation.images.front": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            }
+        },
+        "output_features": {
+            "action": {
+                "type": "ACTION",
+                "shape": [
+                    112
+                ]
+            }
+        },
+        "device": "cuda",
+        "use_amp": false,
+        "use_peft": false,
+        "push_to_hub": false,
+        "repo_id": null,
+        "private": null,
+        "tags": null,
+        "license": null,
+        "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+        "pretrained_revision": null,
+        "chunk_size": 16,
+        "n_action_steps": 16,
+        "normalization_mapping": {
+            "VISUAL": "MEAN_STD",
+            "STATE": "MEAN_STD",
+            "ACTION": "MEAN_STD"
+        },
+        "vision_backbone": "resnet18",
+        "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+        "replace_final_stride_with_dilation": 0,
+        "pre_norm": false,
+        "dim_model": 512,
+        "n_heads": 8,
+        "dim_feedforward": 3200,
+        "feedforward_activation": "relu",
+        "n_encoder_layers": 4,
+        "n_decoder_layers": 1,
+        "use_vae": true,
+        "latent_dim": 32,
+        "n_vae_encoder_layers": 4,
+        "temporal_ensemble_coeff": null,
+        "dropout": 0.1,
+        "kl_weight": 10.0,
+        "optimizer_lr": 2e-05,
+        "optimizer_weight_decay": 0.001,
+        "optimizer_lr_backbone": 1e-05,
+        "bspline_degree": 3,
+        "bspline_chunk_size": 10,
+        "relative_knots": false,
+        "speed_up": 1.0,
+        "n_decoded_steps": 32,
+        "align_segments": true,
+        "align_error_threshold": 0.5,
+        "align_max_fraction": 0.2,
+        "align_exclude_last_dims": 1,
+        "replan_margin": 1.0,
+        "channels_last": true
+    },
+    "reward_model": null,
+    "output_dir": "outputs/pickblock_bspline_act_aug",
+    "job_name": "bspline_act",
+    "resume": true,
+    "seed": 1000,
+    "cudnn_deterministic": false,
+    "num_workers": 8,
+    "batch_size": 32,
+    "prefetch_factor": 4,
+    "persistent_workers": true,
+    "steps": 40000,
+    "env_eval_freq": 20000,
+    "log_freq": 250,
+    "eval_steps": 1000,
+    "max_eval_samples": 512,
+    "tolerance_s": 0.0001,
+    "save_checkpoint": true,
+    "save_freq": 5000,
+    "use_policy_training_preset": true,
+    "optimizer": {
+        "type": "adamw",
+        "lr": 2e-05,
+        "weight_decay": 0.001,
+        "grad_clip_norm": 10.0,
+        "betas": [
+            0.9,
+            0.999
+        ],
+        "eps": 1e-08
+    },
+    "scheduler": null,
+    "eval": {
+        "n_episodes": 50,
+        "batch_size": 22,
+        "use_async_envs": true,
+        "recording": false,
+        "recording_repo_id": null,
+        "recording_private": false
+    },
+    "wandb": {
+        "enable": false,
+        "disable_artifact": false,
+        "project": "lerobot",
+        "entity": null,
+        "notes": null,
+        "run_id": null,
+        "mode": null,
+        "add_tags": true
+    },
+    "peft": null,
+    "job": {
+        "target": null,
+        "image": "huggingface/lerobot-gpu:latest",
+        "timeout": "2d",
+        "detach": false,
+        "tags": []
+    },
+    "save_checkpoint_to_hub": false,
+    "sample_weighting": null,
+    "rename_map": {},
+    "checkpoint_path": "outputs/pickblock_bspline_act_aug/checkpoints/last"
+}

config.json

@@ -0,0 +1,83 @@
+{
+    "type": "bspline_act",
+    "n_obs_steps": 1,
+    "input_features": {
+        "observation.state": {
+            "type": "STATE",
+            "shape": [
+                6
+            ]
+        },
+        "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        },
+        "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+                3,
+                360,
+                640
+            ]
+        }
+    },
+    "output_features": {
+        "action": {
+            "type": "ACTION",
+            "shape": [
+                112
+            ]
+        }
+    },
+    "device": "cuda",
+    "use_amp": false,
+    "use_peft": false,
+    "push_to_hub": false,
+    "repo_id": null,
+    "private": null,
+    "tags": null,
+    "license": null,
+    "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+    "pretrained_revision": null,
+    "chunk_size": 16,
+    "n_action_steps": 16,
+    "normalization_mapping": {
+        "VISUAL": "MEAN_STD",
+        "STATE": "MEAN_STD",
+        "ACTION": "MEAN_STD"
+    },
+    "vision_backbone": "resnet18",
+    "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+    "replace_final_stride_with_dilation": 0,
+    "pre_norm": false,
+    "dim_model": 512,
+    "n_heads": 8,
+    "dim_feedforward": 3200,
+    "feedforward_activation": "relu",
+    "n_encoder_layers": 4,
+    "n_decoder_layers": 1,
+    "use_vae": true,
+    "latent_dim": 32,
+    "n_vae_encoder_layers": 4,
+    "temporal_ensemble_coeff": null,
+    "dropout": 0.1,
+    "kl_weight": 10.0,
+    "optimizer_lr": 2e-05,
+    "optimizer_weight_decay": 0.001,
+    "optimizer_lr_backbone": 1e-05,
+    "bspline_degree": 3,
+    "bspline_chunk_size": 10,
+    "relative_knots": false,
+    "speed_up": 1.0,
+    "n_decoded_steps": 32,
+    "align_segments": true,
+    "align_error_threshold": 0.5,
+    "align_max_fraction": 0.2,
+    "align_exclude_last_dims": 1,
+    "replan_margin": 1.0,
+    "channels_last": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e16d5b72180ba0ecf13a77f9a81b69b3f252d9ca457941ff211c19b65859d85
+size 206360820

policy_postprocessor.json

@@ -0,0 +1,12 @@
+{
+  "name": "policy_postprocessor",
+  "steps": [
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cpu",
+        "float_dtype": null
+      }
+    }
+  ]
+}

policy_preprocessor.json

@@ -0,0 +1,64 @@
+{
+  "name": "policy_preprocessor",
+  "steps": [
+    {
+      "registry_name": "rename_observations_processor",
+      "config": {
+        "rename_map": {}
+      }
+    },
+    {
+      "registry_name": "to_batch_processor",
+      "config": {}
+    },
+    {
+      "registry_name": "device_processor",
+      "config": {
+        "device": "cuda",
+        "float_dtype": null
+      }
+    },
+    {
+      "registry_name": "normalizer_processor",
+      "config": {
+        "eps": 1e-08,
+        "features": {
+          "observation.state": {
+            "type": "STATE",
+            "shape": [
+              6
+            ]
+          },
+          "observation.images.ee": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "observation.images.front": {
+            "type": "VISUAL",
+            "shape": [
+              3,
+              360,
+              640
+            ]
+          },
+          "action": {
+            "type": "ACTION",
+            "shape": [
+              112
+            ]
+          }
+        },
+        "norm_map": {
+          "VISUAL": "MEAN_STD",
+          "STATE": "MEAN_STD",
+          "ACTION": "MEAN_STD"
+        }
+      },
+      "state_file": "policy_preprocessor_step_3_normalizer_processor.safetensors"
+    }
+  ]
+}

policy_preprocessor_step_3_normalizer_processor.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b1863e4f75643077d48e73b9a90b50bec2e8a27fb3cb5d60195ac41d06ef48e
+size 11544

train_config.json

@@ -0,0 +1,230 @@
+{
+    "dataset": {
+        "repo_id": "aryankakad/pick_block_bowl_bspline",
+        "root": null,
+        "episodes": null,
+        "image_transforms": {
+            "enable": true,
+            "max_num_transforms": 3,
+            "random_order": false,
+            "tfs": {
+                "brightness": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "brightness": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "contrast": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "contrast": [
+                            0.8,
+                            1.2
+                        ]
+                    }
+                },
+                "saturation": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "saturation": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "hue": {
+                    "weight": 1.0,
+                    "type": "ColorJitter",
+                    "kwargs": {
+                        "hue": [
+                            -0.05,
+                            0.05
+                        ]
+                    }
+                },
+                "sharpness": {
+                    "weight": 1.0,
+                    "type": "SharpnessJitter",
+                    "kwargs": {
+                        "sharpness": [
+                            0.5,
+                            1.5
+                        ]
+                    }
+                },
+                "affine": {
+                    "weight": 1.0,
+                    "type": "RandomAffine",
+                    "kwargs": {
+                        "degrees": [
+                            -5.0,
+                            5.0
+                        ],
+                        "translate": [
+                            0.05,
+                            0.05
+                        ]
+                    }
+                }
+            }
+        },
+        "revision": null,
+        "use_imagenet_stats": true,
+        "video_backend": "torchcodec",
+        "return_uint8": false,
+        "depth_output_unit": "mm",
+        "streaming": false,
+        "eval_split": 0.15
+    },
+    "env": null,
+    "policy": {
+        "type": "bspline_act",
+        "n_obs_steps": 1,
+        "input_features": {
+            "observation.state": {
+                "type": "STATE",
+                "shape": [
+                    6
+                ]
+            },
+            "observation.images.ee": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            },
+            "observation.images.front": {
+                "type": "VISUAL",
+                "shape": [
+                    3,
+                    360,
+                    640
+                ]
+            }
+        },
+        "output_features": {
+            "action": {
+                "type": "ACTION",
+                "shape": [
+                    112
+                ]
+            }
+        },
+        "device": "cuda",
+        "use_amp": false,
+        "use_peft": false,
+        "push_to_hub": false,
+        "repo_id": null,
+        "private": null,
+        "tags": null,
+        "license": null,
+        "pretrained_path": "outputs/pickblock_bspline_act_aug/checkpoints/last/pretrained_model",
+        "pretrained_revision": null,
+        "chunk_size": 16,
+        "n_action_steps": 16,
+        "normalization_mapping": {
+            "VISUAL": "MEAN_STD",
+            "STATE": "MEAN_STD",
+            "ACTION": "MEAN_STD"
+        },
+        "vision_backbone": "resnet18",
+        "pretrained_backbone_weights": "ResNet18_Weights.IMAGENET1K_V1",
+        "replace_final_stride_with_dilation": 0,
+        "pre_norm": false,
+        "dim_model": 512,
+        "n_heads": 8,
+        "dim_feedforward": 3200,
+        "feedforward_activation": "relu",
+        "n_encoder_layers": 4,
+        "n_decoder_layers": 1,
+        "use_vae": true,
+        "latent_dim": 32,
+        "n_vae_encoder_layers": 4,
+        "temporal_ensemble_coeff": null,
+        "dropout": 0.1,
+        "kl_weight": 10.0,
+        "optimizer_lr": 2e-05,
+        "optimizer_weight_decay": 0.001,
+        "optimizer_lr_backbone": 1e-05,
+        "bspline_degree": 3,
+        "bspline_chunk_size": 10,
+        "relative_knots": false,
+        "speed_up": 1.0,
+        "n_decoded_steps": 32,
+        "align_segments": true,
+        "align_error_threshold": 0.5,
+        "align_max_fraction": 0.2,
+        "align_exclude_last_dims": 1,
+        "replan_margin": 1.0,
+        "channels_last": true
+    },
+    "reward_model": null,
+    "output_dir": "outputs/pickblock_bspline_act_aug",
+    "job_name": "bspline_act",
+    "resume": true,
+    "seed": 1000,
+    "cudnn_deterministic": false,
+    "num_workers": 8,
+    "batch_size": 32,
+    "prefetch_factor": 4,
+    "persistent_workers": true,
+    "steps": 40000,
+    "env_eval_freq": 20000,
+    "log_freq": 250,
+    "eval_steps": 1000,
+    "max_eval_samples": 512,
+    "tolerance_s": 0.0001,
+    "save_checkpoint": true,
+    "save_freq": 5000,
+    "use_policy_training_preset": true,
+    "optimizer": {
+        "type": "adamw",
+        "lr": 2e-05,
+        "weight_decay": 0.001,
+        "grad_clip_norm": 10.0,
+        "betas": [
+            0.9,
+            0.999
+        ],
+        "eps": 1e-08
+    },
+    "scheduler": null,
+    "eval": {
+        "n_episodes": 50,
+        "batch_size": 22,
+        "use_async_envs": true,
+        "recording": false,
+        "recording_repo_id": null,
+        "recording_private": false
+    },
+    "wandb": {
+        "enable": false,
+        "disable_artifact": false,
+        "project": "lerobot",
+        "entity": null,
+        "notes": null,
+        "run_id": null,
+        "mode": null,
+        "add_tags": true
+    },
+    "peft": null,
+    "job": {
+        "target": null,
+        "image": "huggingface/lerobot-gpu:latest",
+        "timeout": "2d",
+        "detach": false,
+        "tags": []
+    },
+    "save_checkpoint_to_hub": false,
+    "sample_weighting": null,
+    "rename_map": {},
+    "checkpoint_path": "outputs/pickblock_bspline_act_aug/checkpoints/last"
+}