| --- |
| license: apache-2.0 |
| library_name: onnxruntime |
| tags: |
| - reinforcement-learning |
| - robotics |
| - mujoco |
| - ppo |
| - microduck |
| |
| model-index: |
| - name: Microduck flat-walking PPO seed42 |
| results: |
| - task: |
| type: reinforcement-learning |
| name: Flat-ground locomotion |
| dataset: |
| name: Microduck simulator seeds 100-109 |
| type: microduck-simulator-seeds-100-109 |
| metrics: |
| - type: success_rate_no_termination |
| name: Success rate without termination |
| value: 1.000000 |
| - type: mean_forward_velocity_mps |
| name: Mean forward velocity (m/s) |
| value: 0.258120 |
| - type: mean_reward_per_step |
| name: Mean reward per step |
| value: 0.147762 |
| |
|
|
| --- |
| |
| # Microduck flat-walking PPO — Colab study run |
|
|
| This repository documents a personal study run that installs Pollen Robotics' Microduck simulator, trains a flat-ground walking policy with PPO, exports the normalized policy to ONNX, and validates the release artifacts before upload. |
|
|
| ## Simulation preview |
|
|
| <video controls src="https://huggingface.co/arabellako22/microduck-walk-seed42/resolve/main/media/simulation_success.mp4" width="640"></video> |
|
|
| [Download the simulation video](media/simulation_success.mp4) |
|
|
| ## Training setup |
|
|
| - Task: `Mjlab-Velocity-Flat-MicroDuck` |
| - Algorithm: PPO via `rsl_rl` |
| - Simulator: `mjlab` / MuJoCo Warp |
| - Parallel environments: 4096 |
| - Training iterations: 4000 |
| - Seed: 42 |
| - Source revision: [`29e887ecfbf5`](https://github.com/pollen-robotics/microduck_rl/commit/29e887ecfbf5d37144759e5a9f8a176dfb83d547) |
| - Export: official `scripts/export.py`; observation normalizer baked into ONNX |
|
|
| ## Validation |
|
|
| - ONNX contract: `[1, 61] -> [1, 14]` |
| - Publisher smoke test: finite and non-constant output |
| - Simulation video: 8.00 seconds, rendered with `egl` |
| - Commanded forward velocity: 0.50 m/s |
| - Measured mean forward velocity: 0.247 m/s |
| - Terminations during rollout: 0 |
| - Policy SHA-256: `82a2369fe642eda505ee6d60c7ed26aafd177eb7484030834cd04e5a53035213` |
| - Video SHA-256: `c73cb27596c2a7c87282e7bcb0583647cc44539928d18e74f8ce8bfbc04704ee` |
|
|
| The video was reviewed before publication as a qualitative success check. This run has not been tested on a physical Microduck. One video and one seed are not a statistical performance benchmark. |
|
|
|
|
| <!-- MULTI_SEED_EVAL_START --> |
| ## Multi-seed headless evaluation |
| |
| The uploaded `policy.onnx` was evaluated without rendering across 10 deterministic evaluation seeds. This tests one policy trained with seed 42; it is not a multiple-training-seed study. |
| |
| | Eval seed | Mean reward/step | Mean velocity (m/s) | Mean absolute velocity error (m/s) | Terminations | Success | |
| |---:|---:|---:|---:|---:|:---:| |
| | 100 | 0.1413 | 0.236 | 0.264 | 0 | ✅ | |
| | 101 | 0.1487 | 0.271 | 0.232 | 0 | ✅ | |
| | 102 | 0.1460 | 0.225 | 0.276 | 0 | ✅ | |
| | 103 | 0.1482 | 0.291 | 0.212 | 0 | ✅ | |
| | 104 | 0.1444 | 0.227 | 0.274 | 0 | ✅ | |
| | 105 | 0.1447 | 0.221 | 0.279 | 0 | ✅ | |
| | 106 | 0.1480 | 0.276 | 0.228 | 0 | ✅ | |
| | 107 | 0.1524 | 0.250 | 0.250 | 0 | ✅ | |
| | 108 | 0.1500 | 0.290 | 0.215 | 0 | ✅ | |
| | 109 | 0.1540 | 0.295 | 0.207 | 0 | ✅ | |
| |
| ### Aggregate results |
| |
| | Metric | Result | |
| |---|---:| |
| | Evaluation seeds | 10 (`100`–`109`) | |
| | Steps per seed | 400 | |
| | Mean reward per step | 0.1478 ± 0.0036 | |
| | Mean forward velocity | 0.258 ± 0.028 m/s | |
| | Mean absolute velocity error | 0.244 ± 0.027 m/s | |
| | Total terminations | 0 | |
| | Success rate without termination | 100.0% | |
| |
| Raw, machine-readable results: [`evaluation.json`](evaluation.json) and [`evaluation_by_seed.csv`](evaluation_by_seed.csv). No new video was generated because the policy did not change. |
| |
| <!-- MULTI_SEED_EVAL_END --> |
|
|
| ## Files |
|
|
| - `policy.onnx` — deployable policy with observation normalization |
| - `manifest.json` — Microduck policy manifest |
| - `training_config.json` — reproducibility settings |
| - `validation.json` — release checks and hashes |
| - `render_metrics.json` — finite-rollout reward, velocity, and termination summary |
| - `media/simulation_success.mp4` — qualitative simulation result |
|
|
| ## Local inference |
|
|
| ```bash |
| uv run scripts/infer_policy.py --walking policy.onnx --new-cmd-obs |
| ``` |
|
|
| ## References |
|
|
| - [Pollen Robotics microduck_rl](https://github.com/pollen-robotics/microduck_rl) |
| - [PPO](https://arxiv.org/abs/1707.06347) |
| - [BAM servo friction models](https://arxiv.org/abs/2410.08650) |
| - [Dynamics randomization](https://arxiv.org/abs/1710.06537) |
| - [Massively parallel locomotion learning](https://arxiv.org/abs/2109.11978) |
|
|
