File size: 4,529 Bytes
172afdd 8cc704a 172afdd 8cc704a 172afdd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | ---
license: apache-2.0
library_name: onnxruntime
tags:
- reinforcement-learning
- robotics
- mujoco
- ppo
- microduck
# MULTI_SEED_MODEL_INDEX_START
model-index:
- name: Microduck flat-walking PPO seed42
results:
- task:
type: reinforcement-learning
name: Flat-ground locomotion
dataset:
name: Microduck simulator seeds 100-109
type: microduck-simulator-seeds-100-109
metrics:
- type: success_rate_no_termination
name: Success rate without termination
value: 1.000000
- type: mean_forward_velocity_mps
name: Mean forward velocity (m/s)
value: 0.258120
- type: mean_reward_per_step
name: Mean reward per step
value: 0.147762
# MULTI_SEED_MODEL_INDEX_END
---
# Microduck flat-walking PPO — Colab study run
This repository documents a personal study run that installs Pollen Robotics' Microduck simulator, trains a flat-ground walking policy with PPO, exports the normalized policy to ONNX, and validates the release artifacts before upload.
## Simulation preview
<video controls src="https://huggingface.co/arabellako22/microduck-walk-seed42/resolve/main/media/simulation_success.mp4" width="640"></video>
[Download the simulation video](media/simulation_success.mp4)
## Training setup
- Task: `Mjlab-Velocity-Flat-MicroDuck`
- Algorithm: PPO via `rsl_rl`
- Simulator: `mjlab` / MuJoCo Warp
- Parallel environments: 4096
- Training iterations: 4000
- Seed: 42
- Source revision: [`29e887ecfbf5`](https://github.com/pollen-robotics/microduck_rl/commit/29e887ecfbf5d37144759e5a9f8a176dfb83d547)
- Export: official `scripts/export.py`; observation normalizer baked into ONNX
## Validation
- ONNX contract: `[1, 61] -> [1, 14]`
- Publisher smoke test: finite and non-constant output
- Simulation video: 8.00 seconds, rendered with `egl`
- Commanded forward velocity: 0.50 m/s
- Measured mean forward velocity: 0.247 m/s
- Terminations during rollout: 0
- Policy SHA-256: `82a2369fe642eda505ee6d60c7ed26aafd177eb7484030834cd04e5a53035213`
- Video SHA-256: `c73cb27596c2a7c87282e7bcb0583647cc44539928d18e74f8ce8bfbc04704ee`
The video was reviewed before publication as a qualitative success check. This run has not been tested on a physical Microduck. One video and one seed are not a statistical performance benchmark.
<!-- MULTI_SEED_EVAL_START -->
## Multi-seed headless evaluation
The uploaded `policy.onnx` was evaluated without rendering across 10 deterministic evaluation seeds. This tests one policy trained with seed 42; it is not a multiple-training-seed study.
| Eval seed | Mean reward/step | Mean velocity (m/s) | Mean absolute velocity error (m/s) | Terminations | Success |
|---:|---:|---:|---:|---:|:---:|
| 100 | 0.1413 | 0.236 | 0.264 | 0 | ✅ |
| 101 | 0.1487 | 0.271 | 0.232 | 0 | ✅ |
| 102 | 0.1460 | 0.225 | 0.276 | 0 | ✅ |
| 103 | 0.1482 | 0.291 | 0.212 | 0 | ✅ |
| 104 | 0.1444 | 0.227 | 0.274 | 0 | ✅ |
| 105 | 0.1447 | 0.221 | 0.279 | 0 | ✅ |
| 106 | 0.1480 | 0.276 | 0.228 | 0 | ✅ |
| 107 | 0.1524 | 0.250 | 0.250 | 0 | ✅ |
| 108 | 0.1500 | 0.290 | 0.215 | 0 | ✅ |
| 109 | 0.1540 | 0.295 | 0.207 | 0 | ✅ |
### Aggregate results
| Metric | Result |
|---|---:|
| Evaluation seeds | 10 (`100`–`109`) |
| Steps per seed | 400 |
| Mean reward per step | 0.1478 ± 0.0036 |
| Mean forward velocity | 0.258 ± 0.028 m/s |
| Mean absolute velocity error | 0.244 ± 0.027 m/s |
| Total terminations | 0 |
| Success rate without termination | 100.0% |
Raw, machine-readable results: [`evaluation.json`](evaluation.json) and [`evaluation_by_seed.csv`](evaluation_by_seed.csv). No new video was generated because the policy did not change.
<!-- MULTI_SEED_EVAL_END -->
## Files
- `policy.onnx` — deployable policy with observation normalization
- `manifest.json` — Microduck policy manifest
- `training_config.json` — reproducibility settings
- `validation.json` — release checks and hashes
- `render_metrics.json` — finite-rollout reward, velocity, and termination summary
- `media/simulation_success.mp4` — qualitative simulation result
## Local inference
```bash
uv run scripts/infer_policy.py --walking policy.onnx --new-cmd-obs
```
## References
- [Pollen Robotics microduck_rl](https://github.com/pollen-robotics/microduck_rl)
- [PPO](https://arxiv.org/abs/1707.06347)
- [BAM servo friction models](https://arxiv.org/abs/2410.08650)
- [Dynamics randomization](https://arxiv.org/abs/1710.06537)
- [Massively parallel locomotion learning](https://arxiv.org/abs/2109.11978)
|