Add 10-seed headless ONNX evaluation

README.md

@@ -7,6 +7,28 @@ tags:
 - mujoco
 - ppo
 - microduck
+# MULTI_SEED_MODEL_INDEX_START
+model-index:
+- name: Microduck flat-walking PPO seed42
+  results:
+  - task:
+      type: reinforcement-learning
+      name: Flat-ground locomotion
+    dataset:
+      name: Microduck simulator seeds 100-109
+      type: microduck-simulator-seeds-100-109
+    metrics:
+    - type: success_rate_no_termination
+      name: Success rate without termination
+      value: 1.000000
+    - type: mean_forward_velocity_mps
+      name: Mean forward velocity (m/s)
+      value: 0.258120
+    - type: mean_reward_per_step
+      name: Mean reward per step
+      value: 0.147762
+# MULTI_SEED_MODEL_INDEX_END
+
 ---
 
 # Microduck flat-walking PPO — Colab study run
@@ -43,6 +65,41 @@ This repository documents a personal study run that installs Pollen Robotics' Mi
 
 The video was reviewed before publication as a qualitative success check. This run has not been tested on a physical Microduck. One video and one seed are not a statistical performance benchmark.
 
+
+<!-- MULTI_SEED_EVAL_START -->
+## Multi-seed headless evaluation
+
+The uploaded `policy.onnx` was evaluated without rendering across 10 deterministic evaluation seeds. This tests one policy trained with seed 42; it is not a multiple-training-seed study.
+
+| Eval seed | Mean reward/step | Mean velocity (m/s) | Mean absolute velocity error (m/s) | Terminations | Success |
+|---:|---:|---:|---:|---:|:---:|
+| 100 | 0.1413 | 0.236 | 0.264 | 0 | ✅ |
+| 101 | 0.1487 | 0.271 | 0.232 | 0 | ✅ |
+| 102 | 0.1460 | 0.225 | 0.276 | 0 | ✅ |
+| 103 | 0.1482 | 0.291 | 0.212 | 0 | ✅ |
+| 104 | 0.1444 | 0.227 | 0.274 | 0 | ✅ |
+| 105 | 0.1447 | 0.221 | 0.279 | 0 | ✅ |
+| 106 | 0.1480 | 0.276 | 0.228 | 0 | ✅ |
+| 107 | 0.1524 | 0.250 | 0.250 | 0 | ✅ |
+| 108 | 0.1500 | 0.290 | 0.215 | 0 | ✅ |
+| 109 | 0.1540 | 0.295 | 0.207 | 0 | ✅ |
+
+### Aggregate results
+
+| Metric | Result |
+|---|---:|
+| Evaluation seeds | 10 (`100`–`109`) |
+| Steps per seed | 400 |
+| Mean reward per step | 0.1478 ± 0.0036 |
+| Mean forward velocity | 0.258 ± 0.028 m/s |
+| Mean absolute velocity error | 0.244 ± 0.027 m/s |
+| Total terminations | 0 |
+| Success rate without termination | 100.0% |
+
+Raw, machine-readable results: [`evaluation.json`](evaluation.json) and [`evaluation_by_seed.csv`](evaluation_by_seed.csv). No new video was generated because the policy did not change.
+
+<!-- MULTI_SEED_EVAL_END -->
+
 ## Files
 
 - `policy.onnx` — deployable policy with observation normalization

evaluation.json

@@ -0,0 +1,148 @@
+{
+  "schema_version": 1,
+  "evaluation_type": "single_trained_policy_multiple_evaluation_seeds",
+  "generated_at_utc": "2026-09-07T03:01:09.935197+00:00",
+  "policy_file": "policy.onnx",
+  "policy_sha256": "82a2369fe642eda505ee6d60c7ed26aafd177eb7484030834cd04e5a53035213",
+  "training_seed": 42,
+  "source_checkpoint": "model_3999.pt",
+  "protocol": {
+    "task_id": "Mjlab-Velocity-Flat-MicroDuck",
+    "evaluation_seeds": [
+      100,
+      101,
+      102,
+      103,
+      104,
+      105,
+      106,
+      107,
+      108,
+      109
+    ],
+    "num_seeds": 10,
+    "steps_per_seed": 400,
+    "commanded_forward_velocity_mps": 0.5,
+    "rendering": false,
+    "video_generated": false,
+    "policy_runtime": "onnxruntime-cpu",
+    "actions": "deterministic"
+  },
+  "per_seed": [
+    {
+      "seed": 100,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14128163651563228,
+      "mean_forward_velocity_mps": 0.23589461453258992,
+      "mean_absolute_velocity_error_mps": 0.2643511279672384,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 101,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14869166431948544,
+      "mean_forward_velocity_mps": 0.27104998455848545,
+      "mean_absolute_velocity_error_mps": 0.23166746033821256,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 102,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14602043539285658,
+      "mean_forward_velocity_mps": 0.2245279321074486,
+      "mean_absolute_velocity_error_mps": 0.2757411253452301,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 103,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14819012811407448,
+      "mean_forward_velocity_mps": 0.29148244574753335,
+      "mean_absolute_velocity_error_mps": 0.2116375322631211,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 104,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14435075111687184,
+      "mean_forward_velocity_mps": 0.22676853544602638,
+      "mean_absolute_velocity_error_mps": 0.273739144433639,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 105,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14471292811096645,
+      "mean_forward_velocity_mps": 0.22117613712325693,
+      "mean_absolute_velocity_error_mps": 0.2788238628767431,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 106,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14796128492802382,
+      "mean_forward_velocity_mps": 0.2755377550085541,
+      "mean_absolute_velocity_error_mps": 0.22814512956538238,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 107,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.15238978615030646,
+      "mean_forward_velocity_mps": 0.24962244590977206,
+      "mean_absolute_velocity_error_mps": 0.2504566774680279,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 108,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.14997595064342023,
+      "mean_forward_velocity_mps": 0.28972968977410346,
+      "mean_absolute_velocity_error_mps": 0.21514052021782845,
+      "termination_count": 0,
+      "success_no_termination": true
+    },
+    {
+      "seed": 109,
+      "rollout_steps": 400,
+      "commanded_forward_velocity_mps": 0.5,
+      "mean_reward_per_step": 0.1540444320999086,
+      "mean_forward_velocity_mps": 0.29541337831818965,
+      "mean_absolute_velocity_error_mps": 0.2067692495981464,
+      "termination_count": 0,
+      "success_no_termination": true
+    }
+  ],
+  "summary": {
+    "mean_reward_per_step": 0.1477618997391546,
+    "std_reward_per_step_population": 0.0036446650806754564,
+    "mean_forward_velocity_mps": 0.258120291852596,
+    "std_forward_velocity_mps_population": 0.02831245410335961,
+    "mean_absolute_velocity_error_mps": 0.24364718300735694,
+    "std_absolute_velocity_error_mps_population": 0.026886847320684554,
+    "total_terminations": 0,
+    "successful_seeds_no_termination": 10,
+    "success_rate_no_termination": 1.0
+  },
+  "limitations": [
+    "This evaluates one policy trained with seed 42; it does not measure training-seed variance.",
+    "The test covers one fixed forward command and simulation only, not physical hardware."
+  ]
+}

evaluation_by_seed.csv

@@ -0,0 +1,11 @@
+seed,rollout_steps,commanded_forward_velocity_mps,mean_reward_per_step,mean_forward_velocity_mps,mean_absolute_velocity_error_mps,termination_count,success_no_termination
+100,400,0.5,0.14128163651563228,0.23589461453258992,0.2643511279672384,0,True
+101,400,0.5,0.14869166431948544,0.27104998455848545,0.23166746033821256,0,True
+102,400,0.5,0.14602043539285658,0.2245279321074486,0.2757411253452301,0,True
+103,400,0.5,0.14819012811407448,0.29148244574753335,0.2116375322631211,0,True
+104,400,0.5,0.14435075111687184,0.22676853544602638,0.273739144433639,0,True
+105,400,0.5,0.14471292811096645,0.22117613712325693,0.2788238628767431,0,True
+106,400,0.5,0.14796128492802382,0.2755377550085541,0.22814512956538238,0,True
+107,400,0.5,0.15238978615030646,0.24962244590977206,0.2504566774680279,0,True
+108,400,0.5,0.14997595064342023,0.28972968977410346,0.21514052021782845,0,True
+109,400,0.5,0.1540444320999086,0.29541337831818965,0.2067692495981464,0,True