tt-model push meteor-p150 (container)

.gitattributes

@@ -33,3 +33,32 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+code/tt_meteor/samples/synthetic_8cam/CAM_BACK_NARROW.png filter=lfs diff=lfs merge=lfs -text
+code/tt_meteor/samples/synthetic_8cam/CAM_BACK_RIGHT.png filter=lfs diff=lfs merge=lfs -text
+code/tt_meteor/samples/synthetic_8cam/CAM_BACK_WIDE.png filter=lfs diff=lfs merge=lfs -text
+code/tt_meteor/samples/synthetic_8cam/CAM_FRONT_LEFT.png filter=lfs diff=lfs merge=lfs -text
+code/tt_meteor/samples/synthetic_8cam/CAM_FRONT_NARROW.png filter=lfs diff=lfs merge=lfs -text
+code/tt_meteor/samples/synthetic_8cam/CAM_FRONT_WIDE.png filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/067872b26b7173f0708cd110500b917d462d08711d6892501a9ea54a38b4c6f6 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/17d0c1caf6bbb45eee78a04317ec8b3dcefb4c37ea065b82c29c20a183f8c765 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/2e19a7ded24ad73423027b973b6c375f16dea19e2aa43cad23833870de4a0680 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/32fa560530289e8b9adb68b3d37bd6fdc2c63479a79ce728a6779e63a861ebda filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/3570876206ec7924f1f40ec6d1a3d74c44e21f77220cee79851fd9ad9b5305a6 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/385eb61defef064a82c5382744c1f00c925a71890ebc03dcbdb65ede3dfbd627 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/3fe767711c183db63e0c1e745ecc00844fa349ac9e24f137b00342b90f7b2ac0 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/5073f3c9466f53e104fc4b3166ce982d7e53f764697592f78c609016187947a4 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/53d35e743cff2bc1f9e506e7ea215b8b1a0c33e7868f8381f1759747d01b33fa filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/725d484635e8379455b63418fdd70e920d1f74853e81240ce98bb67daee60dee filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/8f0daca2adb0ba572e616d0aec982fa0f8419d1ddcbac6e3695c42c168451800 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/98c4455a98982b35380ec62f90d5cf0edff02eeb4b3bb18595d871fdeb0c0c67 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/a225e43749bf82b0c24c6dc81f6cf207e35a01ed8445ec2ba5027d258919ea54 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/b22f52e00ded7b565d8541479dbba1f62b001c32be7efbd3873da891633b36a1 filter=lfs diff=lfs merge=lfs -text
+media/meteor_ns0103_f0009_NC_card_tt.jpg filter=lfs diff=lfs merge=lfs -text
+media/meteor_ns0103_f0009_NC_heads_tt.jpg filter=lfs diff=lfs merge=lfs -text
+media/meteor_ps019_f0020_bev_tt_vs_cpu.png filter=lfs diff=lfs merge=lfs -text
+media/meteor_ps019_f0020_card_tt.jpg filter=lfs diff=lfs merge=lfs -text
+media/meteor_ps019_f0020_heads_tt.jpg filter=lfs diff=lfs merge=lfs -text
+media/meteor_ps019_seq_tt.gif filter=lfs diff=lfs merge=lfs -text
+media/meteor_ps090_f0020_card_tt.jpg filter=lfs diff=lfs merge=lfs -text
+media/meteor_ps090_f0020_heads_tt.jpg filter=lfs diff=lfs merge=lfs -text
+media/meteor_synthetic_8cam_tt_vs_cpu.png filter=lfs diff=lfs merge=lfs -text

OPT_BASELINE.md

@@ -0,0 +1,451 @@
+# meteor-p150 baseline on the p150 (before optimization)
+
+Date 2026-10-10. Code is the baseline commit `e86c13a` (first correct port, no optimization; ETH default with the
+hang fix of PORT_LOG.md section 12). The model code of `code/tt_meteor/` is unchanged since then
+(`git diff e86c13a -- code/tt_meteor` is empty); this file's commit adds only the measurement scripts
+(`code/scripts/bench.py` rewritten with p50 / p99, several inputs, AICLK and the agreement check; `profile_ops.py`;
+`profile_summary.py`). tt-metal `44d66500520` + `patches/tt-metal-eth-dispatch.patch` +
+`patches/tt-metal-reshape-rm-sys1419.patch`; ttaw 0.23.2. Weights `AutowareFoundation/meteor` @ `01a5f6d71df`
+(tag `v1.0`), `meteor_v157c3Z.onnx`, run as released (`input_norm=onnx`, D12).
+
+Inputs, batch 1 (one frame = 8 camera slots of 768×432 + calibration + ego speed):
+- **sample**: `code/tt_meteor/samples/pandaset_019_f40.json`, frame 40 of PandaSet sequence 019 in METEOR's
+  8-slot layout (7 camera JPEGs + the absent BACK_NARROW; preset `pandaset_019`; ego speed 10.88 m/s; CC BY 4.0 +
+  PandaSet Dataset Terms). It is the sample the port ships today. The release may move it to the git-ignored
+  `staging_samples_pandaset/` until the user approves shipping PandaSet data; `bench.py` finds it there too. Its
+  stored fp32 CPU reference gives the agreement check of every run;
+- **synthetic**: eight 768×432 uint8 noise images made by `bench.py` (data generated by us), preset `pandaset_019`,
+  10 m/s. The kind of input a release can always ship;
+- **PandaSet 090 f40** and **nuScenes 0103 kf09**: public-dataset frames of two other rigs, as API requests built
+  from `research/meteor/public_data/inputs` (local only, never shipped; nuScenes is CC BY-NC-SA). Their camera
+  arrays, K, T and v0 equal the goldens' graph inputs exactly (checked on the host before the bench).
+
+Configuration of every number: ETH dispatch, 1 CQ, 12×10 grid (`device.compute_with_storage_grid_size()` printed in
+each log), the shipped precision policy (PLAN §9.3: fp32 weights, HiFi4 + fp32 accumulation; image branch and det
+path in three bf16 terms, `METEOR_IMAGE_PRECISION=terms3`; BEV trunk bf16 activations), unless a row says otherwise.
+
+## Environment
+
+| item | value |
+|---|---|
+| tt-metal | `44d66500520` (main) + `patches/tt-metal-eth-dispatch.patch` (sha256 `08d0ddf6…`) + `patches/tt-metal-reshape-rm-sys1419.patch` (sha256 `74878683…`, the SYS-1419 row-major reshape fix that ended the ETH hangs, PORT_LOG 12.7-12.8). Both are applied to the workspace tree (`git diff --stat`: 5 files) and shipped in `patches/`; `eth_patch: true` in every log |
+| ttaw | 0.23.2, vendored from `common` `60b6dd7` (`code/tt_meteor/ttaw/VENDORED.json`; `vendor.py --check`: 0 differences, so no re-vendor and no gate re-run was needed) |
+| device | Blackhole p150b, KMD 2.10.0, firmware bundle 19.13.1.0. ETH dispatch, 1 CQ, 12×10 = 120 cores, `l1_small_size` 32 KiB, `trace_region_size` 256 MiB. ONE trace (`frame`, the whole network), **4,678 programs per frame**, 1,131 programs in the program cache |
+| AICLK | 1350 MHz median (min 1343) over 8,412 samples (one every 50 ms) during every bench and matrix loop (`ttaw.profiling.AiclkSampler`, sysfs); board power median 69-78 W (max 203 W), ASIC 64.5-71.3 °C. The profiler CSV header reads CHIP_FREQ 1350 MHz |
+| numerics | HiFi4 on all 2,402 compute programs of a frame (profile) |
+| host | AMD EPYC-Rome VM, 8 vCPUs, 90 GB RAM, Linux 6.8, shared with up to 8 other agents' jobs: load average 4.1-12.5 during the stage bench, 4.5-12.1 during the matrix; `OMP_NUM_THREADS=4`. Host rows move with that load (the sample's first plain loop ran at load 12.5 and has the worst e2e p99); device rows do not (trace p50 556.7-556.9 ms on every input and run) |
+
+## How to run
+
+```bash
+ROOT=/home/ubuntu/experiments/tt-models; cd $ROOT/bundles/meteor-p150; source $ROOT/bin/tt-env.sh
+export PYTHONPATH=$PWD/code:$PYTHONPATH HF_HUB_OFFLINE=1 TT_MODEL_WEIGHTS_REVISION=01a5f6d71df5ecbbb5853ec600825481d57b9c6b
+# (every METEOR device job needs METEOR_DEVICE_OK=1 from the orchestrator; never arm tt-triage)
+# accuracy gates (per-module PCC + end-to-end agreement vs the fp32 CPU reference), read-only, one job
+$ROOT/bin/devrun -t 1800 -- env TTAW_GATES_READONLY=1 python -m pytest -q -s \
+    code/tt_meteor/tests/test_pcc_device.py code/tt_meteor/tests/test_e2e_device.py
+# stage breakdown (API split, host_in / H2D / trace / b2b / D2H / unpack, e2e p50 / p99, AICLK, agreement)
+$ROOT/bin/devrun -t 2400 -- python code/scripts/bench.py --iters 60 --inputs sample,synthetic,<name>=<manifest> --json <out>.json
+# dispatch / CQ matrix: one process per configuration
+$ROOT/bin/devrun -t 2700 -- python code/scripts/bench.py --inputs sample --iters 50 --dispatch worker --num-cqs 2 --json <out>.json
+# rig switch (each call writes the other rig's lift tables)
+$ROOT/bin/devrun -t 1500 -- python code/scripts/bench.py --inputs a=<rig A manifest>,b=<rig B manifest> --iters 5 --rig-switch 40 --json <out>.json
+# device profile: one eager frame with module signposts + two signposted traced replays
+$ROOT/bin/devrun -t 3600 -- python -m tracy -r -p -v --no-web-server --op-support-count 6000 \
+    -o $ROOT/generated/profiler/meteor_baseline code/scripts/profile_ops.py --rounds 2
+tt-perf-report <ops_perf_results_*.csv> --start-signpost frame --end-signpost frame_end --tracing-mode --arch blackhole
+python code/scripts/profile_summary.py $ROOT/generated/profiler/meteor_baseline --json s.json --md s.md
+# ttnn-visualizer graph report of one eager frame (slow, never used for timing)
+$ROOT/bin/devrun -t 2700 -- python code/scripts/profile_ops.py --graph-report $ROOT/generated/ttnn_reports/meteor_baseline
+ttnn-visualizer --profiler-path $ROOT/generated/ttnn_reports/meteor_baseline/frame \
+    --performance-path $ROOT/generated/profiler/meteor_baseline/reports/2026_10_10_15_12_27
+```
+
+- **One frame is 4,678 programs**, above the device profiler's default buffer of about 1,000 programs, so the
+  profile runs with `--op-support-count 6000` and flushes the buffer (`ttnn.ReadDeviceProfiler`) before and after
+  each section.
+  - The log warns of dropped markers at 15:05:53-15:06:02. That is the flush right after the compile frame (the
+    first eager frame, 806.6 s of JIT with the profiler build), before any measured window.
+  - The measured windows are complete: both traced rounds have 4,678 programs and kernel sums of 555.28 / 555.61 ms,
+    and the eager frame has the same op sequence as the trace, op for op (4,678 programs, kernel sum 554.86 ms).
+- The job drivers and analysis scripts are in `logs/meteor/baseline/scripts/` of the workspace (`env.sh` holds the
+  common environment, `run_rest.sh` / `run_e.sh` / `run_f.sh` the job order).
+
+## Accuracy (baseline)
+
+The model code, ttaw and weights are those of `e86c13a`, whose gates ran three times under ETH on 2026-10-10
+(PORT_LOG 12.8, `logs/meteor/hangfix/suite_final{1,2,3}.log`: 15 passed each, read-only gates, identical values in
+all three runs). `vendor.py --check` reports 0 differences and `code/tt_meteor/` is unchanged, so the frozen gates
+were not re-run for this file.
+
+| check | gate | baseline (device suite x3 under ETH, read-only gates: 15 passed each, identical values) |
+|---|---|---|
+| teacher-forced stage PCC (26 gates of `tests/test_pcc_device.py`, PandaSet 019 f40) | >= 0.99 | min 0.9999890 (`tf.risk`); `depth_mean` 0.99948 (reported) |
+| teacher-forced lane / seg2d / depth argmax agreement | >= 0.99 | 0.9984 / 0.9965 / 0.9936 |
+| chained frame (the served graph from the cameras), min output PCC: PandaSet 019 / 090, nuScenes 0103, valday #40 | >= 0.99 | 0.99989 / 0.99990 / 0.99991 / 0.99982 |
+| chained lane / seg2d / depth agreement (worst frame) | >= 0.99 | 0.9963 / 0.9965 / 0.9925 |
+| det3d strict recall / precision (served decode): sample, PandaSet 090, nuScenes, valday #40 | >= 0.95 / >= 0.95 | 1.0/1.0, 1.0/1.0, 1.0/1.0, **0.971 / 0.971** (34 / 35) |
+| selected ego path deviation (max waypoint), worst frame | <= 0.3 m | 0.170 m (PandaSet 090) |
+| sample `/predict` vs stored CPU-reference body (smoke compare): recall / precision / max \|Δscore\| | >= 0.95 / >= 0.95 / - | 1.0 / 1.0 / 0.0081 |
+| imagenet + linear variant (`test_variants_device.py`), float outputs min PCC | >= 0.99 | 0.99988 |
+| CPU reference vs ONNX Runtime on the shipped ONNX (the reference itself is right) | >= 0.9999 | min 0.99999990 (155 taps) |
+| ETH stress (`code/scripts/stress_frames.py --mode rigs`: rig change every frame, bit-compare per rig) | no hang, 0 mismatches | 2000 frames clean |
+
+- **Every bench run also checks its own served output.** The sample runs as a fresh stream and is compared with
+  its stored CPU reference with the container-smoke gates (`ttaw.server.smoke.compare_with_reference`):
+  8 / 8 boxes matched, recall = precision = 1.0, max |Δscore| 0.0081, lane agreement 0.99770, selected path
+  deviation 0.108 m. Identical in ETH-1CQ, ETH-2CQ and WORKER-2CQ (WORKER lane agreement 0.99770 too).
+- Host suite on the fake ttnn (the host test files, explicitly listed): **100 passed, 1 skipped**
+  (`test_reference_cpu.py`: no onnxruntime in tt-env), `logs/meteor/baseline/host_suite.log`.
+
+## Performance (warm, batch 1, p50 (p99) of 60 iterations per loop)
+
+Each row of the API split is timed inside one `model(...)` call (wrappers around the parts of
+`TtMETEOR.run_frame`); the `TraceRunner` rows come from a second loop with every stage synchronised on its own.
+Images are decoded before timing (the camera driver hands METEOR decoded images).
+
+| stage | sample (PandaSet 019) | synthetic | PandaSet 090 | nuScenes 0103 |
+|---|---:|---:|---:|---:|
+| host preprocess (`_prepare`: INTER_AREA to 768×432, K / T, METEOR's runtime pre-processing) | 11.2 (35.8) | 11.1 (13.2) | 9.3 (10.3) | 9.2 (10.1) |
+| device stage of the API (`api.device`), of which: | 774.9 (883.9) | 735.1 (756.6) | 755.0 (787.2) | 765.9 (780.8) |
+| … NCHW → NHWC camera rows on the host (`image_rows`) | 26.3 (31.9) | 23.1 (30.9) | 21.6 (32.0) | 22.1 (35.6) |
+| … `TraceRunner` call: host tensor + H2D + replay + segmented D2H | 689.5 (711.4) | 646.6 (668.8) | 677.7 (694.4) | 675.9 (700.1) |
+| … join of the 5 readback segments (`unpack`) | 15.9 (53.6) | 15.6 (16.1) | 15.6 (16.5) | 15.6 (16.4) |
+| … device layouts → the 19 ONNX outputs (`outputs_from_device`, host) | 49.4 (95.9) | 47.8 (51.8) | 46.0 (52.4) | 45.7 (50.9) |
+| host postprocess (METEOR's C++ decode rules + temporal state) | 129.9 (164.6) | 124.0 (130.7) | 130.0 (141.3) | 131.2 (143.1) |
+| **e2e `model()`** | **918.2** (1071.8) | **875.7** (901.8) | **902.8** (927.8) | **914.4** (931.1) |
+| `TraceRunner` stages: host tensor (`host_in`) | 1.17 (1.39) | 1.14 (1.22) | 1.17 (3.11) | 1.12 (1.17) |
+| H2D (8 × 432 × 768 × 3 uint8 + v0, synchronised) | 66.5 (74.2) | 66.4 (84.2) | 66.4 (68.2) | 66.4 (67.6) |
+| **device trace, one blocking replay** | **556.85** (559.34) | 556.87 (562.78) | 556.85 (559.24) | 556.68 (556.97) |
+| **back-to-back replays, per frame** (20) | **556.75** | 556.79 | 556.82 | 556.57 |
+| D2H (5 segments, 75.0 MB fp32) | 17.2 (63.5) | 17.8 (66.7) | 47.5 (70.5) | 46.7 (67.8) |
+| `TraceRunner` e2e (call + unpack) | 658.3 (680.5) | 658.2 (670.2) | 688.3 (720.2) | 688.1 (708.4) |
+
+- **Throughput.** Back-to-back replays give **1.80 frames/s** of device throughput (556.8 ms per frame); a
+  synchronous `model()` call gives 1.09-1.14 frames/s (875.7-918.2 ms).
+- **The device time does not depend on the input** (one fixed-shape graph; the lift tables only change the
+  sampling positions): trace p50 556.7-556.9 ms on all four inputs, AICLK 1350 MHz throughout.
+- **Where the time goes on the sample (918 ms):** device trace 557 ms (61 %); host decode 130 ms (14 %); host
+  conversion of the 75 MB readback (`unpack` + `outputs_from_device`) 65 ms (7 %); H2D 66 ms (7 %); host
+  NCHW → NHWC 26 ms; D2H 17-48 ms; host preprocess 11 ms.
+- **D2H is bimodal** (17 ms or 46-48 ms for the same 75 MB, both within one process; p99 63-71 ms on every input).
+  The device and the bytes are the same, so it is a host-side effect of the shared host (cause not isolated;
+  UNVERIFIED: page faults or memory pressure while other agents run).
+- **Load** with a warm JIT cache: 49.7 s (weights 0.3 s, graph build 2.6 s, warm-up 42.2 s: one eager frame 41.0 s
+  plus the capture 1.2 s). A process on a new dispatch configuration compiles its own kernels: ETH-2CQ 141.9 s,
+  WORKER-2CQ (11×10) 767.7 s.
+- **Rig change** (`--rig-switch`; calls alternate between two calibrations, so every call writes the other rig's
+  `lift_grid` + 20 table chunks): PandaSet 090 ↔ nuScenes 0103, 40 calls (`rigswitch_eth-1cq.json`): the table
+  write takes **182.3 ms p50** (p99 203.9; 211 MB in 21 synchronised transfers, about 1.16 GB/s) and the
+  call 1128.5 ms p50 (p99 1203.1) instead of about 905 ms. The lift geometry itself is cached per rig on the host. A
+  fixed rig (the vehicle case) pays this once. (The stage bench's own `--rig-switch` loop alternated the sample and
+  the synthetic input, which share the `pandaset_019` preset, so it measured no rig change; it is not quoted.)
+
+## Dispatch / CQ matrix (D14)
+
+Setup: the sample (identical frames in every row), one process per configuration, each its own devrun window
+(14:29-14:51 UTC), 50 iterations per loop plus 20 back-to-back frames. The first row is the stage bench.
+
+| configuration | grid | programs (cache) | b2b frame ms | trace p50 (p99) ms | H2D p50 / min ms | D2H p50 ms | runner e2e p50 ms | e2e p50 (p99) / min ms | host load |
+|---|---|---:|---:|---:|---:|---:|---:|---:|---|
+| **ETH-1CQ** (shipped; stage bench) | 12×10 | 1,131 | **556.75** | **556.85** (559.34) | 66.46 / 66.42 | 17.19 | 658.3 | 918.2 (1071.8) / 883.1 | 12.5 → 5.4 |
+| ETH-2CQ | 12×10 | 1,131 | 561.37 | 561.52 (562.05) | 67.36 / 67.33 | 18.02 | 662.4 | 916.8 (941.8) / 881.4 | 5.1 → 4.5 |
+| WORKER-2CQ | 11×10 | 1,161 | 578.85 | 579.04 (579.46) | 62.81 / 62.76 | 17.30 | 675.8 | 965.9 (1086.0) / 907.6 | 12.1 → 10.6 |
+| WORKER-1CQ (PORT_LOG 12.6, measured at an earlier commit, for reference only) | 11×10 | | | 573.0 | | | | | |
+
+- **ETH stays the default.** WORKER-2CQ replays the frame 22.1 ms slower (578.9 vs 556.8 ms, +4.0 %): on 11×10
+  ttnn partitions differently and builds 1,161 programs instead of 1,131. WORKER's H2D is 3.6 ms faster (62.8 vs
+  66.4 ms), which does not make up the difference. WORKER also needs its own 13-minute JIT compile at first load.
+- **1 CQ.** ETH-2CQ replays 4.6 ms slower than ETH-1CQ (561.4 vs 556.8 ms; ETH-2CQ runs without dispatch_s, PLAN
+  §0.2), and a synchronous request gains nothing from the second queue (e2e 916.8 vs 918.2 ms at a lower host load).
+  **Decision: 1 CQ stays pinned** (`DEVICE_DEFAULTS num_command_queues: 1`, `METEOR_NUM_CQS`, D14): the API and the
+  server serve one synchronous request at a time. A pipelined serve mode (opportunity 8) is the only way 2 CQs pay.
+- The ETH-2CQ and WORKER-2CQ processes ran clean (rc 0, outputs identical to ETH-1CQ on the agreement check). Before
+  the SYS-1419 reshape fix, ETH-2CQ hung within 43 frames (PORT_LOG 12.5); with the fix it is a working option again.
+
+## Host vs device, and every host<->device transfer per forward
+
+**Host** (METEOR's own runtimes define pre / post: there is no Autoware package):
+- before the device: request parsing, the INTER_AREA resize to 768×432 (identity on the 768×432 sample), K scaled
+  per axis, T inverted, the absent narrow cameras as zero images with their donor's pose (9-11 ms); the lift geometry
+  and tables per calibration (cached per rig); the NCHW → NHWC transpose of the 8 cameras (22-26 ms);
+- after the readback: the join of the segments (15.6 ms), `outputs_from_device` (device layouts → the 19 ONNX
+  outputs: views, transposes, uint32 → uint8, 46-49 ms), and METEOR's C++ decode rules
+  (`host.result.build_output`, 124-131 ms).
+
+There is no mid-graph host round trip and no host fallback op: the whole network is ONE trace, replayed per call.
+
+**H2D per frame** (`copy_host_to_device_tensor` into the persistent trace inputs, CQ0):
+
+| tensor | shape, dtype, layout | bytes | pages × page size | when |
+|---|---|---:|---|---|
+| `imgs` | `[1, 1, 2654208, 3]` uint8 ROW_MAJOR (8 × 432 × 768 NHWC rows) | 7,962,624 | **2,654,208 × 3 B** | every frame: **66.4 ms (0.12 GB/s)** |
+| `v0` (RT-dev param) | `[1, 1, 1, 1]` fp32 TILE | 4 KB (one tile) | | when the speed changes (unchanged values are not re-uploaded) |
+| `lift_grid` | `[8, 400, 250, 2]` fp32 ROW_MAJOR | 6,400,000 | | when the rig changes |
+| `lift_table_00..19` | 20 × `[1, 1, 40000, 64]` fp32 TILE | 20 × 10,240,000 | | when the rig changes; every chunk is followed by a device sync |
+
+**D2H per frame:** one packed fp32 tensor `[18309, 1024]` (75.0 MB: `pack_outputs` of the 15 device outputs, the
+argmaxes as uint32 → fp32) read as 5 row segments of at most 16.8 MB, 17.2 ms (4.4 GB/s) when the host is calm.
+
+**The image upload is the transfer problem.** 8 MB goes up as 2.65 M pages of 3 B (one pixel per page): 0.12 GB/s.
+The same path moves the 75 MB readback at 4.4 GB/s with 4 KB pages, so camera rows as pages (2,304 B) would take
+about 2-3 ms (UNVERIFIED estimate from the measured D2H rate).
+
+## Grid usage
+
+- **Nothing hard-codes 11×10 or 12×10** (VERIFY_PORT): on 11×10 the WORKER run builds its own program set (1,161 vs
+  1,131) and gives the same served output.
+- **1,916 of the 4,678 programs run on all 120 cores, and they hold 407.6 of the 555.3 ms** (73 %):
+  the DRAM-sliced convs and their slices, typecasts, the binary ops, the TILE reshapes, the lift grid_samples.
+- 1,234 programs on 110-119 cores (65.8 ms): convs and their halo / move on height-sharded slices.
+- 1,198 programs on 64-109 cores (51.0 ms), 220 on 16-63 cores (18.3 ms), 110 on 1-15 cores (12.6 ms). The costly
+  few-core programs:
+  - `head.ego_attn` pooling matmul `[1, 800, 96, 500] × [500, 16]` fp32 on **3 cores: 5.10 ms**;
+  - `head.risk_sample` `[1, 400000] × [400000, 96]` bf16 (a K = 400,000 reduction) on **3 cores: 4.81 ms**;
+  - the W-axis interpolation matmuls of `Resize2d` at 250 → 500 on 16-24 cores: `lift.resize` 2.96 ms,
+    `head.seg_refine` 2.89 ms, `bev.lane` 2.89 ms; `bev.traj` `[400·128, 250] × [250, 16]` on 4 cores 1.48 ms.
+- **Every activation is DRAM-interleaved between ops** (C17 / C27 first-port layout). The convs run ttnn's
+  automatic DRAM slicing with height-sharded L1 slices (386 PaddedSlice + 386 SliceWrite programs).
+- **L1 is empty between ops.** The ttnn-visualizer graph report of one eager frame (2,820 recorded ttnn operations,
+  37,001 buffer records, all DRAM) shows no L1 buffer at any op boundary. DRAM peaks at 95.9 MiB per bank at the
+  lift's fp32 product `[8, 1, 100000, 96] × w` (the 307 MB fp32 camera products next to the 205 MB of fp32 lift
+  tables, the 8-camera fp32 feature maps of the three-term image branch and the 48.25 M weights).
+
+## What is already fused / traced
+
+**Done:**
+- **The whole network is one trace** (`frame`, 4,678 programs), replayed per call with no host work inside. Capture
+  takes 1.2 s; replay == eager is a gate of the port (M7).
+- **Eagerly** (profiler, program cache hot), the same 4,678 programs take the same kernel time (554.86 vs 555.28 ms
+  traced), but the eager frame spans 806.1 ms: **251.3 ms of op-to-op gaps** (host dispatch of 4,678 ops, the most
+  before `img.depth` 67.4 ms and `img.resnet` 55.3 ms). Traced, the gaps add up to 2.74 ms (0.5 %).
+- **Exact rewrites** (PORT_LOG section 2, all checked against the fp32 reference):
+  - BN folded into every conv (float64); the /255 input scale folded into the stem weights; tgate + tfuse merged
+    into one 1×1 conv (96 → 100); merged hm | reg heads (det and det2d); ctx + paint (21 → 96) as one K-split conv;
+    the MHAs in folded form (attention pools); traj's `reg_pre` input folded in;
+  - every Resize as separable fp32 interpolation matmuls (`F.interpolate` half-pixel incl. the edge clamp, also the
+    non-integer ones);
+  - the lift as two `grid_sample` gathers + a per-calibration bin-interpolation table + fp32 camera sums;
+  - pools as constant matmuls, `mean(fused)` computed once;
+  - argmaxes channel-major on the device; one packed readback in <= 16 MB segments.
+- **Fused epilogues:** ReLU in every conv; fp32 conv outputs where a value feeds a threshold, a softmax, a clamp or
+  the planner.
+
+**Not fused / not resident:**
+- every activation round-trips DRAM between ops;
+- **the three-term convs (E20 / E22)**: every conv of the image branch and the det path runs three bf16 convs
+  (`x_hi·W_hi`, `x_hi·W_lo`, `x_lo·W_hi`) plus a typecast pair to split the fp32 activation and two fp32 adds;
+- **TILE reshape copies** between `[1, 1, N·H·W, C]` and `[1, H, W, C]` around every `Resize2d` and every conv
+  whose map width (500, 250, 48, ...) is not a multiple of 32: 435 copies, 128.5 ms;
+- the stem's exact fp32 max pool is a chain of DRAM untilize / pad / strided slices / maximums / tilize (25.3 ms);
+- the output tail untilizes, pads and reshapes every output into the packed rows (12.9 ms);
+- no custom kernel, no megakernel.
+
+## Device profile (traced replay; kernel sum vs span, op-to-op gaps)
+
+Traced replays, ETH-1CQ, the sample (`profile_ops.py --rounds 2`; stage labels from the eager frame's module
+signposts, carried over op for op). `profile_summary.json` / `.md` hold every table below.
+
+| | frame (round 1) | frame (round 2) |
+|---|---:|---:|
+| programs | **4,678** | 4,678 |
+| device kernel sum | **555.28 ms** | 555.61 ms |
+| span (first FW start → last FW end at 1350 MHz) | 558.02 ms (b2b 556.75) | 558.35 ms |
+| op-to-op gaps | **2.74 ms (0.5 %)** | 2.74 ms |
+| math fidelity | HiFi4 on all 2,402 compute programs | |
+
+**The trace is kernel-time bound: dispatch is not a sink.** At 0.6 µs per op, removing every op-to-op gap would
+save 2.7 ms; a megakernel pays only through the kernel time (DRAM round trips, layout copies) it removes.
+
+**Compute vs everything else.** Data movement and layout ops, not math, dominate:
+- TILE reshapes (23.1 %), typecasts (9.2 %), slicing (PaddedSlice / SliceWrite / Slice 13.1 %), tilize / untilize /
+  pad (6.5 %), halo / move / sharding (3.6 %), transpose / concat / permute (1.3 %): **315 ms of the 555 ms (57 %)
+  move or reformat data**;
+- the convs (65.2 ms, 585 programs) run at 63 % of the HiFi4 FLOPs inside their kernels (height-sharded, weighted
+  mean, tt-perf-report) and 31 % (block-sharded); the matmuls at 7.9 % (weighted mean);
+- tt-perf-report models the frame at 2.1 % of the DRAM roofline (11 GB/s).
+
+Per block and module (traced kernel time):
+
+| block / module | ops | kernel ms | share | where the time goes (traced kernel ms) |
+|---|---:|---:|---:|---|
+| **image branch** (8 cameras, terms3) | **2,621** | **268.27** | **48.3 %** | |
+| `img.normalize` (uint8 → bf16) | 1 | 8.57 | 1.5 % | one Typecast of `[2654208, 3]` ROW_MAJOR (3-byte pages) |
+| `img.stem` (7×7 s2 conv + exact fp32 max pool) | 44 | 42.51 | 7.7 % | max pool chain 25.3 (strided Slice 17.56, maximums 4.29, pad 2.15, untilize 0.86, tilize 0.40), PaddedSlice of the 3-channel input 13.85 (6 DRAM slices, padded to 8 channels), the 6 conv slices 0.99 + their SliceWrites 1.22 |
+| `img.resnet` (ResNet-34, terms3) | 949 | 57.54 | 10.4 % | BinaryNg 15.26 (term sums, residuals), conv 13.52, ReshapeView 10.41, Typecast 7.53 |
+| `img.fpn` (laterals, resizes, adds) | 75 | 25.03 | 4.5 % | matmul 8.24 (resizes), ReshapeView 5.82, BinaryNg 4.72, untilize 2.28 |
+| `img.fpn_fuse` | 96 | 7.66 | 1.4 % | |
+| `img.seg2d` (decoder + clamp + argmax) | 373 | 40.89 | 7.4 % | BinaryNg 9.69, conv 6.95, Typecast 6.21, matmul 5.60 |
+| `img.depth` (head + argmax + softmax) | 511 | 43.56 | 7.8 % | BinaryNg 12.25, conv 9.13, Typecast 7.07, SliceWrite 6.10, a `[64, 165888]` fp32 TILE reshape 3.34 |
+| `img.depth_mean`, `img.ctx_paint`, `img.det2d`, `img.tl` (+ the lift-input conversions) | 572 | 42.51 | 7.7 % | `img.det2d` 28.12 (terms3) |
+| **lift** | **61** | **56.26** | **10.1 %** | GridSample 28.21 (two calls of 14.1 ms: `[8, 108, 192, 96 / 64]` bf16 RM maps, `[8, 400, 250, 2]` fp32 grid → `[8, 400, 250, C]`), weights / sums BinaryNg 6.92, tilize 4.25, typecast 2.84; `lift.resize` ×2 10.58 (matmul 4.88, reshape 1.91) |
+| **BEV trunk** (800×500 and 400×250 maps) | **939** | **147.48** | **26.6 %** | ReshapeView 69.88 (104 copies), conv 20.84, matmul 13.41, BinaryNg 11.48, SliceWrite 5.93 |
+| … `bev.lane` | 326 | 58.16 | 10.5 % | ReshapeView 27.11, conv 12.65, matmul 7.52 |
+| … `bev.det` (terms3 det stem + head) | 348 | 48.56 | 8.7 % | ReshapeView 24.00, BinaryNg 6.73, conv 5.20, typecast 3.76 |
+| … `bev.fuse`, `bev.traj`, `bev.occupancy`, `bev.stationary`, `bev.risk` | 265 | 40.76 | 7.3 % | `bev.fuse` 17.13 (reshape 6.66), `bev.traj` 10.36 |
+| **heads + planner** | **1,012** | **70.34** | **12.7 %** | ReshapeView 24.73, matmul 17.42, BinaryNg 5.32, conv 4.23 |
+| … `head.seg_refine` | 264 | 34.19 | 6.2 % | ReshapeView 14.71, matmul 6.05 |
+| … `head.box_refine` (terms3) | 476 | 15.69 | 2.8 % | ReshapeView 6.43, BinaryNg 3.06 |
+| … planner (`ego_stem`, `ego_mlp`, `ego_attn`, `sem`, `risk_sample`, `decoder_wp`, `e2e`) | 272 | 20.46 | 3.7 % | `ego_attn` 8.09 (3-core matmul 5.10), `risk_sample` 5.07 (3-core matmul 4.81), `ego_stem` 4.38 |
+| **output tail** (`out.pack`) | **45** | **12.93** | **2.3 %** | ReshapeView 6.00, pad 3.40, untilize 2.50 |
+| **total** | **4,678** | **555.28** | | |
+
+| sink (op type), frame | ms | % | where |
+|---|---:|---:|---|
+| ReshapeView (435) | 128.52 | 23.1 | TILE copies `[1, 1, N·H·W, C]` ↔ `[1, H, W, C]` at W = 500 / 250 / 48: `[400000, 96]` → `[800, 500, 96]` bf16 10 × 1.65 ms, `[100000, 128]` ↔ `[400, 250, 128]` (bf16 and fp32) 31 copies 18.8 ms, `[10368, 256]` ↔ `[216, 48, 256]` 90 copies 9.8 ms (resnet layer 3-4), ...; by module: `bev.lane` 27.1, `bev.det` 24.0, `head.seg_refine` 14.7, `img.resnet` 10.4, `bev.fuse` 6.7, `head.box_refine` 6.4, `out.pack` 6.0, `img.fpn` 5.8 |
+| BinaryNg (440) | 85.66 | 15.4 | the fp32 term sums of the three-term convs, residual adds, the stem max-pool maximums, lift weights and camera sums |
+| Conv2d (585) | 65.17 | 11.7 | each image / det conv three times (terms3); 63 % (height-sharded) / 31 % (block-sharded) of the HiFi4 FLOPs inside the kernels |
+| Halo + Move + interleaved ↔ sharded (1,580) | 19.86 | 3.6 | the conv glue of the DRAM-sliced convs |
+| Matmul (131) | 54.31 | 9.8 | the planner's two 3-core matmuls 9.91; the rest (44.4) mostly the `Resize2d` interpolation matmuls (fp32, many on 16-48 cores), pools and the paint fold |
+| Typecast (359) | 50.91 | 9.2 | fp32 → bf16 TILE (234 programs, 25.68 ms) and bf16 → fp32 (121, 16.58 ms): the term split of the three-term convs; uint8 → bf16 input 8.57 |
+| PaddedSlice + SliceWrite + Slice (820) | 72.56 | 13.1 | ttnn's automatic DRAM slicing of the convs 49.3 (the stem's 3-channel input 13.85); the max-pool strided slices 17.56; lift camera slices 2.08 |
+| GridSample (3) | 28.22 | 5.1 | the lift (2 × 14.1 ms) and the planner's risk sampling |
+| Tilize / Untilize / Pad (194) | 36.34 | 6.5 | the output tail 6.4, lift sampled-row tilize 4.25, the stem max pool 3.4, resize and planner glue |
+
+Top 10 device ops of the frame (traced):
+
+| # | op | where | µs | cores |
+|---:|---|---|---:|---:|
+| 1 | GridSample | `lift`: ctx `[8, 108, 192, 96]` bf16 RM, `[8, 400, 250, 2]` fp32 grid → `[8, 400, 250, 96]` | 14,119 | 120 |
+| 2 | GridSample | `lift`: depth probabilities `[8, 108, 192, 64]` → `[8, 400, 250, 64]` | 14,093 | 120 |
+| 3 | Typecast | `img.normalize`: `[2654208, 3]` uint8 → bf16 ROW_MAJOR | 8,575 | 120 |
+| 4 | Matmul | `head.ego_attn`: `[1, 800, 96, 500] × [500, 16]` fp32 (attention pooling) | 5,098 | **3** |
+| 5 | Matmul | `head.risk_sample`: `[1, 400000] × [400000, 96]` bf16 | 4,809 | **3** |
+| 6 | Slice | `img.stem` max pool: `[8, 218, 386, 64]` fp32 RM → `[8, 218, 192, 64]` (stride 2) | 3,936 | 120 |
+| 7 | Slice | `img.stem` max pool (second column phase) | 3,914 | 120 |
+| 8 | Slice | `img.stem` max pool (third column phase) | 3,910 | 120 |
+| 9 | ReshapeView | `img.depth`: `[1, 1, 64, 165888]` → `[1, 64, 5184, 32]` fp32 TILE | 3,335 | 120 |
+| 10 | Matmul | `lift.resize`: `[1, 400, 96, 250] × [250, 500]` fp32 (W axis ×2) | 2,961 | 24 |
+
+Then the two other W-axis resize matmuls at 250 → 500 (2,890 / 2,885 µs on 16 cores), the six PaddedSlices of the
+stem's 3-channel input (2,285-2,361 µs each), the max-pool pad (2,150 µs) and the output tail's `[400384, 9]` →
+`[3519, 1024]` reshape (2,086 µs).
+
+- Shapes in the CSV come from the first program of a cached program (binary_ng reuses one program across shapes),
+  so the per-module attribution uses the stage labels, not the CSV's shape columns. Ops a module issues inline after
+  its last call belong to the module signposted last (e.g. the depth softmax to `img.depth_mean`, the lift-input
+  conversions to `img.tl`).
+- `tt_perf_report_frame.csv` has the per-op FLOPs and DRAM figures; `profile_extra.json` the per-module op-code
+  tables, the reshape shapes and the core-count histogram.
+
+**The precision cost found in the port** (A/B only: the bench's agreement check on the sample against its stored
+CPU reference with the container-smoke gates; the frozen gates were not run):
+
+| policy | b2b frame ms | Δ vs shipped | sample check (smoke gates): matched, recall / precision, max \|Δscore\|, lane agreement, path deviation | frozen gates |
+|---|---:|---:|---|---|
+| **shipped: image branch and det path in three bf16 terms (fp32 activations), BEV trunk bf16** | **556.75** | – | 8 / 8, 1.0 / 1.0, 0.0081, 0.99770, 0.108 m | all pass (above) |
+| image branch with bf16 activations (`METEOR_IMAGE_PRECISION=bf16`; det path still terms3), measured here (`precision_image-bf16.json`, 20 iterations) | **390.83** | **−165.9 ms (−29.8 %)** | 8 / 8, 1.0 / 1.0, 0.0046, 0.99781, 0.077 m (center error up to 0.37 m) | **fails** the depth argmax gate: 0.9797 < 0.99 (PORT_LOG E20, port measurement on the same frame; not re-run here) |
+| det path in bf16 (`det_precision="bf16"`, no env knob; port A/B, PORT_LOG 11.4: trace 548.5 vs 507.4 ms before the ETH / reshape changes) | | −41 ms (port) | | **fails** strict det3d on valday #40: recall 0.943 < 0.95 (PORT_LOG E22) |
+
+- **The three-term precision of the image branch is the largest single cost of the port: 165.9 ms per frame
+  (29.8 %)**, measured as a same-day A/B on the same build. Together with the det path (+41 ms at the port) the
+  three-term policy accounts for about 207 ms (37 %). Both cheaper policies fail a frozen gate (depth argmax
+  agreement, strict det3d recall), so the cost is real accuracy, not over-engineering. Item 2 targets the mechanism
+  (three convs, a typecast split and two fp32 adds per conv), not the precision.
+- The bf16 run passes the sample's smoke comparison, which shows how loose that check is next to the frozen gates.
+
+## Ranked optimization opportunities
+
+How to read the estimates:
+- Gains are on the device frame (556.8 ms, 1.80 frames/s) unless marked e2e. Each one is the measured kernel time
+  of the programs the item removes, with a range for what replaces them.
+- Items overlap, so the gains do not add up.
+- Every item keeps the frozen gates above and the precision policy (PLAN §9.3), and lands as one commit with one
+  `METEOR_*` A/B knob (PLAN §5.1).
+
+1. **TILE reshape copies (trace −90..−120 ms; the largest item).**
+   - 435 `ReshapeView` programs take 128.5 ms (23.1 %). They are real copies because the map widths (500, 250,
+     48, 125, ...) are not multiples of 32: `[1, 1, N·H·W, C]` ↔ `[1, H, W, C]` around every `Resize2d` and every
+     conv whose neighbour wants the other form. By module: `bev.lane` 27.1, `bev.det` 24.0, `head.seg_refine` 14.7,
+     `img.resnet` 10.4, `bev.fuse` 6.7, `head.box_refine` 6.4, `out.pack` 6.0, `img.fpn` 5.8 ms.
+   - Fixes, one knob each: keep every FeatureMap in one form per stage (the convs take the flattened form; give
+     `Resize2d` a flattened-input path); ROW_MAJOR views where a reshape is page-aligned (safe again under ETH with
+     the SYS-1419 patch); or pad the BEV width to 512 / 256 so the TILE reshape is a view.
+   - The same C17 / C24 item as in CenterPoint and BEVFormer (shared ttaw fix, re-vendor rule PLAN §5.1 step 5).
+2. **Three-term convs as one fused op (trace −90..−150 ms; keeps the precision).**
+   - Measured cost of the policy: 165.9 ms for the image branch (above) + about 41 ms for the det path.
+   - Mechanism today, per conv: a typecast pair splits the fp32 activation into `x_hi` / `x_lo` (359 typecasts,
+     50.9 ms per frame in total, of which 42.3 ms are TILE fp32 ↔ bf16 conversions, mostly these splits), three
+     bf16 convs (585 convs, 65.2 ms), two fp32 adds (the image branch alone runs 282 BinaryNg programs, 61.9 ms).
+   - A fused terms conv (one program: hi / lo split in the reader / unpacker, the three products accumulated in the
+     fp32 dest, one output) removes the typecasts, the adds and two of three activation reads. Alternatively a true
+     fp32-activation conv path of TF32+ quality (I7 investigates why ttnn's own fp32 path is worse).
+   - Precision-preserving stepping stone: drop the `x_lo·W_hi` / `x_hi·W_lo` term per layer where the
+     `precision_emulation.py` gates allow (E20 found two-term activations insufficient as a whole; per-layer is
+     untested).
+3. **Input path and stem (trace −35..−45 ms; e2e −60 ms).**
+   - `img.normalize` 8.6 ms: a uint8 → bf16 typecast over 2.65 M ROW_MAJOR pages of 3 bytes.
+   - The stem conv's DRAM slicing reads the 3-channel input six times and pads it to 8 channels (PaddedSlice
+     13.9 ms).
+   - The exact fp32 max pool is a DRAM chain (untilize, pad, 6 strided slices, 4 maximums, tilize): 25.3 ms. A
+     generic_op 3×3 / s2 fp32 max pool on the conv's height-sharded output, or the pool folded into the stem
+     output, takes it to about 1-3 ms (UNVERIFIED).
+   - H2D: upload whole camera rows (2,304-byte pages) or a pre-padded 8-channel layout instead of 3-byte pages:
+     66.4 ms → about 2-5 ms at the measured D2H rate (e2e).
+4. **DRAM slicing and conv glue (trace −25..−40 ms).**
+   - 386 PaddedSlice + 386 SliceWrite programs (49.3 ms) and the halo / move / sharding glue (19.9 ms) around the
+     DRAM-sliced convs of the 8-camera image maps and the 800×500 BEV maps.
+   - Larger slices, explicit height-sharded inputs (L1 holds 171 MB for tensors; layer-1 maps of 8 × 108 × 192 × 64
+     fp32 are 42 MB), and the band streaming of item 9 (MK4).
+5. **Lift as one gather kernel (K5 / MK3; trace −30..−45 ms).**
+   - The lift takes 56.3 ms: two `grid_sample` calls of 14.1 ms that write `[8, 400, 250, C]` bf16 to DRAM
+     (256 MB), then tilize, fp32 typecast, the product with the 205 MB fp32 table, sums over bins and cameras,
+     divide, and the ×2 resize (10.6 ms).
+   - A kernel per BEV block reading the L1-resident camera maps (ctx + prob: 53 MB bf16, fits the 171 MB of L1),
+     with the per-calibration ELL weights (K slots) and fp32 accumulation, writes only `[400, 250, 96]`.
+6. **Few-core and fp32 matmuls (trace −12..−20 ms).**
+   - `head.ego_attn` `[800·96, 500] × [500, 16]` fp32 on 3 cores (5.10 ms) and `head.risk_sample` `[1, 400000] ×
+     [400000, 96]` on 3 cores (4.81 ms): put the long dimension into M or split K over cores with a final
+     reduction: well under 1 ms each.
+   - The W-axis `Resize2d` matmuls at 250 → 500 run on 16-24 cores (2.9 ms each, 3 calls); `bev.traj`'s
+     `[400·128, 250] × [250, 16]` on 4 cores (1.5 ms).
+7. **Host side (e2e −150..−220 ms; latency only).**
+   - Host decode 124-131 ms (`host_post_profile.json`, CPU-only cProfile on the golden outputs: 150 ms on the
+     loaded host): the temporal seg fusion 73 ms (log-softmax of `[9, 800, 500]` 25 ms, contiguous copies 26 ms)
+     and the 2D decode 69 ms (1,210 per-class `peak_mask` / `sigmoid32` calls on 8 cameras × 10 classes × 3
+     scales). Vectorise the 2D peaks over classes and cameras and work in place: about −80..−100 ms (UNVERIFIED).
+   - Readback conversion 61-65 ms (`unpack` 15.6 + `outputs_from_device` 46-49) and D2H 17-48 ms of 75 MB fp32:
+     bf16 / uint8 outputs packed in their final layouts (views only) halve the bytes and remove most of the
+     conversion: about −40..−60 ms.
+   - The host NCHW → NHWC transpose (22-26 ms): pre-processing can write NHWC directly (the INTER_AREA output
+     is HWC already): about −20 ms.
+8. **Throughput by pipelining (frames/s; latency unchanged).**
+   - The device allows 1.80 frames/s (556.8 ms b2b); a synchronous call gives 1.09-1.14 frames/s.
+   - A pipelined serve mode (2 CQs: frame k+1's host pre-processing and upload, and frame k−1's readback and
+     decode, overlapped with replay k) is bounded by max(device 561 ms on ETH-2CQ, host about 250-330 ms): about
+     1.7-1.8 frames/s (UNMEASURED). It needs double-buffered outputs and reads on CQ1 (`TraceRunner` reads on CQ0
+     today), and the per-stream host state must stay ordered.
+9. **Megakernel and fusion candidates (D19, PLAN §5.2 / §5.3; SPEC §9 MK0-MK7).**
+   - The §5.2 gate is met through DRAM round trips and layout copies (57 % of the kernel time), not dispatch:
+     op-to-op gaps are 0.5 % of the frame (2.74 ms for 4,678 programs).
+   - Candidates, by measured block time:
+     - **MK4 band-streamed BEV trunk** (`bev.*` 147.5 ms + `head.seg_refine` / `box_refine` 49.9 ms = 197 ms, of
+       which 91 ms are reshape copies): 800×500 bands in L1, no reshapes, no DRAM slicing. First, after item 1;
+     - **MK1 / MK2 image encoder + heads** (268.3 ms, 48 %): L1-resident per camera group; item 2's fused terms conv
+       is the precondition;
+     - **MK3 lift** (56.3 ms; item 5);
+     - **MK0 input** (normalize + stem slicing + max pool: 51 ms; item 3);
+     - **MK6 planner** (20.5 ms, of which the two 3-core matmuls are 9.9 ms; item 6 first);
+     - **MK7 output tail** (12.9 ms of untilize / pad / reshape in `out.pack`, plus the host conversion of item 7).
+   - 2-4-op fusions to try first: the three-term conv triple + split + adds (item 2); conv + residual add + ReLU;
+     the fp32 → bf16 typecast folded into the producing conv's output dtype; untilize + pad + reshape of each output
+     into one pack kernel; the 4-op argmax chains.
+   - A single whole-model megakernel is not realistic: 4,678 programs, 190 heterogeneous convs, 48.25 M parameters,
+     77 MB BEV maps (SPEC §9).
+
+## Raw logs, CSVs and reports
+
+Paths are under `/home/ubuntu/experiments/tt-models/`. `L` = `logs/meteor/baseline/`.
+
+| what | where |
+|---|---|
+| devrun jobs (`logs/devrun/history.log`, all rc 0, all with `METEOR_DEVICE_OK=1`, no tt-triage) | A 14:06:10-14:17:55 (`L/scripts/jobA_bench.sh`: stage bench, 4 inputs); B1 14:29:17-14:33:52 (`jobB_one.sh eth 2`); B2 14:36:03-14:51:10 (`jobB_one.sh worker 2`); C 14:51:59-15:13:28 (`jobC_profile.sh`: Tracy); D 15:26:43-15:27:46 (`jobD_visualizer.sh`); E 15:34:26-15:39:45 (`jobE_precision.sh`); F 15:41:19-15:43:36 (`jobF_rigswitch.sh`). Order: `L/scripts/run_rest.sh`, `run_e.sh`, `run_f.sh`; `L/run_all.rcs` |
+| stage bench (ETH-1CQ) | `L/bench_eth-1cq.{json}`, `L/jobA_bench.log`; inputs `L/inputs/{pandaset_090_f40,nuscenes_0103_kf09}.json` (`L/scripts/make_manifests.py`) |
+| dispatch matrix | `L/matrix_{eth-2cq,worker-2cq}.json`, `L/jobB_{eth-2cq,worker-2cq}.log` |
+| rig switch | `L/rigswitch_eth-1cq.json`, `L/jobF_rigswitch.log` |
+| precision cost | `L/precision_image-bf16.json`, `L/jobE_precision.log` |
+| device profile | `generated/profiler/meteor_baseline/reports/2026_10_10_15_12_27/ops_perf_results_2026_10_10_15_12_27.csv` (+ `profile_log_device.csv.zst` (zstd-compressed 2026-10-10 to free disk; `zstd -d` restores the 6.0 GB CSV), `tracy_profile_log_host.tracy`; the ttnn-visualizer performance report); `L/profile_summary.{json,md}` (`code/scripts/profile_summary.py`); `L/profile_extra.json` (`L/scripts/profile_extra.py`: core histogram, per-module op codes, reshape shapes, typecast directions); `L/tt_perf_report_frame.{txt,csv}`, `L/tt_perf_report_summary_frame.{csv,png}`; `L/jobC_profile.log`, `L/profile_meta.json` |
+| ttnn-visualizer graph report | `generated/ttnn_reports/meteor_baseline/frame/` (`db.sqlite`, `config.json`; `profile_ops.py --graph-report`, plain graph capture, 8.1 s, 52.7 MB), `L/jobD_visualizer.log`, `L/graph_report_meta.json` |
+| host post-processing (CPU only, loaded host) | `L/host_post_profile.{json,log}` (`L/scripts/host_post_profile.py`) |
+| host suite | `L/host_suite.log` (100 passed, 1 skipped) |
+| gates of this code (`e86c13a`) | `logs/meteor/hangfix/suite_final{1,2,3}.log`, `values_final*`; the 2000-frame ETH stress `logs/meteor/hangfix/meteor_eth_final2000.*` |

OPT_REPORT.md

@@ -0,0 +1,113 @@
+# meteor-p150 optimization report (p150, ETH dispatch, 12×10 grid)
+
+**Status: baseline port; optimization pending.** This is the first public release: the functional port, measured in
+[`OPT_BASELINE.md`](OPT_BASELINE.md) and published with the precision policy of the first release (fp32 weights and
+biases, HiFi4, fp32 and packer L1 accumulation; the image branch and the 3D detection path with fp32 activations,
+every conv as three bf16 terms; the BEV trunk with bf16 activations). No optimization round has run yet. The whole
+network runs as ONE metal trace of **4,678 programs**, which replays in **556.85 ms** (back to back 556.75 ms,
+1.80 frames/s device-bound). A synchronous `model()` call takes 876-918 ms (p50, by input): around the replay come
+the host decode (124-131 ms), the conversion of the 75 MB readback (61-65 ms), the image upload (66 ms) and the host
+camera transpose (22-26 ms). The trace is kernel-bound (2.74 ms of op-to-op gaps), and 57 % of the kernel time moves
+or reformats data rather than computing. The ranked plan below starts with the TILE reshape copies and the three-term
+convs. Every round will be added to this file, newest first, with its commits, its numbers and its accuracy gates.
+
+All numbers: `code/scripts/bench.py` (warm, batch 1, p50 of 60 iterations unless marked), ETH dispatch, 1 CQ,
+12×10 grid, AICLK 1350 MHz (minimum 1343), tt-metal `44d66500520` + `patches/tt-metal-eth-dispatch.patch` +
+`patches/tt-metal-reshape-rm-sys1419.patch`, measured 2026-10-10 on the baseline commit `e86c13a` (ttaw 0.23.2;
+OPT_BASELINE.md committed as `cc35027`). Device profile: `code/scripts/profile_ops.py` under the Tracy device profiler
+(two traced replays). Accuracy gates: `OPT_BASELINE.md` "How to run".
+
+## Summary
+
+| | baseline `e86c13a` | **this release** |
+|---|---|---|
+| device, one blocking replay of `frame` | 556.85 ms (p99 559.34) | same graph (no device code changed); re-measured **556.92 ms** (p99 557.5) on the PandaSet frame, 556.80 ms on the shipped sample |
+| back-to-back replays | 556.75 ms per frame (1.80 frames/s) | re-measured **556.89 / 556.65 ms** |
+| e2e `model()` p50 (p99): PandaSet 019 sample / synthetic noise / PandaSet 090 / nuScenes 0103 | 918.2 (1,071.8) / 875.7 (901.8) / 902.8 (927.8) / 914.4 (931.1) ms | PandaSet 019 935.9 (999.6) ms on a busier host (30 iterations); the new shipped synthetic sample 871.6 (942.1) ms; served `/predict` `timing_ms.total` 904-909 ms |
+| host preprocess / H2D / D2H / readback conversion / postprocess (PandaSet 019) | 11.2 / 66.5 / 17.2 (17-48, bimodal) / 65.3 / 129.9 ms | unchanged |
+| rig change (lift tables of a new calibration) | 182.3 ms table write; call 1,128.5 ms p50 | unchanged |
+| device programs per frame | 4,678 (1,131 program-cache entries) | same trace |
+| kernel sum / span (profile) | 555.28 / 558.02 ms; 2.74 ms of op-to-op gaps | |
+| image branch / lift / BEV trunk / heads + planner / output tail (kernel ms) | 268.27 / 56.26 / 147.48 / 70.34 / 12.93 | |
+| accuracy gates | every PLAN 2.13 gate green, frozen values (OPT_BASELINE.md "Accuracy") | the same frozen gates (values in VERIFICATION_2026-10-10.md) + the container smoke's check on the new shipped synthetic sample |
+
+## Steps (chronological; each row is one commit)
+
+| step | commit | what | b2b frame | e2e (PandaSet 019 p50) | accuracy | revert switch |
+|---|---|---|---|---|---|---|
+| baseline | `e86c13a` | the verified functional port (VERIFY_PORT round 1 fixed; ETH default with the SYS-1419 reshape fix, ttaw 0.23.2) | 556.75 ms | 918.2 ms | every gate green, frozen values; 2000-frame ETH rig-switch stress clean | |
+| release | the release commit (`git log`: "Release docs + demo media") | release changes, none on the device graph: the synthetic shipped sample (`samples/synthetic_8cam*`); the PandaSet sample moved unchanged to the git-ignored staging dir; `opencv-python-headless` declared for the image (host post-processing, PORT_LOG I3); the smoke test's sample expectation filled; `server/client.py --sample / --ego-speed`; `examples/quickstart.py` fixed (it called `model(path)`); `stress_frames.py --rigs shipped` (golden-free, for the container) | 556.89 ms | 935.9 ms (busier host) | frozen gates unchanged: 16 device tests passed, every stored value identical to the baseline's ETH runs; the new shipped-sample check 8 / 8 | |
+
+## Round 1
+
+Not started. The first items are fixed by the baseline profile (below and OPT_BASELINE.md).
+
+## Known hangs (all rounds)
+
+All hangs below happened before this baseline, during the port (PORT_LOG.md section 12;
+`logs/meteor/hangfix/ETH_DISPATCH_HANG_ISSUE.md` of the workspace). None happened at or after `e86c13a`.
+
+| when (UTC) | command | cause | status |
+|---|---|---|---|
+| 10-08 / 10-09 (4 jobs) | METEOR device suites and benches under ETH dispatch | a non-deterministic hang of the trace replay; tt-triage launched on the hung chip then hard-reset the server 4 times | triage is now refused by the workspace's device lock; root cause below |
+| 10-10 04:17-05:38 (3 hangs) | `stress_frames.py --mode rigs` under ETH 1 CQ (frames 263, 248) and ETH 2 CQs (frame 43); WORKER 1,600 frames clean | ETH dispatch implicated; contained (no triage, no reset; the host stayed up) | bisected |
+| 10-10 07:15-09:28 (4 hangs + repros) | segmented-trace bisect: lift trace -> `lift_rest` -> `Resize2d` (400×250 -> 800×500) -> its tail -> one op | **one row-major `ttnn.reshape` [800, 48000] -> [400000, 96] (dual-kernel, 192 / 384 B DRAM pages from both NOCs) hangs under ETH dispatch: Blackhole DRAM-arbiter issue SYS-1419**; reproduced standalone in plain ttnn (`code/scripts/repro_resize2d_eth.py`) | **fixed**: `patches/tt-metal-reshape-rm-sys1419.patch` (single-kernel path for small DRAM destination pages; 20,000 replays clean at 192 / 384 B) applied to the image's tt-metal; ttaw 0.23.2 keeps a TILE tail when the patch is absent |
+| 10-10 11:4x-13:51 | after the fix: device suite ×3, a 2000-frame ETH rig-switch stress at `18e0fbd` and another at `e86c13a`, and the baseline's 7 device jobs | none | ETH is the default again |
+
+## Rejected / not kept
+
+From the port and the baseline (PORT_LOG.md E20 / E22, OPT_BASELINE.md "precision cost"), kept here so the
+optimization rounds do not repeat them blindly:
+
+- **bf16 activations in the image branch** (`METEOR_IMAGE_PRECISION=bf16`): −165.9 ms per frame (−29.8 %), the
+  largest single saving measured, but it fails the depth argmax gate (agreement 0.9797 < 0.99): the 64 depth bins are
+  near-tied (13 % of the pixels have a top-2 logit margin < 0.02). The shipped sample's loose smoke comparison still
+  passes with it, which shows how much stricter the frozen gates are.
+- **ttnn's own fp32-activation conv path**: worse than bf16 (depth agreement 0.9617): it is not TF32 quality.
+- **Two-term activations with fp32 weights**: emulated 0.9940 on the depth argmax gate as a whole (per-layer two
+  terms are untested).
+- **A bf16 det stem / box refiner** (`det_precision="bf16"`): −41 ms, but the strict det3d recall on the in-domain
+  METEOR frame falls to 0.943 < 0.95 (PORT_LOG E22).
+- **The near-tie aware det3d gate** (E21): withdrawn; the strict PLAN gate is kept on every frame.
+- **WORKER dispatch as the default**: 22.1 ms slower per frame (578.9 ms, 11×10) and a 13-minute JIT compile; it was
+  the default only while the ETH hang was open.
+- **A second command queue**: ETH-2CQ replays 4.6 ms slower and a synchronous request gains nothing (D14).
+
+## Profile at the end (trace replay)
+
+The baseline profile is the current one (OPT_BASELINE.md "Device profile"): ReshapeView 128.5 ms (23.1 %),
+BinaryNg 85.7, PaddedSlice / SliceWrite / Slice 72.6, Conv2d 65.2, Matmul 54.3, Typecast 50.9, Tilize / Untilize / Pad
+36.3, GridSample 28.2 ms.
+
+## Remaining backlog (gains on the device frame, 556.8 ms, unless marked e2e)
+
+The ranked opportunities of OPT_BASELINE.md; the estimates overlap and do not add up. Each item keeps the frozen
+gates and the precision policy (PLAN §9.3) and lands as one commit with one `METEOR_*` A/B knob.
+
+1. **TILE reshape copies** (−90..−120 ms): 435 `ReshapeView` copies (128.5 ms) between `[1, 1, N·H·W, C]` and
+   `[1, H, W, C]` at map widths that are not multiples of 32 (500, 250, 48, ...). One layout per stage, a flattened
+   input path for `Resize2d`, ROW_MAJOR views where page-aligned (safe again under ETH with the SYS-1419 patch), or
+   padding the BEV width to 512 / 256. The same C17 / C24 item as CenterPoint and BEVFormer (shared ttaw fix).
+2. **Three-term convs as one fused op** (−90..−150 ms, precision kept): a conv kernel that splits hi / lo in the
+   reader and accumulates the three products in the fp32 dest removes 359 typecasts (50.9 ms), most of the 440
+   BinaryNg adds and two of three activation reads. Or a TF32+-quality fp32-activation conv (I7).
+3. **Input path and stem** (−35..−45 ms; e2e −60 ms): the uint8 -> bf16 typecast over 3-byte pages (8.6 ms), the
+   stem's 6 DRAM slices of the 3-channel input (13.9 ms), the exact fp32 max pool as a DRAM chain (25.3 ms; a
+   generic_op pool about 1-3 ms, UNVERIFIED), and the H2D of 2.65 M pages of 3 bytes (66.4 ms; whole camera rows
+   about 2-5 ms, UNVERIFIED).
+4. **DRAM slicing and conv glue** (−25..−40 ms): 386 PaddedSlice + 386 SliceWrite (49.3 ms) and the halo / sharding
+   glue (19.9 ms) of the DRAM-sliced convs.
+5. **Lift as one gather kernel** (−30..−45 ms): two `grid_sample` calls of 14.1 ms writing 256 MB, then tilize,
+   typecast, the product with the 205 MB fp32 table and the sums; a kernel per BEV block with L1-resident camera maps
+   and per-calibration ELL weights writes only `[400, 250, 96]`.
+6. **Few-core and fp32 matmuls** (−12..−20 ms): `head.ego_attn` (5.10 ms) and `head.risk_sample` (4.81 ms) run on 3
+   cores; the W-axis `Resize2d` matmuls at 250 -> 500 on 16-24 cores (2.9 ms each).
+7. **Host side** (e2e −150..−220 ms): the host decode (124-131 ms: the seg fusion and the 2D peak loops), the readback
+   conversion (61-65 ms) and D2H of 75 MB fp32 (bf16 / uint8 outputs in final layouts), the NCHW -> NHWC transpose
+   (22-26 ms).
+8. **Throughput by pipelining** (frames/s, latency unchanged): 2 CQs with double-buffered I/O, bounded by the
+   device's 1.8 frames/s.
+9. **Megakernels and fusions** (D19): MK4 band-streamed BEV trunk (197 ms of BEV + refiners, 91 ms of it reshapes)
+   first, then MK1 / MK2 image encoder + heads (after item 2), MK3 lift, MK0 input, MK6 planner, MK7 output tail.
+   Dispatch is not a sink (0.5 % of the frame), so a megakernel pays only through the DRAM round trips and layout
+   copies it removes.

PYTHON.md

@@ -0,0 +1,165 @@
+# Python API: METEOR (TIER IV, AutowareFoundation/meteor) on Blackhole
+
+Use this API from Python code (a pipeline, a notebook, a ROS 2 node wrapper). You do not need the HTTP server: the
+API and the server share the decoders, the device graph and the post-processing, so the outputs and the speed are
+the same.
+
+## Install
+
+Install the package on top of an environment that already has `ttnn`: a tt-metal `python_env` at `44d66500520`
+with `patches/tt-metal-eth-dispatch.patch` and `patches/tt-metal-reshape-rm-sys1419.patch` applied, or the tt-model
+container. From the root of the model repository (the directory that holds `pyproject.toml`, `README.md` and
+`code/`):
+
+```bash
+pip install -e .                        # the Python API (numpy<2, pillow, pyyaml, onnx, huggingface_hub, safetensors, opencv-python-headless)
+pip install -e ".[server,test]"         # + the HTTP server and the tests
+```
+
+The pip project is the repository's top-level `pyproject.toml`; it installs the package from `code/tt_meteor` (there
+is no `pyproject.toml` inside `code/`, because the container build copies `code/` over the tt-metal tree). `ttnn`
+and `torch` are not declared, so pip never replaces tt-metal's own build; tt-metal's `python_env` already has
+OpenCV 4.8.1, which satisfies the `opencv-python-headless>=4.8,<4.12` requirement.
+
+The package carries `tt_meteor.ttaw`, the shared code of the Autoware ports to Blackhole (device open, trace runner,
+decoders, model base class, HTTP app), vendored at the version recorded in `code/tt_meteor/ttaw/VENDORED.json`
+(0.23.2).
+
+| You want to run | Extras |
+|---|---|
+| the Python API | none |
+| the HTTP server (`tt_meteor.server.app`, see `SERVING.md`) | `server` |
+| host tests (no device; the device tests are skipped): `TT_VISIBLE_DEVICES=none python -m pytest -q code/tt_meteor/tests` | `server,test` |
+| device tests: `python -m pytest -q -s code/tt_meteor/tests/test_pcc_device.py code/tt_meteor/tests/test_e2e_device.py code/tt_meteor/tests/test_variants_device.py` (most of them need the fp32 goldens of the development workspace and skip without them) | `test` |
+
+## Quickstart
+
+```python
+from tt_meteor import METEOR, load_sample
+
+with METEOR.from_pretrained(device_id=0) as model:
+    out = model(**load_sample("code/tt_meteor/samples/synthetic_8cam.json"))   # 8 cameras + calibration + ego speed + stream
+print(out.to_dict()["plan"]["mode"], [d["label"] for d in out.to_dicts()])
+```
+
+`examples/quickstart.py` runs the same snippet and writes the `/predict` JSON and a bird's-eye view
+(`quickstart_bev.png`). The shipped sample is a synthetic test frame generated by this repository (see
+`code/tt_meteor/samples/README.md`); feed your own rig's eight images and calibration for real use.
+
+## `METEOR.from_pretrained(...)`
+
+```python
+METEOR.from_pretrained(
+    model_id=None,          # HF repo or a local directory with the weights files; default AutowareFoundation/meteor
+    *,
+    revision=None,          # default for the default repo: the validated commit 01a5f6d71df (tag v1.0)
+    variant=None,           # load-time variant: "default" (the only one); default $METEOR_VARIANT
+    device_id=None,         # chip to open; default $TT_DEVICE_ID or 0
+    device=None,            # an already-opened ttnn device (tt_meteor.device.open_device); close() does not close it
+    dispatch=None,          # "eth" (p150 target, 12x10 grid) | "worker" (A/B only, 11x10) | "auto"; default $METEOR_DISPATCH or "eth"
+    num_command_queues=None,  # default $METEOR_NUM_CQS or 1 (D14: a second queue gains nothing for synchronous calls)
+    weights_dir=None,       # explicit local weights directory; no Hub access
+    warmup_variants="default",  # trace variants to capture now; see "Warm-up"
+    verbose=False,
+    **compile_params,       # load-time knobs (below), e.g. input_norm="imagenet"
+) -> METEOR
+```
+
+What it does: resolves the weights first, so a Hub problem never claims the chip (`weights_dir` >
+`$METEOR_WEIGHTS_DIR` > a local `model_id` directory > the HF snapshot at the pinned revision, restricted to
+`meteor_v157c3Z.onnx, meteor_v157.param.yaml, LICENSE, SHA256SUMS`, with an offline fallback to the cache), opens the
+chip (ETH dispatch, 12×10, 1 CQ; the other open parameters are `DEVICE_DEFAULTS` in `tt_meteor/device.py`:
+`l1_small_size` 32 KiB, `trace_region_size` 256 MiB, overridable with `METEOR_*`), reads the parameters of the ONNX
+file by consuming node, builds the graph, then compiles and captures the metal trace. If ETH dispatch cannot open
+(tt-metal without the patch), it warns and falls back to WORKER dispatch.
+
+Load-time knobs (`compile_params` in lower case, or the environment variable `METEOR_<NAME>`; read once at load):
+
+| knob | default | meaning |
+|---|---|---|
+| `input_norm` | `onnx` | `onnx` = the released graph's `/255` (ONNX Runtime parity; decision D12); `imagenet` = the trained ImageNet mean / std (upstream export defect, research/meteor SPEC section 10 risk 1); with `imagenet` an absent camera is zeroed after normalising, as in training |
+| `depth_mean_bins` | `log` | `depth_mean` bin centres: log-spaced as exported (parity) or `linear` 1 + 1.25 b m (the trained bins) |
+| `max_streams` | 16 | host temporal states kept (seg fusion, yaw tracks, mode hysteresis); the least recently used is dropped |
+| `image_precision` | `terms3` | image branch with fp32 activations, every conv as three bf16 terms (needed by the depth / seg2d argmax gates); `bf16` is an A/B only: 166 ms faster per frame, but below the depth argmax gate (0.9797 < 0.99) |
+
+## Warm-up
+
+The load runs one eager frame of the whole graph (it compiles every kernel and fills the program cache) and then
+captures it as ONE metal trace, `frame` (4,678 programs). The first call is therefore as fast as the later ones.
+With a warm JIT cache the load takes about 45-50 s (49.7 s measured: weights 0.3 s, graph build 2.6 s, eager frame
+41.0 s, capture 1.2 s). A cold kernel cache compiles for minutes (a new dispatch configuration on the build host:
+ETH-2CQ 142 s, WORKER 11×10 768 s). `warmup_variants="none"` skips the capture (the first call then captures).
+
+## Call: `model(...)`
+
+| Argument | Type | Description |
+|---|---|---|
+| `images` | list or mapping | the cameras `CAM_FRONT_WIDE`, `CAM_FRONT_LEFT`, `CAM_FRONT_RIGHT`, `CAM_BACK_WIDE`, `CAM_BACK_LEFT`, `CAM_BACK_RIGHT`, `CAM_FRONT_NARROW`, `CAM_BACK_NARROW`, any order: `CameraImage`s, dicts `{"camera", "image", "intrinsics", "T_ref_from_camera"}` or `{name: image}`. Raw, unrectified frames of at least 768×432; larger frames are resized with OpenCV INTER_AREA semantics (bit-exact numpy port) and K is scaled per axis. The six wide / corner cameras are required; a missing or all-zero narrow camera is **absent**: a zero image plus its donor's K and pose (`CAM_FRONT_WIDE` / `CAM_BACK_WIDE`), the trained 7-camera configuration |
+| `calibration` | dict | `{"cameras": {name: {"intrinsics", "T_ref_from_camera"}}}` (K of the image as sent; camera optical frame -> base_link: x forward, y left, z up, origin on the road) or `{"preset": name}` (`code/tt_meteor/calib/`); inline calibration of a camera wins |
+| `ego_speed` | float | m/s, the graph's `v0` (required) |
+| `stream` | dict | optional: `{"id", "reset", "timestamp_s", "T_world_from_ego"}` (or `"pose": [x, y, yaw]`): METEOR's host temporal post-processing per stream id (BEV seg fusion needs the pose; yaw smoothing; plan-mode hysteresis). A new id or `reset=True` starts fresh |
+| runtime params | keyword | host post-processing only (`METEOR.RUNTIME_PARAMS`, METEOR's C++ renderer defaults): 3D `det3d_threshold` 0.15, `det3d_topk` 64, `vehicle_threshold` 0.35, `vru_threshold` 0.15, `bev_nms_iou` 0.3, `bev_nms_containment` 0.6, `stationary_logit_threshold` 0.0; 2D `det2d_threshold` 0.30, `det2d_topk` 48, `det2d_hide` "7" (road paint); `unk2d` True, `unk2d_threshold` (= det2d), `ground_z` 0.0; plan `mode_hysteresis` 0.35, `straight_margin` 1.0; `seg_fuse` True, `thin_road_edge` True, `yaw_smoothing` True; `heads` False (adds the dense heads to `to_dict("npz")`) |
+
+`tt_meteor.load_sample(path)` turns a sample manifest (`samples/<name>.json`: image paths relative to the file, a
+preset, the ego speed and a stream) into these keyword arguments.
+
+### Input types
+
+- Images: a path, PNG / JPEG bytes, a `PIL.Image`, a uint8 H×W×3 RGB array, or a float array in [0, 1].
+- Transforms: 4×4 (or 3×4) matrices, `{"translation", "rotation_wxyz"}` (or `rotation_xyzw`), or Autoware
+  `{"x", "y", "z", "roll", "pitch", "yaw"}`. Intrinsics: 3×3 K, 3×4 P or `{"fx", "fy", "cx", "cy"}`.
+
+## Output
+
+`model(...)` returns a `MeteorOutput` (`tt_meteor.Output`); `out.to_dict()` is the `/predict` JSON:
+
+| key | content |
+|---|---|
+| `detections` (`out.to_dicts()`) | 3D BEV boxes sorted by score: `label` (VEHICLE / VRU), `label_id`, `score`, `center` [x, y] and `size` [length, width] in metres (base_link; METEOR predicts no z, height or velocity), `yaw` (rad, CCW from +x), `stationary`, `future` (6 × [x, y] at 0.5 .. 3 s, the agent's best mode), `future_mode` |
+| `trajectory`, `columns` | the selected ego path, 6 × [x, y] at 0.5 s steps |
+| `plan` | `mode`, `mode_probs` (softmax of the hysteresis-adjusted logits), `mode_logits`, `paths` (3 × 6 × [x, y]), `steer` (rad), `accel` (m/s²), `brake_prob`, `dt` |
+| `detections_2d` | per camera: `label` (10 classes), `label_id`, `score`, `box_xyxy` (pixels at 768×432) |
+| `unknown_obstacles` | [x, y] of 2D "obstacle" boxes placed on the ground plane (METEOR's `unk2d`) |
+| `traffic_light` | `state` (none / green / yellow / red), `state_id`, `probs` |
+| `lane`, `lane_classes` | the BEV lane map, uint8 [800, 500] as PNG (0.2 m cells, row 0 = +80 m ahead, column 0 = +50 m left; classes bg, road, sidewalk, crosswalk, laneline, stopline, road_edge, marking, parking), after the optional seg fusion and road-edge thinning |
+| `stationary_head_healthy`, `meta` | METEOR's health check of the stationary head; `present` cameras, `v0`, `stream_id` |
+| `heads` (`to_dict("npz")` with `heads=True`) | the dense outputs: `seg2d`, `depth` (bins), `depth_mean`, `occupancy` (class per voxel), `risk` (sigmoid), `stationary` |
+| `timing_ms` | `preprocess`, `device` (tables + upload + replay + read + conversion), `postprocess`, `total` |
+
+`out.path` is the selected path as an array; `out.lane` the lane map; `out.boxes3d` / `out.boxes2d` the box objects.
+
+## Lifetime and information
+
+- `model.close()` releases the trace and the device tensors and closes the chip if the model opened it; idempotent.
+  `with` calls it for you; an unclosed model is closed when Python exits.
+- `model.info`: weights (repo, tag, revision, path), device (dispatch, grid), variant, warm variants, warm-up times,
+  runtime parameter defaults, the load-time knobs and the trace description.
+- Calls from several threads are safe: the device calls are serialised. One model per process per chip.
+- **Rig changes.** The lift tables (205 MB of fp32 bin weights + the sampling grid) are built on the host per
+  calibration (cached per rig) and written to the chip only when the calibration changes: 182 ms for the write, so a
+  call with a new rig takes about 1.13 s instead of about 0.9 s. A fixed rig pays it once.
+
+## Speed
+
+Warm, batch 1, ETH dispatch, 1 CQ, 12×10, AICLK 1350 MHz, p50 of 60 calls on a shared 8-core host
+([`OPT_BASELINE.md`](OPT_BASELINE.md); the device rows do not depend on the input):
+
+| stage | ms |
+|---|---:|
+| device trace, one replay (the whole network) | 556.85 (back to back 556.75: 1.80 frames/s) |
+| upload of the eight cameras (8 MB uint8) | 66.5 |
+| readback (75 MB, 5 segments) + conversion to the 19 outputs | 17-48 + 65 |
+| host pre-processing / camera transpose | 9-11 / 22-26 |
+| host post-processing (METEOR's C++ decode rules, temporal state) | 124-131 |
+| `model(...)` end to end | 876-918 (1.09-1.14 calls/s) |
+
+This is the first release: optimization has not started ([`OPT_REPORT.md`](OPT_REPORT.md)).
+
+## Limits
+
+- Batch 1 on the chip; one model per process; eight 768×432 camera slots in METEOR's order, a 400×250 lift grid at
+  0.4 m and an 800×500 BEV at 0.2 m (fixed shapes of the released graph); at most 3 cameras per lift cell on the
+  rigs validated.
+- The released graph is camera-only and single-frame (its temporal memory was baked out upstream); the temporal
+  behaviour of METEOR's runtime is host post-processing.
+- Numerics: bf16 / fp32 on the chip; outputs differ slightly from the fp32 reference (README "Demo & Performances").

README.md

@@ -0,0 +1,199 @@
+---
+tags:
+- blackhole
+- p150
+- tt-dit-server
+- tt-model-cache
+- tt-model-container
+- tenstorrent
+- ttnn
+- tt-metal
+- tt-nn
+- autoware
+- autonomous-driving
+- camera
+- multi-view
+- bird-eye-view
+- multi-task
+- e2e
+- planning
+- 3d-object-detection
+- semantic-segmentation
+- depth-estimation
+- meteor
+- tt-model-catalog
+pipeline_tag: robotics
+license: apache-2.0
+license_link: https://huggingface.co/AutowareFoundation/meteor
+base_model:
+- AutowareFoundation/meteor
+---
+
+# meteor-p150
+
+METEOR, TIER IV's surround-view multi-task driving network released by the Autoware Foundation (`meteor_v157c3Z.onnx`, `DepthSegIPMNetV52`: a ResNet-34 + FPN encoder over eight cameras, image heads, a depth-gated IPM lift into one 96-channel 800×500 bird's-eye-view map at 0.2 m, BEV heads, an ego planner and three residual refiners; 48.25 M parameters, 2.58 TFLOP per frame), ported to one Tenstorrent Blackhole p150 with tt-nn. Eight camera images (768×432 RGB, raw and unrectified) with their intrinsics and extrinsics and the ego speed in; a BEV lane map, 3D boxes (vehicle / VRU) with stationary flags and agent futures, per-camera 2D boxes, 2D segmentation and depth, 3D occupancy, the ego-relevant traffic light, a risk field and three ego paths with steer / accel / brake out.
+Weights: [AutowareFoundation/meteor `v1.0`](https://huggingface.co/AutowareFoundation/meteor/tree/01a5f6d71df5ecbbb5853ec600825481d57b9c6b) · Paper: none (GTC 2026 talk S81897; [tier4/METEOR README](https://github.com/tier4/METEOR/blob/dc193a81e9959de57b751e269127ae9b67e3f300/README.md)) · Autoware package: none (METEOR's own runtimes, [tier4/METEOR `deploy/cpp`](https://github.com/tier4/METEOR/tree/dc193a81e9959de57b751e269127ae9b67e3f300/deploy/cpp)) · Training code: [tier4/METEOR](https://github.com/tier4/METEOR/tree/dc193a81e9959de57b751e269127ae9b67e3f300) (`bevlane/train.py`) · Port: [`code/`](https://huggingface.co/changh95/meteor-p150/tree/main/code)
+
+Runs on **p150** (mesh `P150`). Configuration: dispatch on the ETH cores, 1 command queue, 12×10 compute grid. The whole network runs on the chip as ONE metal trace per frame (`frame`, 4,678 programs: the image encoder and heads of the eight cameras, the lift, the BEV trunk and heads, the planner, the refiners and the output packing); the image resize, the lift geometry tables (once per calibration), METEOR's C++ decode rules and the temporal state run on the host. All numbers on this card were measured in this configuration.
+
+Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1).
+
+## Quickstart (Python)
+
+Prerequisite: a tt-metal / ttnn environment at tt-metal [`44d66500520`](https://github.com/tenstorrent/tt-metal/commit/44d66500520fda9f2c7060c0f6b41ec48f7ab37e) with [`patches/tt-metal-eth-dispatch.patch`](patches/tt-metal-eth-dispatch.patch) and [`patches/tt-metal-reshape-rm-sys1419.patch`](patches/tt-metal-reshape-rm-sys1419.patch) applied (the second one fixes an intermittent hang of row-major reshapes under ETH dispatch on Blackhole, see Caveats). ttnn is not on PyPI.
+
+```bash
+hf download changh95/meteor-p150 --exclude "image/*" --local-dir meteor-p150 && cd meteor-p150
+pip install -e .                        # adds numpy<2, pillow, pyyaml, onnx, huggingface_hub, safetensors, opencv-python-headless; ttnn and torch come from tt-metal
+pip install -e ".[server,test]"         # optional: the HTTP server and the tests
+```
+
+Run the snippet from the model repo root (the sample path is relative to it).
+
+```python
+from tt_meteor import METEOR, load_sample
+
+with METEOR.from_pretrained(device_id=0) as model:      # weights -> your HF cache, trace captured
+    out = model(**load_sample("code/tt_meteor/samples/synthetic_8cam.json"))   # 8 cameras + calibration + ego speed + stream
+
+for d in out.to_dicts():
+    print(d["label"], d["score"], d["center"], d["size"], d["yaw"], d["stationary"])
+body = out.to_dict()
+print(body["plan"]["mode"], body["trajectory"], body["traffic_light"]["state"])
+```
+
+- `from_pretrained` downloads [`AutowareFoundation/meteor`](https://huggingface.co/AutowareFoundation/meteor) at the pinned commit `01a5f6d71df` (tag `v1.0`; `meteor_v157c3Z.onnx`, 335 MB, plus `meteor_v157.param.yaml`, `LICENSE`, `SHA256SUMS`) to your HF cache, opens the chip, builds the graph and captures the metal trace. The first load on a machine compiles the kernels (minutes: the container's first boot on an empty JIT cache was ready after 799 s); later loads take about 45-50 s (49.7 s measured with a warm kernel cache: most of it is the eager warm-up frame of 4,678 programs).
+- The first call is as fast as the later calls: the load runs the whole graph eagerly before it captures the trace.
+- The shipped sample is a synthetic test frame (a rendered street seen by a generic eight-camera rig; see [`code/tt_meteor/samples/README.md`](code/tt_meteor/samples/README.md)); feed your own rig's eight images, calibration and speed for real use.
+- The `with` block releases the trace and closes the chip. Without `with`, call `model.close()`.
+
+| | |
+|---|---|
+| **Input** | `images`: the eight cameras `CAM_FRONT_WIDE`, `CAM_FRONT_LEFT`, `CAM_FRONT_RIGHT`, `CAM_BACK_WIDE`, `CAM_BACK_LEFT`, `CAM_BACK_RIGHT`, `CAM_FRONT_NARROW`, `CAM_BACK_NARROW`, any order (path, bytes, PIL, uint8 array; raw, unrectified, at least 768×432, resized with OpenCV INTER_AREA semantics). An absent narrow camera (omitted, or an all-zero image) gets a zero input and its donor's pose, as trained. `calibration`: per camera `intrinsics` (of the image as sent) and `T_ref_from_camera` (camera optical frame -> base_link), or a preset. `ego_speed` (m/s). Optional `stream`: `id`, `reset`, `T_world_from_ego` for METEOR's host temporal state (BEV seg fusion, yaw smoothing, plan-mode hysteresis). |
+| **Options** | Host post-processing (METEOR's C++ renderer defaults): `det3d_threshold=0.15`, `det3d_topk=64`, `vehicle_threshold=0.35`, `vru_threshold=0.15`, `bev_nms_iou=0.3`, `bev_nms_containment=0.6`, `det2d_threshold=0.30`, `det2d_topk=48`, `det2d_hide="7"`, `unk2d=True`, `mode_hysteresis=0.35`, `straight_margin=1.0`, `seg_fuse=True`, `thin_road_edge=True`, `yaw_smoothing=True`, `heads=False`. Load time: `from_pretrained(device_id=0, dispatch="eth", weights_dir=None, device=None, input_norm="onnx", depth_mean_bins="log", max_streams=16)`. |
+| **Output** | `MeteorOutput`: 3D BEV boxes in base_link (`center` [x, y], `size` [length, width], `yaw`, `score`, `label` VEHICLE / VRU, `stationary`, `future`: 6 × [x, y] at 0.5-3 s), the selected ego path (6 × [x, y]) and the plan (`mode`, `mode_probs`, 3 `paths`, `steer`, `accel`, `brake_prob`), 2D boxes per camera (10 classes), unknown obstacles, the traffic-light state, the BEV lane map (uint8 800×500, 9 classes), `timing_ms`. |
+| **Methods** | `out.to_dict()` gives the `/predict` JSON (`out.to_dict("npz")` with `heads=True` adds seg2d, depth, depth_mean, occupancy, risk, stationary); `out.to_dicts()` one dict per 3D box; `out.path`, `out.lane`. |
+
+- The API gives the same output as the HTTP server `/predict`: both share the decoders, the device trace and the host post-processing. One model uses one chip; calls from several threads are serialised.
+- Full reference: [`code/PYTHON.md`](code/PYTHON.md). Runnable example: [`examples/quickstart.py`](examples/quickstart.py) (writes the `/predict` JSON and a bird's-eye view, `quickstart_bev.png`).
+
+## Serving (HTTP)
+
+```bash
+tt-model pull  changh95/meteor-p150 --with-weights
+tt-model serve changh95/meteor-p150       # or with tt-cli: tt serve changh95/meteor-p150
+python3 code/tt_meteor/server/client.py --sample code/tt_meteor/samples/synthetic_8cam.json --out req.json
+curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
+tt model stop changh95/meteor-p150
+```
+
+- The image does not contain the weights. `--with-weights` puts them in your HF cache.
+- The server uses port 20000 (or the next free port). It is ready when the log shows `Application startup complete`.
+- `POST /predict`: `images` (all eight cameras, base64 JPEG / PNG / `.npy`, rgb8; an absent narrow camera as an all-zero image), `calibration` (per camera `intrinsics` and `T_ref_from_camera`, or `{"preset": name}`), `ego_speed` (m/s, required); optional `stream` (`id`, `reset`, `T_world_from_ego`), `params`, `output_format`. For your own frame: `client.py --image CAM_FRONT_WIDE=f.jpg ... --calib rig.json --ego-speed 8.3`. Also `GET /health`, `GET /info`, `GET /v1/models` (stub). Contract: [`SERVING.md`](SERVING.md) section 3.
+
+The shipped sample (a request after the warm-up, host server, 2026-10-10; trimmed):
+
+```json
+{"model": "meteor-p150", "frame_id": "base_link", "num_detections": 8,
+ "detections": [
+   {"label": "VEHICLE", "label_id": 0, "score": 0.9932, "center": [-12.86, 0.095], "size": [4.399, 1.742], "yaw": -0.0687, "stationary": false, "future_mode": 0, "future": [[-11.92, -0.09], [-10.74, -0.15], ...]},
+   {"label": "VEHICLE", "label_id": 0, "score": 0.9872, "center": [-7.176, -3.215], "size": [3.965, 1.731], "yaw": 0.0183, "stationary": false, ...},
+   {"label": "VEHICLE", "label_id": 0, "score": 0.9624, "center": [10.84, 0.019], "size": [4.334, 1.761], "yaw": -0.0189, "stationary": false, ...},
+   ...],
+ "trajectory": [[0.74, 0.91], [0.85, 1.54], [0.41, 2.34], [0.31, 3.08], [0.62, 3.82], [0.3, 4.45]], "columns": ["x", "y"],
+ "plan": {"mode": 0, "mode_probs": [0.233, 0.296, 0.471], "mode_logits": [...], "paths": [...], "steer": 0.0026, "accel": 0.021, "brake_prob": 0.131, "dt": 0.5},
+ "detections_2d": {"CAM_FRONT_WIDE": [{"label": "car", "label_id": 1, "score": 0.5239, "box_xyxy": [412.76, 211.02, 438.5, 230.29]}, ...], ...},
+ "unknown_obstacles": [],
+ "traffic_light": {"state": "red", "state_id": 3, "probs": [0.363, 0.140, 0.031, 0.465]},
+ "lane": {"format": "png", "key": "lane", "dtype": "uint8", "shape": [800, 500], "data": "iVBORw0KGgo..."},
+ "lane_classes": ["bg", "road", "sidewalk", "crosswalk", "laneline", "stopline", "road_edge", "marking", "parking"],
+ "stationary_head_healthy": true,
+ "meta": {"present": [true, true, true, true, true, true, true, true], "v0": 8.0, "stream_id": "synthetic_8cam"},
+ "timing_ms": {"decode": 83.95, "preprocess": 10.69, "device": 722.42, "postprocess": 86.08, "model_call": 820.65, "total": 908.67}}
+```
+
+- Boxes, paths and the lane map are in base_link metres (x forward, y left, origin on the road below the ego reference point); METEOR predicts BEV boxes without z, height or velocity. `yaw` is counter-clockwise from +x; boxes are sorted by score.
+- `trajectory` is the selected ego path (METEOR's renderer: straight preference and mode hysteresis per stream); `plan.paths` holds all three modes. The synthetic sample is a test pattern (a red light, a car 11 m ahead): its plan barely moves forward (under 1 m in 3 s, drifting 4.5 m to the left) and is not meaningful driving; the stored CPU reference has the same path.
+- `lane` is a PNG of the 800×500 class map (row 0 = 80 m ahead, column 0 = 50 m to the left); `lane_classes` names its values.
+- `meta` carries the present cameras, `v0` and the stream id; `/info` reports the device (ETH, 12×10), the camera order, the calibration presets and the load-time knobs.
+
+## Demo
+
+| PandaSet 019, frame 40 (San Francisco; CC BY 4.0): TT output | nuScenes scene-0103, key-frame 9 (Boston; CC BY-NC-SA 4.0, non-commercial): TT output |
+|:---:|:---:|
+| ![PandaSet 019 frame 40: eight camera slots with 2D boxes, the bird's-eye view with lanes, risk, 3D boxes and the plan, the front view with 3D boxes and the planned path](media/meteor_ps019_f0020_card_tt.jpg) | ![nuScenes scene-0103 key-frame 9, non-commercial](media/meteor_ns0103_f0009_NC_card_tt.jpg) |
+
+| TT vs the fp32 CPU reference: PandaSet 019 frame 40, bird's-eye view | The shipped synthetic sample: TT vs the stored CPU reference |
+|:---:|:---:|
+| ![TT vs CPU, bird's-eye view](media/meteor_ps019_f0020_bev_tt_vs_cpu.png) | ![The synthetic sample](media/meteor_synthetic_8cam_tt_vs_cpu.png) |
+
+More: [PandaSet 019 frames 0-48 as a GIF](media/meteor_ps019_seq_tt.gif) (every 4th frame, real-time playback), [PandaSet 090 frame 40](media/meteor_ps090_f0020_card_tt.jpg), and the camera heads (2D segmentation overlay and depth per slot) of [PandaSet 019](media/meteor_ps019_f0020_heads_tt.jpg), [PandaSet 090](media/meteor_ps090_f0020_heads_tt.jpg) and [nuScenes scene-0103](media/meteor_ns0103_f0009_NC_heads_tt.jpg) (non-commercial). Every image shows outputs of this port on the p150, some next to the fp32 CPU reference; the BEV panels compare the plan with the logged ego path (white circles) and the 3D boxes with the dataset annotations (light blue). The public rigs map onto METEOR's eight slots as 6 real cameras, a virtual FRONT_NARROW centre crop of the front camera and an absent BACK_NARROW (zero image + the BACK_WIDE pose, as trained). Head and licence-plate regions of annotated people and vehicles are blurred. Sources, licences and changes: [`media/ATTRIBUTION.md`](media/ATTRIBUTION.md). The weights were trained on TIER IV's own rig in Japan only, so both datasets are out of domain: the detections show the domain gap, not the port.
+
+## Demo & Performances
+
+Warm, batch 1, eight cameras. Accuracy: the TT output against the port's fp32 CPU reference of the same network (same weights, same pre- and post-processing: METEOR's C++ runtime rules), which matches ONNX Runtime on `meteor_v157c3Z.onnx` (min PCC 0.99999990 over 155 taps and outputs). The gates are the PLAN thresholds, frozen at the first green run and never loosened. Speed: `code/scripts/bench.py`, p50 (p99) of 60 iterations, on a shared 8-core host ([`OPT_BASELINE.md`](OPT_BASELINE.md)).
+
+| Metric | Performance |
+|---|---:|
+| Per-stage PCC vs the CPU reference, teacher-forced (26 gates, PandaSet 019 frame 40) | every tap **≥ 0.999989** (gate 0.99); lane / seg2d / depth argmax agreement **0.9984 / 0.9965 / 0.9936** (gate 0.99) |
+| The served graph from the cameras, 4 frames of 3 rigs (PandaSet 019 / 090, nuScenes scene-0103, an in-domain METEOR demo frame) | min output PCC **0.99982**; worst lane / seg2d / depth agreement **0.9963 / 0.9965 / 0.9925**; selected ego path within **0.170 m** (gate 0.3 m); 3D boxes strict recall / precision **1.0 / 1.0** on three frames, **0.971 / 0.971** (34 / 35) on the METEOR frame (gate 0.95) |
+| 38 public frames (PandaSet 019 and 090: 13 frames each, nuScenes scene-0103: 12 key-frames) through the served graph | argmax agreement lane ≥ **0.9971**, seg2d ≥ **0.9962**, depth ≥ **0.9925**; dense-head PCC ≥ **0.9999** (hm, reg, stationary; risk ≥ 0.9996); all 19 outputs on one frame per sequence: PCC ≥ **0.9990** (`depth_mean`; the rest ≥ 0.99989); selected ego path **0.04-0.11 m** from the CPU path (mean distance), the same plan mode and traffic-light state on all 38 frames; 3D boxes: every TT box matches a CPU box (precision 1.0), **408 of 418** CPU boxes matched (0.976): 9 of the 10 misses are CPU boxes within 0.004 of the class threshold (0.3507-0.3538 vs 0.35, 0.1505-0.1513 vs 0.15), the 10th a pedestrian whose TT peak lies 2.1 m away |
+| Planned path vs the logged ego path (ADE, frames with a full 3 s future; a sanity metric, METEOR has no paper metric) | PandaSet 019 **TT 0.598 m** / CPU 0.552 m; PandaSet 090 2.570 / 2.472 m; nuScenes scene-0103 1.846 / 1.885 m |
+| The shipped synthetic sample vs its stored CPU reference (the container smoke's check) | **8 / 8** boxes (recall 1.000, precision 1.000), max \|score difference\| **0.0020**, lane map agreement **0.9987**, path ADE / FDE **0.018 / 0.030 m** |
+| `input_norm=imagenet` + `depth_mean_bins=linear` (the load-time option, two absent cameras) vs its own CPU reference | every float output PCC ≥ **0.99988**; lane / seg2d / depth agreement 0.9980 / 0.9947 / 0.9934 |
+| Device, one frame (`frame` replay, 4,678 programs) | **556.85 ms**; back to back **556.75 ms**: **1.80 frames/s** device-bound |
+| Python `model()`, the PandaSet 019 frame (eight 768×432 JPEGs, decoded) | **918.2 ms** (1,071.8): preprocess 11.2, camera transpose 26.3, upload 66.5, device 556.9, readback 17.2 + conversion 65.3, postprocess 129.9 |
+| Python `model()`, other inputs | synthetic noise 875.7 ms, PandaSet 090 902.8 ms, nuScenes 914.4 ms; the shipped sample **871.6 ms** (release re-check, 30 iterations) |
+| A call with a new calibration (rig change) | lift-table write **182.3 ms**; call 1,128.5 ms |
+| Served `/predict` on the host, shipped sample (uvicorn, loopback, after the first request) | `timing_ms.total` **904-909 ms** (decode of the 1.8 MB body 81-84, preprocess 10.6, device 720, postprocess 86-87); client round trip 0.92-0.93 s; boot to ready 44.6 s (warm kernel cache) |
+| ETH-dispatch stress (`stress_frames.py --mode rigs`: a rig change, table write, upload, replay and bit-compared read every frame) | **2,000 frames** clean on the baseline code (4 public rigs); **200 frames** clean in this release (4 synthetic rigs, the golden-free mode), and 200 more inside the published container image (per-rig output hashes identical to the host run) |
+
+All numbers in this table were measured with dispatch on the ETH cores, 1 command queue and a 12×10 compute grid on one p150, AICLK 1350 MHz. The device is kernel-bound (555.3 ms of kernels and 2.74 ms of op-to-op gaps for 4,678 programs); the image branch takes 268 ms of it (the three-term fp32 convs of the precision policy), and 57 % of the replay is layout, data movement and precision glue. The host rows move with the load of the shared host (a busier host added 18 ms to the PandaSet call in the release re-check). This is the first release: optimization has not started ([`OPT_REPORT.md`](OPT_REPORT.md) ranks the plan; [`OPT_BASELINE.md`](OPT_BASELINE.md) has every measurement). Verification: [`VERIFICATION_2026-10-10.md`](VERIFICATION_2026-10-10.md).
+
+Box agreement: a CPU box counts as matched when a TT box of the same class lies within 1 m (0.5 m for the gated frames and the shipped sample), one to one. The public frames are out of domain for METEOR, so they test the port (the same network on new rigs, lift tables and scenes), not the model's accuracy. nuScenes dataset © Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of Use (https://www.nuscenes.org/terms-of-use); non-commercial: the raw data were used for local validation only and are not distributed here (only the `_NC` renders in `media/` are); Motional does not endorse this work. Cite: H. Caesar et al., *nuScenes: A Multimodal Dataset for Autonomous Driving*, CVPR 2020.
+
+No GPU comparison: no GPU was available on the host where this port was built and measured, so this card makes no GPU speed claim. The reference rows are the port's own fp32 CPU reference on the same host (a correctness baseline, not a speed target). Upstream reports 67.4 ms median per eight-camera frame on a Jetson AGX Orin (TensorRT INT8 with the 2:4-sparse trunk, CUDA Graph, zero-copy input; tier4/METEOR README) and about 30 ms with TensorRT fp16 on a data-centre GPU (the upstream model card); both are other precisions and, on the Orin, a different lift (the optional plugin path uses a coarser 0.8 m lift grid with the calibration baked in), so they are not like-for-like. p150 power was not measured, so no efficiency comparison is made.
+
+## Caveats
+
+- **Deployment status.** There is no Autoware package for METEOR: neither autoware_universe nor the Autoware launch / ansible artifacts contain it; the upstream card and README list an Autoware (ROS 2) node as future work. METEOR is a reference model with its own Python and C++ TensorRT runtimes ([tier4/METEOR](https://github.com/tier4/METEOR)), whose pre- and post-processing this port follows. It is not a ROS node and not a certified Autoware component: do not use it for safety-critical driving decisions (the upstream card: a research artefact that must not control a vehicle on public roads).
+- **First release: optimization pending.** This is the functional baseline port; no optimization round has run yet ([`OPT_REPORT.md`](OPT_REPORT.md) ranks the plan: TILE reshape copies, fused three-term convs, the input path, the lift as one gather kernel).
+- **ETH-dispatch hang, fixed.** During the port the model hung intermittently under ETH dispatch (about 1 in 300 frames). The cause was one row-major `ttnn.reshape` writing 192-384 B DRAM pages from both NOCs (the Blackhole DRAM-arbiter issue SYS-1419), reproduced standalone in plain ttnn. `patches/tt-metal-reshape-rm-sys1419.patch` (built into the image) routes small destination pages to a single-kernel path; with it, 20,000 replays of the repro and two 2,000-frame rig-switch stress runs of the whole model were clean. Without the patch, ttaw 0.23.2 switches that resize tail to a TILE reshape (also clean, 38 ms slower per frame). Both patches are needed for the published configuration.
+- **Precision** (the first release's policy: fp32 weights and biases, HiFi4, fp32 dest and packer L1 accumulation). The image branch and the 3D detection path run with fp32 activations, every conv as three bf16 terms (bf16 activations fail the depth argmax gate: 0.9797 < 0.99); the BEV trunk runs bf16 activations; the lift sums, the planner and every value that feeds a threshold, a softmax or an argmax are fp32. Outputs differ slightly from the fp32 reference (agreement figures above); objects and plan modes near a threshold can flip. The three-term image branch costs 166 ms of the 557 ms frame.
+- **Documented behaviour and deviations** (PORT_LOG.md section 2, [`SERVING.md`](SERVING.md)):
+  - The input normalisation is the released graph's `/255` (ONNX Runtime parity; decision D12), although METEOR was trained with ImageNet mean / std (an upstream export defect): `input_norm="imagenet"` (`METEOR_INPUT_NORM`) gives the trained normalisation, validated on the chip against its own CPU reference.
+  - `depth_mean` keeps the exported log-spaced bin centres (parity); the trained bins are linear, 1 + 1.25 b m: `depth_mean_bins="linear"`.
+  - The temporal state (seg fusion, yaw tracks, plan hysteresis) is kept per `stream.id` on the host and resets on a new id or `reset`; upstream resets only the seg accumulator on a pose jump.
+  - The released graph is single-frame and camera-only (its temporal memory and LiDAR input were baked out upstream).
+- **Domain gap.** The weights were trained on TIER IV's eight-camera DRS rig in Japan (98 deg wide cameras, corner cameras pitched about 25 deg down, vertically squashed images). Other rigs work (the network takes K and T as inputs) but lose accuracy: on PandaSet and nuScenes the depth head is the main gap (only the camera with matching intrinsics gets METEOR-grade depth), 3D boxes are accurate but sparse, and the planner follows steady driving but not unseen intent (fp32 CPU reference on PandaSet 019: vehicles recall / precision 0.46 / 0.91 within 20 m of the LiDAR-supported annotations, planned-path ADE 0.55 m against the logged path vs 0.96 m on in-domain METEOR demo frames; the front camera's depth reads 1.65× the LiDAR range; on nuScenes scene-0103 the BEV drivable-area IoU is 0.39 against 0.70-0.74 on PandaSet).
+- **Fixed shapes.** Eight 768×432 camera slots in METEOR's order, a 400×250 lift grid at 0.4 m, an 800×500 BEV at 0.2 m (±80 m × ±50 m); batch 1, one frame per request; requests are serialised on the chip. A new calibration costs about 0.2 s once (the lift tables, cached per rig).
+- **Validation scope.** Agreement with the fp32 CPU reference on public data (PandaSet, nuScenes), on an in-domain METEOR demo frame and on the synthetic sample; no dataset-level metric exists for METEOR (no paper, no public split). The shipped sample is synthetic; a PandaSet sample is prepared but not shipped yet.
+- Does not scale to multiple p150 in a mesh configuration. The build uses a 12×10 compute grid of Tensix cores: the dispatch functions move from one Tensix column to the ETH cores (`patches/tt-metal-eth-dispatch.patch`), so this build assumes that you do not need chip-to-chip ethernet communication.
+- `dispatch="worker"` (server: `METEOR_DISPATCH=worker`) is an A/B opt-in. On a p150 it gives an 11×10 grid (578.9 ms per frame with 2 CQs; the gates passed under WORKER during the port). If ETH dispatch is not available (tt-metal without the patch), the model falls back to it with a warning. The numbers on this card do not apply to that mode.
+- Not an OpenAI-compatible API; `GET /v1/models` is a stub so the tt-model ready card does not 404.
+
+## Licensing
+
+- Weights: [AutowareFoundation/meteor](https://huggingface.co/AutowareFoundation/meteor) at tag `v1.0` (commit `01a5f6d71df5ecbbb5853ec600825481d57b9c6b`), Apache-2.0 per its model card and `LICENSE`. Not redistributed here: the package only points to them. The upstream card: trained with `bevlane/train.py` on TIER IV Co-MLOps DRS recordings in Japan, every supervision signal auto-generated (no human labels; adverse conditions added as NVIDIA Cosmos Transfer re-renderings of real scenes); the recordings are not public, and accuracy figures are measured on an internal split and not published. The AutowareFoundation/meteor-demo-scenes frames are for research and demonstration use only and are never redistributed or rendered here.
+- Pre- and post-processing ported from METEOR's own runtimes, [tier4/METEOR](https://github.com/tier4/METEOR/tree/dc193a81e9959de57b751e269127ae9b67e3f300) `deploy/cpp` and `hf/onnx_smoke_test.py` (Apache-2.0); there is no Autoware package.
+- Port and serving code (`code/`): Apache-2.0.
+- Sample data: `code/tt_meteor/samples/synthetic_8cam*` and the preset `calib/synthetic_8cam.json` were generated by this repository (`code/scripts/make_synthetic_sample.py`; no third-party data), Apache-2.0. No dataset sample is shipped; nuScenes and the METEOR demo scenes never ship, and a PandaSet sample (CC BY 4.0) is prepared but not yet included.
+- Demo media (`media/`; per-file sources, frames and changes: [`media/ATTRIBUTION.md`](media/ATTRIBUTION.md)):
+  - PandaSet renders (`media/meteor_ps*`): Contains data from PandaSet (Scale AI and Hesai), https://pandaset.org, licensed under CC BY 4.0 and the PandaSet Dataset Terms. Changes: camera JPEGs resized to 768x432 (and a centre crop for the virtual narrow camera), re-encoded, calibration converted to the METEOR layout; downscaled, heads and licence plates of annotated people and vehicles blurred; model outputs drawn. Scale AI and Hesai do not endorse this work. Cite: P. Xiao et al., *PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving*, ITSC 2021.
+  - nuScenes renders (`media/*_NC.*`), **non-commercial, CC BY-NC-SA 4.0**: Rendered from the nuScenes dataset, © Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of Use (https://www.nuscenes.org/terms-of-use). Non-commercial use only; adaptations under the same license. Motional does not endorse this work. Cite: H. Caesar et al., *nuScenes: A Multimodal Dataset for Autonomous Driving*, CVPR 2020.
+  - The synthetic-sample render (`media/meteor_synthetic_8cam_tt_vs_cpu.png`): generated by this repository, Apache-2.0.
+
+## Provenance
+
+These are the exact sources the container image was built from:
+
+| component | built from |
+| --- | --- |
+| tt-metal | [`44d66500520fda9f2c7060c0f6b41ec48f7ab37e`](https://github.com/tenstorrent/tt-metal/commit/44d66500520fda9f2c7060c0f6b41ec48f7ab37e) + [`patches/tt-metal-eth-dispatch.patch`](patches/tt-metal-eth-dispatch.patch) + [`patches/tt-metal-reshape-rm-sys1419.patch`](patches/tt-metal-reshape-rm-sys1419.patch) (dirty tree: the image includes both patches; the image build asserts both markers) |
+| weights | [`AutowareFoundation/meteor@01a5f6d71df5ecbbb5853ec600825481d57b9c6b`](https://huggingface.co/AutowareFoundation/meteor/tree/01a5f6d71df5ecbbb5853ec600825481d57b9c6b) (tag `v1.0`), files `meteor_v157c3Z.onnx` (sha256 `50397b4d…b1a7`), `meteor_v157.param.yaml`, `LICENSE`, `SHA256SUMS` |
+| pre- / post-processing reference | no Autoware package; METEOR's runtimes, tier4/METEOR [`dc193a81e9959de57b751e269127ae9b67e3f300`](https://github.com/tier4/METEOR/tree/dc193a81e9959de57b751e269127ae9b67e3f300) (`deploy/cpp`, `hf/onnx_smoke_test.py`) |
+| shared port code | `ttaw` 0.23.2 @ common `60b6dd7` (vendored in `code/tt_meteor/ttaw`, `VENDORED.json`) |
+| `code/` digest (image) | `40de720599c6b5b9` (sha256, first 16 hex digits; `built.code_sha256` of `tt_kernel_manifest.json`) |
+| image | `tt-model/meteor-p150:823dc1fbd4c7` (`sha256:823dc1fbd4c7ec836a6e74486495d2801a71d6fa4e18d41011fa873749962afc`) |
+| base images | build stage `ghcr.io/tenstorrent/tt-metal/tt-metalium/ubuntu-22.04-dev-amd64:latest` @ `sha256:df9d279c7f85c17c6fad982d196802682d669cca1b7ced9cbaad8181339cd5fc`; runtime stage `docker.io/library/ubuntu:22.04` @ `sha256:5ec03bb3441e8b0bf3b4f9cd4629a1ae763010dc3035bb8da3ae6cf026486401` (tt-model's `FROM` tags float; these are the digests this build resolved, see [`build_info.json`](build_info.json)) |
+| built | 2026-10-10T17:04:30+00:00 by tt-model 0.1.0 |

SERVING.md

@@ -0,0 +1,230 @@
+# Serving METEOR (TIER IV, AutowareFoundation/meteor) on Blackhole with tt-model-manager
+
+This repo is the authoring source of the **tt-model container package** `changh95/meteor-p150`
+(`kind: tt-dit-server`, schema 5.1): `tt-model.yaml` is the manifest, `code/` the port, and
+`code/tt_meteor/server/app.py` the ASGI app uvicorn runs. Weights are a pinned pointer, never baked into the image.
+The device open, the decoders, the Python-API contract and the HTTP contract come from `code/tt_meteor/ttaw/`, the
+shared package of the Autoware ports vendored into this repo (`ttaw/VENDORED.json` records its version and file
+hashes; it is not edited here).
+
+| item | value |
+|---|---|
+| tt-metal tree | `/home/ubuntu/experiments/tt-models/tt-metal` (main `44d66500520`, `v0.80.0-dev20261006-78-g44d66500520`, + `patches/tt-metal-eth-dispatch.patch` + `patches/tt-metal-reshape-rm-sys1419.patch`; torch 2.11.0+cpu) |
+| weights | `AutowareFoundation/meteor` @ `01a5f6d71df5ecbbb5853ec600825481d57b9c6b` (tag `v1.0`; files `meteor_v157c3Z.onnx, meteor_v157.param.yaml, LICENSE, SHA256SUMS`; Apache-2.0; public, ungated) |
+| pre- / post-processing reference | **no Autoware package exists for METEOR**: METEOR's own runtimes define them, [tier4/METEOR](https://github.com/tier4/METEOR) @ `dc193a81e9959de57b751e269127ae9b67e3f300` (`deploy/cpp`: `realtime_main.cpp`, `decode.cpp`, `render.cpp`; `hf/onnx_smoke_test.py`) |
+| app | `tt_meteor.server.app:app` (uvicorn; the ASGI lifespan does weights -> device -> graph -> trace capture) |
+| device recipe | `ttnn.open_device(device_id, dispatch_core_config=DispatchCoreConfig(ETH), l1_small_size=32768, trace_region_size=256 MiB, num_command_queues=1)` (`DEVICE_DEFAULTS` in `code/tt_meteor/device.py`) |
+| hardware | one Blackhole p150 (`hardware: p150`, `mesh_device: P150`, `TT_MESH_SHAPE=1x1`), 12×10 compute grid |
+
+## 1. Run on the HOST (hardware validation, no Docker)
+
+The tt-metal `python_env` has ttnn and torch; put the HTTP stack in a side directory instead of installing into it.
+On the shared workspace box every command that opens the chip goes through the device lock (`bin/devrun`).
+
+```bash
+ROOT=/home/ubuntu/experiments/tt-models
+source $ROOT/bin/tt-env.sh                     # TT_METAL_HOME, PYTHONPATH, tt-metal python_env
+mkdir -p /tmp/meteor-http && uv pip install --python $TT_METAL_HOME/python_env/bin/python \
+    --target /tmp/meteor-http fastapi uvicorn onnx
+cd $ROOT/bundles/meteor-p150
+export PYTHONPATH=$PWD/code:$TT_METAL_HOME:$TT_METAL_HOME/ttnn:/tmp/meteor-http
+export HF_MODEL=AutowareFoundation/meteor TT_MODEL_WEIGHTS_REVISION=01a5f6d71df5ecbbb5853ec600825481d57b9c6b TT_MESH_SHAPE=1x1
+# (on the shared workspace box every METEOR device job also needs METEOR_DEVICE_OK=1 from the orchestrator)
+
+# host tests (no device; the device tests are skipped; ttnn is the fake of common/tests/host: a real ttnn host
+# tensor opens the chip, so only devrun jobs may build one)
+PYTHONPATH=$ROOT/research/packaging/scripts:$PYTHONPATH TT_VISIBLE_DEVICES=none \
+    python -m pytest -q -p no:cacheprovider -p fake_ttnn_plugin code/tt_meteor/tests
+# device tests and the server, under the lock
+$ROOT/bin/devrun -t 2400 -- env TTAW_GATES_READONLY=1 python -m pytest -q -s code/tt_meteor/tests/test_pcc_device.py \
+    code/tt_meteor/tests/test_e2e_device.py code/tt_meteor/tests/test_variants_device.py
+$ROOT/bin/devrun -t 1800 -- bash -c 'python -m uvicorn --host 127.0.0.1 --port 20000 --lifespan on tt_meteor.server.app:app & \
+    S=$!; python code/tt_meteor/server/smoke_test.py --url http://127.0.0.1:20000 --wait 1200; R=$?; kill -TERM $S; wait $S; exit $R'
+```
+
+Boot log landmarks (they drive `tt-model serve`'s progress view, `boot_progress.py` TT_DIT_PHASES):
+`Loading weights` -> `Opening device` -> `Warming up: capturing trace ...` -> `Warmup complete` -> uvicorn
+`Application startup complete`. The warm-up runs one eager frame of the whole graph and captures it as one trace:
+about 45-50 s with a warm JIT cache (most of it the eager frame of 4,678 programs); a cold JIT cache compiles kernels
+for minutes (the first container boot on an empty JIT cache was ready after 799 s, 2026-10-10). Startup
+failures raise and uvicorn exits non-zero (no CPU fallback). SIGTERM: the lifespan releases the traces and closes the
+chip (within `tt-model stop`'s 120 s budget).
+
+Offline override (no Hub): `METEOR_WEIGHTS_DIR=<dir holding meteor_v157c3Z.onnx, meteor_v157.param.yaml, LICENSE, SHA256SUMS>`.
+
+## 2. Package, serve, push (Docker)
+
+Docker 29 on this box is **rootful** (`/var/lib/docker`), and the user's session may not carry the `docker` group:
+wrap every docker-using command in `sg docker -c "..."` (no `docker-env.sh` is needed here; the reference bundles'
+`docker-env.sh` was the authors' rootless-Docker box). Run every `tt-model` command **from this directory**
+(`source.tt_metal` and `extra_code[].root: code` are CWD-relative) and point `--out` outside it.
+
+```bash
+ROOT=/home/ubuntu/experiments/tt-models; cd $ROOT/bundles/meteor-p150
+TTM=$HOME/.local/share/uv/tools/tt-model/bin/python
+
+# offline validation (seconds; no docker, no device) -- must print VALID
+$TTM -c "from tt_kernel.container_manifest import load_container_manifest as L; m=L('tt-model.yaml', check_sources=True); p=m.resolve_profile(); print('VALID', m.name, m.kind, p.hardware, p.mesh_device, m.weights_ref)"
+
+# build (one at a time on the box: the absolute lock file): ~35 min cold, ~12 min after a code/ change (the C++
+# build re-runs, ccache-warm, whenever code/ changes), ~1 min for a manifest- or lock-only change
+flock $ROOT/.package.lock sg docker -c "tt-model package --container tt-model.yaml --out $ROOT/build"   # log: ~/.cache/tt-model/build/meteor-p150.log
+# provenance: the base-image digests this build resolved (tt-model's FROM tags float) -> build_info.json (published)
+sg docker -c "python3 $ROOT/research/packaging/scripts/record_build_info.py --staged $ROOT/build/meteor-p150 --bundle ."
+
+# serve + smoke + stop, all inside ONE device-lock window, once per serve profile (`tt-model profiles
+# $ROOT/build/meteor-p150/tt_kernel_manifest.json` lists them; serve returns once READY and leaves the container
+# running; the script's trap saves the container log and always runs `tt-model stop`, also when devrun's timeout
+# fires: -k 150 > the 120 s grace). No token reaches the public weights download. Evidence: logs/smoke/ (--log-dir).
+$ROOT/bin/devrun -t 3600 -k 150 -- env -u HF_TOKEN -u HUGGING_FACE_HUB_TOKEN \
+    sg docker -c "bash code/scripts/container_smoke.sh $ROOT/build/meteor-p150 20000 [--profile NAME]"
+
+# publish: overlay the hand-written docs, the pip project and the build record onto the staged dir, then push
+# (HF_TOKEN only from the environment)
+rsync -a README.md SERVING.md OPT_BASELINE.md OPT_REPORT.md VERIFICATION_*.md tt-model.yaml pyproject.toml build_info.json \
+    examples media patches $ROOT/build/meteor-p150/
+cp PYTHON.md $ROOT/build/meteor-p150/PYTHON.md     # the top-level copy (its links are relative to the repo root)
+tt-model push $ROOT/build/meteor-p150 --publish
+```
+
+The smoke test fails unless `/info` reports ETH dispatch, the 12x10 grid and what the staged package pins for the
+profile (`METEOR_DISPATCH`, `METEOR_NUM_CQS`, `METEOR_VARIANT`, the weights revision), and it compares the
+served output of the shipped sample (`code/tt_meteor/samples/synthetic_8cam.json`) with its stored CPU-reference
+output (`synthetic_8cam.reference.json`: 3D boxes within 0.5 m and 0.05 of score, recall and precision >= 0.95; lane
+map agreement >= 0.99; selected ego path ADE <= 0.3 m, FDE <= 0.6 m) and checks the expected boxes per label.
+
+For the first release of this bundle the container check also runs the rig-switch stress (the ETH-dispatch hang
+history of PORT_LOG.md section 12): `code/scripts/stress_frames.py --mode rigs --rigs shipped` for >= 200 frames
+inside the image, with `tt-model serve --print`'s device, cache and env arguments (no port) plus
+`METEOR_WEIGHTS_DIR=/hf/hub/models--AutowareFoundation--meteor/snapshots/<revision>` (the script reads the ONNX
+directly, without the server's HF resolve; `serve` has pre-fetched the snapshot):
+`python -u /opt/tt-metal/scripts/stress_frames.py --mode rigs --rigs shipped --frames 200 --jsonl <mounted dir>/stress.jsonl`.
+
+`serve` publishes the first free port from 20000 and exports into the container: `HF_MODEL=AutowareFoundation/meteor`,
+`MESH_DEVICE=P150`, `TT_MESH_SHAPE=1x1`, `TT_MODEL_WEIGHTS_REVISION=01a5f6d71df5ecbbb5853ec600825481d57b9c6b`, then `serve.env`. The HF cache is
+mounted at `/hf` (rw), the JIT cache at `/cache` (host `~/.cache/tt-model/meteor-p150/cache`), and the container
+sees only its own chip (`--device /dev/tenstorrent/<n>`). tt-cli users: `tt serve changh95/meteor-p150` /
+`tt model stop changh95/meteor-p150` (point tt at this tt-model with
+`tt config set tools.override.tt-model ~/.local/bin/tt-model` or `TT_TOOL_BIN_TT_MODEL`).
+
+What `push` does to the repo: `code/` and `image/` on the Hub become exactly the staged trees (`extra_code.paths` +
+`models/common/lightweightmodule.py`); every top-level file of the staged dir is uploaded (the overlaid README
+replaces the generated card; `pyproject.toml` and `build_info.json` arrive the same way, never through `code/`);
+top-level files already on the Hub and not in the staged dir are kept.
+
+## 3. The request / response contract
+
+| route | purpose |
+|---|---|
+| `GET /health` | `{"status": "ok" \| "starting" \| "error", "model", "device": {"dispatch", "grid", "cores", "device_id"}, "error"}`, always 200; `ok` only after warm-up |
+| `GET /v1/health` | same as `/health` (tt-model's hint for non-chat packages points here) |
+| `GET /info` | model, task, io, Autoware package + commit, weights {repo, tag, revision, path, license}, device (dispatch, grid), input limits (point formats, fields, camera order, calibration presets), labels, variant, warm variants, runtime params + defaults, compile params, warm-up and boot times |
+| `GET /v1/models` | `{"object": "list", "data": [{"id": "AutowareFoundation/meteor", "object": "model", "owned_by": "changh95"}]}` -- a stub so OpenAI-shaped probes do not 404; NOT a chat API |
+| `POST /predict` | one frame -> the model output (below) |
+
+### 3.1 Request (`application/json`; unknown fields are a 422)
+
+| field | type | meaning |
+|---|---|---|
+| `images` | list | **all eight cameras** `[{"camera": "CAM_FRONT_WIDE", "data": <base64 PNG / JPEG / .npy>, "intrinsics", "T_ref_from_camera"}, ...]`, any order (reordered to `CAM_FRONT_WIDE, CAM_FRONT_LEFT, CAM_FRONT_RIGHT, CAM_BACK_WIDE, CAM_BACK_LEFT, CAM_BACK_RIGHT, CAM_FRONT_NARROW, CAM_BACK_NARROW`); RGB, raw and unrectified, at least 768×432 (larger frames are resized with OpenCV INTER_AREA semantics and K is scaled per axis). An absent narrow camera is sent as an **all-zero image**: the host gives it a zero input and its donor's K and pose (`CAM_FRONT_WIDE` / `CAM_BACK_WIDE`), the trained 7-camera configuration |
+| `calibration` | object | `{"frame_id", "cameras": {name: {"intrinsics", "T_ref_from_camera", "image_size"}}}` (K of the image as sent; `T_ref_from_camera` = camera optical frame -> base_link: x forward, y left, z up, origin on the road) or `{"preset": "<name>"}` (presets in `/info`: `synthetic_8cam`, the rig of the shipped sample). Inline image calibration wins |
+| `ego_speed` | number | **required**: the ego speed in m/s (the graph's `v0`) |
+| `stream` | object | optional: `{"id": "default", "reset": false, "timestamp_s", "T_world_from_ego"}`: METEOR's host temporal post-processing per stream id (the BEV seg fusion needs `T_world_from_ego`; yaw smoothing; plan-mode hysteresis); a new id or `reset` starts fresh. Without `stream`, every request uses the stream `default` |
+| `params` | object | per-request knobs (host post-processing only; METEOR's C++ renderer defaults): 3D `det3d_threshold` (0.15), `det3d_topk` (64), `vehicle_threshold` (0.35), `vru_threshold` (0.15), `bev_nms_iou` (0.3), `bev_nms_containment` (0.6), `stationary_logit_threshold` (0.0); 2D `det2d_threshold` (0.30), `det2d_topk` (48), `det2d_hide` ("7": road paint); unknown obstacles `unk2d` (true), `unk2d_threshold` (= det2d), `ground_z` (0.0); plan `mode_hysteresis` (0.35), `straight_margin` (1.0); `seg_fuse` (true), `thin_road_edge` (true), `yaw_smoothing` (true); `heads` (false: with `output_format: "npz"`, adds the dense heads) |
+| `output_format` | str | `json` (default) or `npz` (adds lossless base64 NPZ arrays) |
+
+`points`, `sweeps` and `inputs` of the common envelope are not used by this model (a request with `points` is a 400).
+`server/client.py` builds a request: `--sample <manifest>` (the shipped sample: images, preset, ego speed, stream),
+or `--image NAME=PATH` (×8) + `--calib <file>` / `--calib-preset <name>` + `--ego-speed <m/s>` [+ `--stream-id`,
+`--param k=v`].
+
+Transforms (`T_<to>_from_<from>`): a 4×4 (or 3×4) row-major matrix, `{"translation": [x, y, z], "rotation_wxyz": [w, x, y, z]}`
+(or `rotation_xyzw`), or `{"x", "y", "z", "roll", "pitch", "yaw"}` (Autoware `sensor_kit_calibration.yaml`, tf2 RPY).
+Intrinsics: 3×3 K, 3×4 P, or `{"fx", "fy", "cx", "cy"}`. Base64 may be standard or URL-safe, wrapped, with or without a
+`data:` prefix.
+
+### 3.2 Response (200)
+
+```json
+{"model": "meteor-p150", "frame_id": "base_link", "num_detections": 8,
+ "detections": [
+   {"label": "VEHICLE", "label_id": 0, "score": 0.9932, "center": [-12.86, 0.095], "size": [4.399, 1.742], "yaw": -0.0687, "stationary": false, "future_mode": 0, "future": [[-11.92, -0.09], [-10.74, -0.15], ...]},
+   {"label": "VEHICLE", "label_id": 0, "score": 0.9872, "center": [-7.176, -3.215], "size": [3.965, 1.731], "yaw": 0.0183, "stationary": false, ...},
+   {"label": "VEHICLE", "label_id": 0, "score": 0.9624, "center": [10.84, 0.019], "size": [4.334, 1.761], "yaw": -0.0189, "stationary": false, ...},
+   ...],
+ "trajectory": [[0.74, 0.91], [0.85, 1.54], [0.41, 2.34], [0.31, 3.08], [0.62, 3.82], [0.3, 4.45]], "columns": ["x", "y"],
+ "plan": {"mode": 0, "mode_probs": [0.233, 0.296, 0.471], "mode_logits": [...], "paths": [...], "steer": 0.0026, "accel": 0.021, "brake_prob": 0.131, "dt": 0.5},
+ "detections_2d": {"CAM_FRONT_WIDE": [{"label": "car", "label_id": 1, "score": 0.5239, "box_xyxy": [412.76, 211.02, 438.5, 230.29]}, ...], ...},
+ "unknown_obstacles": [],
+ "traffic_light": {"state": "red", "state_id": 3, "probs": [0.363, 0.140, 0.031, 0.465]},
+ "lane": {"format": "png", "key": "lane", "dtype": "uint8", "shape": [800, 500], "data": "iVBORw0KGgo..."},
+ "lane_classes": ["bg", "road", "sidewalk", "crosswalk", "laneline", "stopline", "road_edge", "marking", "parking"],
+ "stationary_head_healthy": true,
+ "meta": {"present": [true, true, true, true, true, true, true, true], "v0": 8.0, "stream_id": "synthetic_8cam"},
+ "timing_ms": {"decode": 83.95, "preprocess": 10.69, "device": 722.42, "postprocess": 86.08, "model_call": 820.65, "total": 908.67}}
+```
+
+(The shipped synthetic sample, a request after the warm-up, host server, 2026-10-10; trimmed.)
+
+| key | content |
+|---|---|
+| `detections` | 3D BEV boxes sorted by score: `label` (VEHICLE / VRU), `label_id`, `score`, `center` [x, y] and `size` [length, width] in metres (base_link: x forward, y left; METEOR predicts no z, height or velocity), `yaw` (rad, CCW from +x), `stationary`, `future` (6 × [x, y] at 0.5-3 s, the agent's best mode; METEOR's renderer draws vehicle futures only), `future_mode` |
+| `trajectory`, `columns` | the selected ego path, 6 × [x, y] at 0.5 s (straight preference and per-stream mode hysteresis, METEOR's `render.cpp`) |
+| `plan` | `mode`, `mode_probs` (softmax of the hysteresis-adjusted logits), `mode_logits`, `paths` (3 × 6 × [x, y]), `steer` (rad), `accel` (m/s²), `brake_prob`, `dt` |
+| `detections_2d` | per camera: `label` (10 classes; road paint hidden by default), `label_id`, `score`, `box_xyxy` (pixels at 768×432) |
+| `unknown_obstacles` | 2D "obstacle" boxes placed on the ground plane (`unk2d`) |
+| `traffic_light` | the ego-relevant light: `state` (none / green / yellow / red), `state_id`, `probs` |
+| `lane`, `lane_classes` | the BEV lane map as PNG (uint8 [800, 500], 0.2 m cells, row 0 = 80 m ahead, column 0 = 50 m to the left), after the seg fusion (with a stream pose) and the road-edge thinning |
+| `stationary_head_healthy`, `meta` | METEOR's stationary-head health check; the present cameras, `v0`, the stream id |
+| `heads` | only with `output_format: "npz"` and `params.heads: true`: seg2d, depth, depth_mean, occupancy, risk, stationary |
+
+The synthetic sample is a test pattern (a red light and a car 11 m ahead): its plan barely moves forward (under 1 m in 3 s,
+drifting 4.5 m to the left), not meaningful driving; the stored CPU reference has the same path.
+
+`timing_ms`: `decode` (base64 + parsing), `preprocess`, `device` (H2D + trace + D2H), `postprocess`, `model_call`,
+`total` (server side, after the body arrived).
+
+### 3.3 Errors
+
+**400** undecodable / malformed input (bad base64, wrong row size, missing calibration, unknown or out-of-range
+`params`, a missing required input), **422** schema violation (unknown field, wrong type), **503** while starting
+(or when the boot failed), **500** `inference failed: <ExceptionType>: <message>` if the device call raises.
+
+### 3.4 Environment the app reads (lifespan only, never at import)
+
+| var | set by | meaning / default |
+|---|---|---|
+| `HF_MODEL` | launcher (`weights.repo`) | weights repo id; default `AutowareFoundation/meteor` |
+| `TT_MODEL_WEIGHTS_REVISION` | launcher (`weights.revision`) | pinned commit; `TT_WEIGHTS_REVISION` (`serve.env`) is the same for older tt-model |
+| `METEOR_WEIGHTS_DIR` | you (offline) | local weights directory; overrides the Hub |
+| `TT_MESH_SHAPE` | launcher (`runtime.mesh_shape_env`) | `1x1`; anything else -> RuntimeError at startup |
+| `TT_DEVICE_ID` | you | chip to open, default 0 |
+| `METEOR_DISPATCH` | `serve.env` | `eth` (default) \| `worker` (A/B only) \| `auto` (ETH if the patch is present) |
+| `METEOR_NUM_CQS` | `serve.env` | `1` |
+| `METEOR_VARIANT` | `serve.env` / profile | `default` |
+| `METEOR_WARMUP` | you | JSON list of warm-up variants, `default` or `none` |
+| `METEOR_TRACE_REGION`, `METEOR_L1_SMALL`, `METEOR_WORKER_L1_SIZE` | you | device-open overrides (validated values: `DEVICE_DEFAULTS` in `device.py`) |
+| `METEOR_MAX_BODY_MB` | you | request size guard, default 256 |
+| `TT_METAL_VISIBLE_DEVICES`, `MESH_DEVICE` | `serve.env` / launcher | `0`, `P150` (informational) |
+
+Server-side pipeline: JSON -> `tt_meteor.io` decoders -> `METEOR.__call__` (host pre-processing of METEOR's runtime:
+INTER_AREA resize, K scaling, extrinsics inversion, the absent-camera donor rule; the lift tables of a new
+calibration, cached per rig -> H2D of the eight cameras and `v0` into persistent device inputs -> `execute_trace` of
+the `frame` trace -> segmented D2H -> host post-processing ported from `deploy/cpp`: 3D peaks + rotated BEV NMS,
+stationary flags, agent futures, 2D boxes, unknown obstacles, the plan with hysteresis, seg fusion) under one lock
+-> `Output.to_dict()`.
+
+## 4. Caveats
+
+- Batch 1, one frame per request; concurrent clients queue on the lock.
+- Fixed shapes of the released graph: eight 768×432 camera slots in METEOR's order, a 400×250 lift grid at 0.4 m,
+  an 800×500 BEV at 0.2 m; at most 3 cameras per lift cell on the rigs validated. A new calibration costs about
+  0.2 s once (the lift tables), then it is cached per rig.
+- The request bodies are large (eight images as base64): about 1-4 MB for PNG / JPEG at 768×432.
+- Warm-up captures every trace variant inside the lifespan, so READY means warm; the first cold boot pays the ttnn JIT.
+- Weights are pinned by sha in `weights.revision`, exported by the launcher and repeated in `serve.env`;
+  `snapshot_download(..., revision=<sha>)` is a cache hit after `serve`'s pre-download and falls back to
+  `local_files_only=True` if the Hub is unreachable.
+- The runtime image has no host C/C++ compiler: device kernels JIT-compile (sfpi ships in the image), host-side C
+  helpers must have a numpy fallback.
+- `tt-model curl` and the ready card's `/v1/models` hint are OpenAI-shaped and are not this API; use the routes above.

VERIFICATION_2026-10-10.md

@@ -0,0 +1,182 @@
+# meteor-p150: verification of the first release, 2026-10-10
+
+## Verdict
+
+**PASS.** This is the light verification pass of the first publish (`research/PLAN.md` §6.3): the device suite
+re-run with every frozen gate (read-only), the baseline numbers re-checked, the published configuration exercised end
+to end (quickstart, host server + smoke test, a golden-free ETH stress), and every device-open path audited for ETH
+dispatch. It was made by the agent that wrote the release docs, on the release tree (bundle commit `cc35027` plus the
+release changes that this file is committed with). Against the baseline commit `e86c13a` (`OPT_BASELINE.md`,
+committed as `cc35027`):
+
+- **Accuracy:** the device suite passes (**16 passed** in one devrun job, 93 s). Every stored value
+  (`pcc_<frame>.json` of the 4 frames, `e2e_device.json`, `variants_device.json`) is **identical** to the baseline
+  code's three ETH suite runs (`logs/meteor/hangfix/values_final1/`), timings aside; the only new entry is the
+  shipped synthetic sample's check (8 / 8 boxes, max |score difference| 0.0020, lane agreement 0.9987, path ADE
+  0.018 m), gated with the existing thresholds. 38 public frames through the served graph agree with the CPU goldens
+  (below).
+- **Speed:** the device rows reproduce `OPT_BASELINE.md` to 0.15 ms (replay 556.92 vs 556.85 ms, back to back 556.89
+  vs 556.75 ms); the end-to-end PandaSet call is 1.9 % slower (935.9 vs 918.2 ms, p50) on a busier shared host.
+- **Audit:** ETH dispatch, 1 CQ and the 12×10 grid on every device-open path; no fallback in any run; both tt-metal
+  patches present (`eth_patch: true`; the reshape-patch marker is asserted by the image build).
+- **No numerics change of the device graph** since the verified port: `e86c13a..` touches no file under `tt/`, no
+  precision knob and no vendored ttaw file (`vendor.py --check`: 0 differences, ttaw 0.23.2 @ common `60b6dd7`). The
+  release changes are host-side and documented (OPT_REPORT.md "release" row). No gate threshold moved; the gate files
+  are byte-identical.
+- **No hang, no `.device.FAULT`, tt-triage never armed**: all 6 device jobs exited with rc 0 (every METEOR device job
+  ran with the orchestrator's `METEOR_DEVICE_OK=1`).
+
+The adversarial verification of the port is `VERIFY_PORT.md` (round 1, 2026-10-09: FAIL on two medium findings, V1
+and V2, plus 7 low ones). Both mediums were fixed in PORT_LOG.md section 11 (E22, E23); "Review findings" below
+lists where each finding stands. No second full verification round was run before this release.
+
+## Setup
+
+| item | value |
+|---|---|
+| Hardware | one Blackhole p150b, ETH dispatch, 1 CQ, 12×10 compute grid (120 cores), as printed by every run (`dispatch eth`, `fallback null`, `eth_patch true`); AICLK 1350 MHz (median of 163 samples during the bench, minimum 1343), board power median 69 W |
+| tt-metal | `44d66500520` with `patches/tt-metal-eth-dispatch.patch` (sha256 `08d0ddf6…`) and `patches/tt-metal-reshape-rm-sys1419.patch` (sha256 `74878683…`), both byte-identical to the workspace copies the tree was built with |
+| Weights | `AutowareFoundation/meteor` @ `01a5f6d71df5ecbbb5853ec600825481d57b9c6b` (tag `v1.0`) from the HF cache at the pinned revision (`HF_HUB_OFFLINE=1`, no token) |
+| Verified code | `cc35027` + the release changes (ttaw 0.23.2 @ common `60b6dd7`, `vendor.py --check`: 0 differences) |
+| Baseline | `e86c13a` (`OPT_BASELINE.md`) |
+| Workloads | the shipped sample `samples/synthetic_8cam` (8 PNGs, 768×432, all cameras present, preset `synthetic_8cam`); the staged PandaSet 019 frame 40 sample (not shipped); the 4 golden frames of the gates (research goldens, local); PandaSet 019 / 090 and nuScenes scene-0103 frames of `research/meteor/public_data` (38 frames, local, never shipped) |
+| Bench | `code/scripts/bench.py --inputs shipped,sample --iters 30 --b2b 20` (p50 / p99 of 30 per loop), one run |
+| Host | AMD EPYC-Rome VM, 8 vCPUs, `OMP_NUM_THREADS=4`, shared with other agents' jobs (load average 6.0 → 8.7 during the bench) |
+| Logs | `logs/meteor/release/` of the workspace (`job_{media,suite,quickstart,serve,bench,stress}.{sh,log}`, `run_all.rcs`); devrun windows 16:13:59, 16:30:01, 16:35:42, 16:38:44, 16:45:31, 16:50:22 UTC (`logs/devrun/history.log`), all rc 0 |
+
+## Measured baseline vs re-check
+
+| measurement | baseline `e86c13a` (`OPT_BASELINE.md`) | re-check (release tree) | change |
+|---|---:|---:|---:|
+| one blocking replay of `frame`, p50, PandaSet 019 | 556.85 ms (p99 559.34) | 556.92 ms (p99 557.5) | +0.07 ms |
+| back-to-back replays, per frame | 556.75 ms | 556.89 ms | +0.14 ms |
+| H2D (8 cameras + v0) | 66.5 ms | 66.6 ms | +0.1 ms |
+| D2H (75 MB, 5 segments) | 17.2 ms (17-48 bimodal) | 46.4 ms | in the documented 17-48 ms range (host side) |
+| host preprocess / camera transpose | 11.2 / 26.3 ms | 13.8 / 27.0 ms | host load |
+| readback conversion (unpack + outputs) | 15.9 + 49.4 ms | 14.9 + 43.3 ms | host |
+| host postprocess | 129.9 ms | 133.8 ms | host load |
+| e2e `model()` PandaSet 019, p50 (p99) | 918.2 (1,071.8) ms | 935.9 (999.6) ms | +1.9 % |
+| `from_pretrained` load, warm kernel cache | 49.7 s | 45.3 s (weights 0.3, build 2.3, warm-up 38.6 s) | −4.4 s |
+| PandaSet 019 frame vs its stored CPU reference (bench check) | 8 / 8, recall / precision 1.0 / 1.0, max \|Δscore\| 0.0081, lane 0.99770, path 0.108 m | identical | 0 |
+
+The device rows reproduce; the host rows move with the load of the shared host.
+
+Not in the baseline, measured for the card on the release tree:
+
+| measurement | value |
+|---|---:|
+| shipped sample through `model()` (bench, 30 iterations) | e2e **871.6 ms** (942.1); device 755.6, postprocess 95.2; replay 556.80 ms, back to back 556.65 ms |
+| shipped sample vs its stored CPU reference (`test_e2e_device.py::test_shipped_sample_vs_stored_reference`, the bench check and the server smoke) | 8 / 8, recall 1.000, precision 1.000, max \|score difference\| 0.0020, centre error max 0.002 m, lane agreement 0.9987, path ADE / FDE 0.0177 / 0.0298 m, same plan mode |
+| served `/predict`, host uvicorn, the smoke request (first request: includes the rig change of the new calibration) | `timing_ms.total` 1,226.0 ms (device 991.7), client round trip 1,250.7 ms |
+| served `/predict`, three more requests of the same body (1.8 MB) | `timing_ms` decode 81.3-84.0 · preprocess 10.5-10.8 · device 719.5-722.4 · postprocess 86.1-87.1 · **total 904.0-908.7 ms**; client round trip 0.92-0.93 s |
+| server boot to ready (warm kernel cache) | 44.6 s (`ready in 44.6 s`; warm-up 37.8 s) |
+| served body vs the stored CPU reference (`server/smoke_test.py`, the container-smoke rule, `--expect VEHICLE:6,VRU:2`) | **PASS**: 8 / 8, recall 1.000, precision 1.000, dscore 0.0020; trajectory ADE 0.0177, FDE 0.0298; lane 0.9987; `/info` dispatch eth, grid 12x10, 1 CQ |
+| `examples/quickstart.py` (from the repo root, default input) | rc 0: 8 boxes (6 VEHICLE, 2 VRU, scores 0.887-0.993), plan mode 0, traffic light red, 10 2D boxes; `quickstart.json` and `quickstart_bev.png` written |
+| golden-free ETH stress, `stress_frames.py --mode rigs --rigs shipped --frames 200` (4 synthetic rigs, a rig change + table write + upload + replay + read every frame) | **200 frames clean**, 0 mismatches (4 stable per-rig digests); table write 164.6 ms, run 708.0 ms (max 784.8) |
+| 38 public frames through the served graph (`code/scripts/run_public_frames.py`; every feed's sha256 equal to the CPU golden's) | 707-1,005 ms per frame (the first frame of each rig includes the table write) |
+
+## Accuracy gate results
+
+`test_pcc_device.py` + `test_e2e_device.py` + `test_variants_device.py` in one pytest, one devrun job,
+`TTAW_GATES_READONLY=1`: **16 passed** (93 s). The gate files are unchanged (`git diff --stat` empty for
+`*.gates.json`); the new shipped-sample test reuses the existing `det3d.*`, `ego.path_dev_m` and `lane.agreement`
+gates and froze nothing.
+
+| gate | threshold | value (identical to the baseline's ETH runs) |
+|---|---:|---:|
+| teacher-forced stage PCC, 26 gates (PandaSet 019 f40) | ≥ 0.99 | min 0.9999890 (`tf.risk`); `depth_mean` 0.99948 (reported) |
+| teacher-forced lane / seg2d / depth argmax agreement | ≥ 0.99 | 0.9984 / 0.9965 / 0.9936 |
+| chained frame, min output PCC: PandaSet 019 / 090, nuScenes 0103, valday #40 | ≥ 0.99 | 0.99989 / 0.99990 / 0.99991 / 0.99982 |
+| chained lane / seg2d / depth agreement, worst frame | ≥ 0.99 | 0.9963 / 0.9965 / 0.9925 |
+| replay == eager | bit-identical | yes (19 outputs) |
+| det3d strict recall / precision: PandaSet sample, PandaSet 090, nuScenes, valday #40 | ≥ 0.95 / ≥ 0.95 | 1.0/1.0, 1.0/1.0, 1.0/1.0, 0.971/0.971 (34 / 35) |
+| det3d near-tie extras (recall_nt ≥ recall, ambiguous_frac) | ≥ 0.95, ≤ 0.10 | pass on every frame |
+| selected ego path deviation, worst frame | ≤ 0.3 m | 0.170 m (PandaSet 090) |
+| lane agreement after the host decode, worst frame | ≥ 0.99 | 0.9966 |
+| PandaSet sample `/predict` vs its stored reference body | ≥ 0.95 / ≥ 0.95 | 8 / 8, max \|Δscore\| 0.0081 |
+| **shipped synthetic sample** `/predict` vs its stored reference body (new) | the same gates | 8 / 8, max \|Δscore\| 0.0020, path 0.030 m, lane 0.9987 |
+| API == `/predict` (shipped sample) | identical bodies | yes |
+| `input_norm=imagenet` + `depth_mean_bins=linear` variant, float outputs min PCC | ≥ 0.99 | 0.99988 |
+
+**Public frames** (not gated by the device suite; `research/meteor/public_data/scripts/compare_tt.py` with the
+PLAN thresholds, the TT outputs written by `run_public_frames.py`; `logs/meteor/release/public_tt/compare/`):
+
+| sequence | frames | lane / seg2d / depth agreement (min) | dense-head PCC min (hm / reg / stationary / risk) | all 19 outputs, PCC min (one frame) | path dev max | 3D boxes: CPU matched / TT precision | same mode / TL |
+|---|---:|---|---|---:|---:|---|---|
+| PandaSet 019 | 13 | 0.99761 / 0.99637 / 0.99330 | 0.99995 / 0.99995 / 0.99999 / 0.99962 | 0.99948 (`depth_mean`) | 0.069 m | 99 / 103, 1.0 | 13 / 13 |
+| PandaSet 090 | 13 | 0.99708 / 0.99684 / 0.99483 | 0.99996 / 0.99996 / 0.99999 / 0.99982 | 0.99983 | 0.110 m | 69 / 71, 1.0 | 13 / 13 |
+| nuScenes scene-0103 | 12 | 0.99764 / 0.99624 / 0.99253 | 0.99997 / 0.99993 / 0.99998 / 0.99962 | 0.99902 | 0.081 m | 240 / 244, 1.0 | 12 / 12 |
+
+Every frame passes the argmax, PCC and path gates. 8 of the 38 frames fall below the 95 % "golden boxes matched"
+check of `compare_tt.py` because they have few boxes (5-14) and miss one or two: of the 10 missed CPU boxes, 9 score
+within 0.004 of their class threshold on the CPU (0.3507-0.3538 vs 0.35; 0.1505-0.1513 vs 0.15), so a tiny score
+difference drops them; the 10th is a pedestrian in a nuScenes crowd whose TT peak lies 2.1 m away. The TT never
+produces a box the CPU does not (precision 1.0 on every frame). This is the documented near-threshold behaviour of a
+bf16 / fp32 port (README Caveats), not a gate of the port: the frozen det3d gates run on the four golden frames above.
+
+## Disclosed numerics changes
+
+None on the device: the release does not touch `tt/`, the precision knobs or the vendored ttaw. The precision policy
+is the port's (PLAN §9.3; PORT_LOG E20 / E22): fp32 weights and biases, HiFi4, fp32 dest and packer L1 accumulation;
+the image branch and the 3D detection path with fp32 activations as three bf16-term convs; the BEV trunk in bf16.
+
+Host-side changes of the release (no effect on any gate value, confirmed by the identical stored values):
+- the shipped sample is now `samples/synthetic_8cam*` (generated, Apache-2.0); the PandaSet sample, its preset and
+  its small goldens moved unchanged to the git-ignored `staging_samples_pandaset/` (sha256-checked), where
+  `tests/paths.py` finds them (inside the workspace a missing staging dir fails those tests);
+- `load_sample` resolves a preset from the `calib/` beside the manifest's `samples/` directory first (a staged sample
+  carries its own preset) and returns it inline; `server/smoke_test.py` sends a staged preset inline;
+- `opencv-python-headless>=4.8,<4.12` declared in `pyproject.toml` and `runtime.packages` (+ an `import cv2` verify
+  line): the host post-processing needs OpenCV (PORT_LOG I3) and tt-metal's ttnn install does not bring it;
+- `server/smoke_test.py`: the default sample and `DEFAULT_EXPECT = "VEHICLE:6,VRU:2"` (it was the unfilled template
+  string (the template placeholder `SMOKE_EXPECT` in double braces), which the expectation parser would have read as a required label: the container smoke
+  would have failed);
+- `examples/quickstart.py` called `model(path)` (a path is the point-cloud argument: an `InputError`) and `len(out)`;
+  fixed to `model(**load_sample(path))`, and it now writes the `quickstart_bev.png` it promised;
+- `server/client.py`: `--sample <manifest>` and `--ego-speed` (the vendored client cannot send the required
+  `ego_speed`);
+- `stress_frames.py --rigs shipped[:N]`: rigs from the shipped sample, no golden needed (for the container check).
+
+## Review findings
+
+- No test threshold was loosened, no gate file edited; the device tests compare with goldens written by the fp32
+  CPU reference before any device code existed (VERIFY_PORT 3(e)); the shipped sample's reference was written by the
+  same CPU reference from the PNG files exactly as the API reads them (`make_synthetic_sample.py finalise` asserts the
+  PNGs read back bit-exactly).
+- The shipped sample was optimised against the CPU reference (object pixels only) so that its published boxes sit
+  far from the thresholds (scores 0.889-0.993 vs 0.35 / 0.15; the strongest other heatmap cell 0.179 vehicle /
+  0.072 VRU). This makes the container smoke stable; it does not make the TT output agree better (the optimisation
+  never saw a TT output).
+- Timing loops measure what they claim (`bench.py` synchronises each stage; the b2b row replays 20 frames back to
+  back); every bench run also checks its own output against the stored reference.
+- Nothing hung; the stress jobs ran contained (devrun timeout, no triage, no operation timeout).
+- VERIFY_PORT round 1 findings:
+  - V1 (det3d gate redefined after a failure): **fixed** by E22 (three-term det path): the strict PLAN gate is gated
+    on every frame again, valday #40 0.971 / 0.971; re-confirmed here.
+  - V2 (`input_norm=imagenet` never ran on the card): **fixed** by E23 (`test_variants_device.py`, also covering
+    `depth_mean_bins=linear`, L1); re-confirmed here.
+  - L1 (`IMAGE_PRECISION=bf16` exposed): the knob's documentation states it is below the depth gate (A/B only).
+  - L2 (`depth_mean` max abs error from bin flips) and L3 (centre errors up to 0.4 m on public frames): reported
+    metrics, unchanged.
+  - L4 (fp32 D2H, host transpose): on the optimization backlog (OPT_REPORT items 3 and 7).
+  - L5 (the API falls back to WORKER if the ETH open fails): unchanged by design (shared ttaw); the container smoke
+    fails unless `/info` reports eth / 12x10, and the tests open with `allow_fallback=False`.
+  - L6 (`_tables_key` set before the upload succeeded): fixed in the hang round (PORT_LOG 12.1 A3: the key is set
+    after the last chunk lands). `tt/terms.py` stays unwired (documented). I3 (cv2 in the image): fixed in this
+    release (declared + verify line). I4 (docs pointing at autoware_universe): fixed in this release (README,
+    SERVING, card, pyproject point at tier4/METEOR).
+  - L7: cosmetic.
+
+## p150 ETH-dispatch compliance
+
+| path | dispatch / CQs / grid |
+|---|---|
+| Python API `METEOR.from_pretrained()` | `DEVICE_DEFAULTS` (`device.py`): no dispatch override, so the vendored `ttaw.device` default ETH; 1 CQ; `l1_small_size` 32 KiB, `trace_region_size` 256 MiB. Every run of this pass printed `dispatch eth`, `grid 12x10`, `cores 120`, `fallback null`, `eth_patch true` |
+| HTTP server (`METEOR_DISPATCH` default) | the same defaults; the host server run reported `/info` device `eth`, `12x10`, 1 CQ, `fallback null` |
+| `tt-model.yaml` serve env (every serve profile) | one profile (`default`): `METEOR_DISPATCH: "eth"`, `METEOR_NUM_CQS: "1"`, `METEOR_VARIANT: "default"`; `check_bundle.py` refuses any other dispatch |
+| image `verify:` | the ETH-patch marker in `topology.cpp`, `ttaw.device.eth_dispatch_patch_present()`, the reshape-patch marker (`dual_kernel_min_dram_dest_page_bytes`) in `reshape_rm_program_factory.cpp`, `import cv2`, the shipped sample and preset present |
+| container smoke per serve profile (`/info` asserted: eth, 12x10, pins) | `server/smoke_test.py` hard-codes `EXPECT_DISPATCH, EXPECT_GRID = "eth", "12x10"` and checks the staged package's pins; run at packaging |
+| gate tests (`code/conftest.py`) | `open_device(allow_fallback=False)`: a failing ETH open is an error, never a silent WORKER run |
+| bench / profile / stress / public-frame scripts | `bench.py` and `profile_ops.py`: `--dispatch` default None = the API default (eth); `stress_frames.py`: `--dispatch eth` default with `allow_fallback=False`; `run_public_frames.py`: the API default, reports the device (eth, 12x10); the repro scripts default to eth |
+| no hard-coded grid | `grep` for `CoreGrid`, `CoreRange`, `grid_x`, `core_grid` in `tt/` finds nothing (VERIFY_PORT 3(b)); WORKER 11×10 builds its own program set (OPT_BASELINE "Grid usage") |
+| vendored `ttaw` | 0.23.2 @ common `60b6dd7` (`VENDORED.json`, `source_dirty: false`); `vendor.py --check`: 0 differences; `check_bundle.py` drift check: clean |

build_info.json

@@ -0,0 +1,48 @@
+{
+ "schema": "ttaw-build-info/1",
+ "bundle": "meteor-p150",
+ "recorded_at": "2026-10-10T17:14:45Z",
+ "image": {
+  "tag": "tt-model/meteor-p150:823dc1fbd4c7",
+  "digest": "sha256:823dc1fbd4c7ec836a6e74486495d2801a71d6fa4e18d41011fa873749962afc",
+  "built_at": "2026-10-10T17:04:30+00:00",
+  "code_sha256": "40de720599c6b5b9be0687f81b391539cf3e57dfb5614eab5a7bc721791748fc",
+  "size_bytes": 4265173718,
+  "layers": 23
+ },
+ "base_images": {
+  "build": {
+   "stage": "prep",
+   "ref": "ghcr.io/tenstorrent/tt-metal/tt-metalium/ubuntu-22.04-dev-amd64:latest",
+   "digest": "sha256:df9d279c7f85c17c6fad982d196802682d669cca1b7ced9cbaad8181339cd5fc",
+   "source": "docker build log",
+   "local_tag_matches": false,
+   "local_id": "sha256:3fd1e6013e658c65df1bc6b084543e7174d854c15af97373e493df0c1787c393",
+   "created": "2026-10-07T01:17:29.186036377Z",
+   "note": "the local tag moved after this build; the digest above is the one the image was built from"
+  },
+  "runtime": {
+   "stage": "runtime",
+   "ref": "docker.io/library/ubuntu:22.04",
+   "digest": "sha256:5ec03bb3441e8b0bf3b4f9cd4629a1ae763010dc3035bb8da3ae6cf026486401",
+   "source": "docker build log",
+   "local_tag_matches": null,
+   "note": "not in the local image store (BuildKit pulled it into its cache)"
+  }
+ },
+ "tt_metal": {
+  "sha": "44d66500520fda9f2c7060c0f6b41ec48f7ab37e",
+  "describe": "v0.80.0-dev20261006-78-g44d6650052-dirty",
+  "dirty": true,
+  "scm_version": "0.65.2.dev11169+g44d66500520",
+  "mode": "local",
+  "remote": "https://github.com/tenstorrent/tt-metal.git",
+  "branch": "main",
+  "pushed": true
+ },
+ "tools": {
+  "tt_model": "0.1.0",
+  "docker": "29.8.2"
+ },
+ "notes": "tt-model's FROM tags float (build: tt-metalium dev image :latest, runtime: ubuntu:<version>); these are the digests this image was built from. A moved build base costs a cold C++ build."
+}

code/PYTHON.md

@@ -0,0 +1,165 @@
+# Python API: METEOR (TIER IV, AutowareFoundation/meteor) on Blackhole
+
+Use this API from Python code (a pipeline, a notebook, a ROS 2 node wrapper). You do not need the HTTP server: the
+API and the server share the decoders, the device graph and the post-processing, so the outputs and the speed are
+the same.
+
+## Install
+
+Install the package on top of an environment that already has `ttnn`: a tt-metal `python_env` at `44d66500520`
+with `patches/tt-metal-eth-dispatch.patch` and `patches/tt-metal-reshape-rm-sys1419.patch` applied, or the tt-model
+container. From the root of the model repository (the directory that holds `pyproject.toml`, `README.md` and
+`code/`):
+
+```bash
+pip install -e .                        # the Python API (numpy<2, pillow, pyyaml, onnx, huggingface_hub, safetensors, opencv-python-headless)
+pip install -e ".[server,test]"         # + the HTTP server and the tests
+```
+
+The pip project is the repository's top-level `pyproject.toml`; it installs the package from `code/tt_meteor` (there
+is no `pyproject.toml` inside `code/`, because the container build copies `code/` over the tt-metal tree). `ttnn`
+and `torch` are not declared, so pip never replaces tt-metal's own build; tt-metal's `python_env` already has
+OpenCV 4.8.1, which satisfies the `opencv-python-headless>=4.8,<4.12` requirement.
+
+The package carries `tt_meteor.ttaw`, the shared code of the Autoware ports to Blackhole (device open, trace runner,
+decoders, model base class, HTTP app), vendored at the version recorded in `code/tt_meteor/ttaw/VENDORED.json`
+(0.23.2).
+
+| You want to run | Extras |
+|---|---|
+| the Python API | none |
+| the HTTP server (`tt_meteor.server.app`, see `SERVING.md`) | `server` |
+| host tests (no device; the device tests are skipped): `TT_VISIBLE_DEVICES=none python -m pytest -q code/tt_meteor/tests` | `server,test` |
+| device tests: `python -m pytest -q -s code/tt_meteor/tests/test_pcc_device.py code/tt_meteor/tests/test_e2e_device.py code/tt_meteor/tests/test_variants_device.py` (most of them need the fp32 goldens of the development workspace and skip without them) | `test` |
+
+## Quickstart
+
+```python
+from tt_meteor import METEOR, load_sample
+
+with METEOR.from_pretrained(device_id=0) as model:
+    out = model(**load_sample("code/tt_meteor/samples/synthetic_8cam.json"))   # 8 cameras + calibration + ego speed + stream
+print(out.to_dict()["plan"]["mode"], [d["label"] for d in out.to_dicts()])
+```
+
+`examples/quickstart.py` runs the same snippet and writes the `/predict` JSON and a bird's-eye view
+(`quickstart_bev.png`). The shipped sample is a synthetic test frame generated by this repository (see
+`code/tt_meteor/samples/README.md`); feed your own rig's eight images and calibration for real use.
+
+## `METEOR.from_pretrained(...)`
+
+```python
+METEOR.from_pretrained(
+    model_id=None,          # HF repo or a local directory with the weights files; default AutowareFoundation/meteor
+    *,
+    revision=None,          # default for the default repo: the validated commit 01a5f6d71df (tag v1.0)
+    variant=None,           # load-time variant: "default" (the only one); default $METEOR_VARIANT
+    device_id=None,         # chip to open; default $TT_DEVICE_ID or 0
+    device=None,            # an already-opened ttnn device (tt_meteor.device.open_device); close() does not close it
+    dispatch=None,          # "eth" (p150 target, 12x10 grid) | "worker" (A/B only, 11x10) | "auto"; default $METEOR_DISPATCH or "eth"
+    num_command_queues=None,  # default $METEOR_NUM_CQS or 1 (D14: a second queue gains nothing for synchronous calls)
+    weights_dir=None,       # explicit local weights directory; no Hub access
+    warmup_variants="default",  # trace variants to capture now; see "Warm-up"
+    verbose=False,
+    **compile_params,       # load-time knobs (below), e.g. input_norm="imagenet"
+) -> METEOR
+```
+
+What it does: resolves the weights first, so a Hub problem never claims the chip (`weights_dir` >
+`$METEOR_WEIGHTS_DIR` > a local `model_id` directory > the HF snapshot at the pinned revision, restricted to
+`meteor_v157c3Z.onnx, meteor_v157.param.yaml, LICENSE, SHA256SUMS`, with an offline fallback to the cache), opens the
+chip (ETH dispatch, 12×10, 1 CQ; the other open parameters are `DEVICE_DEFAULTS` in `tt_meteor/device.py`:
+`l1_small_size` 32 KiB, `trace_region_size` 256 MiB, overridable with `METEOR_*`), reads the parameters of the ONNX
+file by consuming node, builds the graph, then compiles and captures the metal trace. If ETH dispatch cannot open
+(tt-metal without the patch), it warns and falls back to WORKER dispatch.
+
+Load-time knobs (`compile_params` in lower case, or the environment variable `METEOR_<NAME>`; read once at load):
+
+| knob | default | meaning |
+|---|---|---|
+| `input_norm` | `onnx` | `onnx` = the released graph's `/255` (ONNX Runtime parity; decision D12); `imagenet` = the trained ImageNet mean / std (upstream export defect, research/meteor SPEC section 10 risk 1); with `imagenet` an absent camera is zeroed after normalising, as in training |
+| `depth_mean_bins` | `log` | `depth_mean` bin centres: log-spaced as exported (parity) or `linear` 1 + 1.25 b m (the trained bins) |
+| `max_streams` | 16 | host temporal states kept (seg fusion, yaw tracks, mode hysteresis); the least recently used is dropped |
+| `image_precision` | `terms3` | image branch with fp32 activations, every conv as three bf16 terms (needed by the depth / seg2d argmax gates); `bf16` is an A/B only: 166 ms faster per frame, but below the depth argmax gate (0.9797 < 0.99) |
+
+## Warm-up
+
+The load runs one eager frame of the whole graph (it compiles every kernel and fills the program cache) and then
+captures it as ONE metal trace, `frame` (4,678 programs). The first call is therefore as fast as the later ones.
+With a warm JIT cache the load takes about 45-50 s (49.7 s measured: weights 0.3 s, graph build 2.6 s, eager frame
+41.0 s, capture 1.2 s). A cold kernel cache compiles for minutes (a new dispatch configuration on the build host:
+ETH-2CQ 142 s, WORKER 11×10 768 s). `warmup_variants="none"` skips the capture (the first call then captures).
+
+## Call: `model(...)`
+
+| Argument | Type | Description |
+|---|---|---|
+| `images` | list or mapping | the cameras `CAM_FRONT_WIDE`, `CAM_FRONT_LEFT`, `CAM_FRONT_RIGHT`, `CAM_BACK_WIDE`, `CAM_BACK_LEFT`, `CAM_BACK_RIGHT`, `CAM_FRONT_NARROW`, `CAM_BACK_NARROW`, any order: `CameraImage`s, dicts `{"camera", "image", "intrinsics", "T_ref_from_camera"}` or `{name: image}`. Raw, unrectified frames of at least 768×432; larger frames are resized with OpenCV INTER_AREA semantics (bit-exact numpy port) and K is scaled per axis. The six wide / corner cameras are required; a missing or all-zero narrow camera is **absent**: a zero image plus its donor's K and pose (`CAM_FRONT_WIDE` / `CAM_BACK_WIDE`), the trained 7-camera configuration |
+| `calibration` | dict | `{"cameras": {name: {"intrinsics", "T_ref_from_camera"}}}` (K of the image as sent; camera optical frame -> base_link: x forward, y left, z up, origin on the road) or `{"preset": name}` (`code/tt_meteor/calib/`); inline calibration of a camera wins |
+| `ego_speed` | float | m/s, the graph's `v0` (required) |
+| `stream` | dict | optional: `{"id", "reset", "timestamp_s", "T_world_from_ego"}` (or `"pose": [x, y, yaw]`): METEOR's host temporal post-processing per stream id (BEV seg fusion needs the pose; yaw smoothing; plan-mode hysteresis). A new id or `reset=True` starts fresh |
+| runtime params | keyword | host post-processing only (`METEOR.RUNTIME_PARAMS`, METEOR's C++ renderer defaults): 3D `det3d_threshold` 0.15, `det3d_topk` 64, `vehicle_threshold` 0.35, `vru_threshold` 0.15, `bev_nms_iou` 0.3, `bev_nms_containment` 0.6, `stationary_logit_threshold` 0.0; 2D `det2d_threshold` 0.30, `det2d_topk` 48, `det2d_hide` "7" (road paint); `unk2d` True, `unk2d_threshold` (= det2d), `ground_z` 0.0; plan `mode_hysteresis` 0.35, `straight_margin` 1.0; `seg_fuse` True, `thin_road_edge` True, `yaw_smoothing` True; `heads` False (adds the dense heads to `to_dict("npz")`) |
+
+`tt_meteor.load_sample(path)` turns a sample manifest (`samples/<name>.json`: image paths relative to the file, a
+preset, the ego speed and a stream) into these keyword arguments.
+
+### Input types
+
+- Images: a path, PNG / JPEG bytes, a `PIL.Image`, a uint8 H×W×3 RGB array, or a float array in [0, 1].
+- Transforms: 4×4 (or 3×4) matrices, `{"translation", "rotation_wxyz"}` (or `rotation_xyzw`), or Autoware
+  `{"x", "y", "z", "roll", "pitch", "yaw"}`. Intrinsics: 3×3 K, 3×4 P or `{"fx", "fy", "cx", "cy"}`.
+
+## Output
+
+`model(...)` returns a `MeteorOutput` (`tt_meteor.Output`); `out.to_dict()` is the `/predict` JSON:
+
+| key | content |
+|---|---|
+| `detections` (`out.to_dicts()`) | 3D BEV boxes sorted by score: `label` (VEHICLE / VRU), `label_id`, `score`, `center` [x, y] and `size` [length, width] in metres (base_link; METEOR predicts no z, height or velocity), `yaw` (rad, CCW from +x), `stationary`, `future` (6 × [x, y] at 0.5 .. 3 s, the agent's best mode), `future_mode` |
+| `trajectory`, `columns` | the selected ego path, 6 × [x, y] at 0.5 s steps |
+| `plan` | `mode`, `mode_probs` (softmax of the hysteresis-adjusted logits), `mode_logits`, `paths` (3 × 6 × [x, y]), `steer` (rad), `accel` (m/s²), `brake_prob`, `dt` |
+| `detections_2d` | per camera: `label` (10 classes), `label_id`, `score`, `box_xyxy` (pixels at 768×432) |
+| `unknown_obstacles` | [x, y] of 2D "obstacle" boxes placed on the ground plane (METEOR's `unk2d`) |
+| `traffic_light` | `state` (none / green / yellow / red), `state_id`, `probs` |
+| `lane`, `lane_classes` | the BEV lane map, uint8 [800, 500] as PNG (0.2 m cells, row 0 = +80 m ahead, column 0 = +50 m left; classes bg, road, sidewalk, crosswalk, laneline, stopline, road_edge, marking, parking), after the optional seg fusion and road-edge thinning |
+| `stationary_head_healthy`, `meta` | METEOR's health check of the stationary head; `present` cameras, `v0`, `stream_id` |
+| `heads` (`to_dict("npz")` with `heads=True`) | the dense outputs: `seg2d`, `depth` (bins), `depth_mean`, `occupancy` (class per voxel), `risk` (sigmoid), `stationary` |
+| `timing_ms` | `preprocess`, `device` (tables + upload + replay + read + conversion), `postprocess`, `total` |
+
+`out.path` is the selected path as an array; `out.lane` the lane map; `out.boxes3d` / `out.boxes2d` the box objects.
+
+## Lifetime and information
+
+- `model.close()` releases the trace and the device tensors and closes the chip if the model opened it; idempotent.
+  `with` calls it for you; an unclosed model is closed when Python exits.
+- `model.info`: weights (repo, tag, revision, path), device (dispatch, grid), variant, warm variants, warm-up times,
+  runtime parameter defaults, the load-time knobs and the trace description.
+- Calls from several threads are safe: the device calls are serialised. One model per process per chip.
+- **Rig changes.** The lift tables (205 MB of fp32 bin weights + the sampling grid) are built on the host per
+  calibration (cached per rig) and written to the chip only when the calibration changes: 182 ms for the write, so a
+  call with a new rig takes about 1.13 s instead of about 0.9 s. A fixed rig pays it once.
+
+## Speed
+
+Warm, batch 1, ETH dispatch, 1 CQ, 12×10, AICLK 1350 MHz, p50 of 60 calls on a shared 8-core host
+([`OPT_BASELINE.md`](../OPT_BASELINE.md); the device rows do not depend on the input):
+
+| stage | ms |
+|---|---:|
+| device trace, one replay (the whole network) | 556.85 (back to back 556.75: 1.80 frames/s) |
+| upload of the eight cameras (8 MB uint8) | 66.5 |
+| readback (75 MB, 5 segments) + conversion to the 19 outputs | 17-48 + 65 |
+| host pre-processing / camera transpose | 9-11 / 22-26 |
+| host post-processing (METEOR's C++ decode rules, temporal state) | 124-131 |
+| `model(...)` end to end | 876-918 (1.09-1.14 calls/s) |
+
+This is the first release: optimization has not started ([`OPT_REPORT.md`](../OPT_REPORT.md)).
+
+## Limits
+
+- Batch 1 on the chip; one model per process; eight 768×432 camera slots in METEOR's order, a 400×250 lift grid at
+  0.4 m and an 800×500 BEV at 0.2 m (fixed shapes of the released graph); at most 3 cameras per lift cell on the
+  rigs validated.
+- The released graph is camera-only and single-frame (its temporal memory was baked out upstream); the temporal
+  behaviour of METEOR's runtime is host post-processing.
+- Numerics: bf16 / fp32 on the chip; outputs differ slightly from the fp32 reference (README "Demo & Performances").

code/conftest.py

@@ -0,0 +1,59 @@
+# SPDX-License-Identifier: Apache-2.0
+"""pytest fixtures of meteor-p150 (no dependency on tt-metal's own conftest).
+
+- ``device`` (session): one chip opened like the published numbers (``tt_meteor.device.open_device``:
+  ETH dispatch, 12x10 grid, the port's validated sizes and CQs). ``--device-id N`` or ``TT_DEVICE_ID`` selects
+  the chip (default 0); ``METEOR_DISPATCH=worker`` is the A/B opt-in, and the other ``METEOR_*``
+  variables apply as for the server. A failing ETH open is an error here, never a silent WORKER fallback: gates
+  are only valid on the published setup.
+- Tests marked ``device`` are skipped when ttnn is missing or ``TT_VISIBLE_DEVICES=none`` (host-only runs).
+  On the shared workspace box run them through the lock and name the test files:
+  ``bin/devrun -t 1800 -- python -m pytest -q -s code/tt_meteor/tests/test_pcc_device.py``.
+"""
+from __future__ import annotations
+
+import gc
+import importlib.util
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+    parser.addoption("--device-id", action="store", default=None, help="chip id (default $TT_DEVICE_ID or 0)")
+
+
+def pytest_configure(config):
+    config.addinivalue_line("markers", "device: needs a Tenstorrent chip (skipped on host-only runs)")
+
+
+def _no_device_reason():
+    if os.environ.get("TT_VISIBLE_DEVICES", "").lower() == "none":
+        return "TT_VISIBLE_DEVICES=none (host-only run)"
+    if importlib.util.find_spec("ttnn") is None:
+        return "ttnn is not installed"
+    return None
+
+
+def pytest_collection_modifyitems(config, items):
+    reason = _no_device_reason()
+    if reason:
+        skip = pytest.mark.skip(reason=reason)
+        for item in items:
+            if "device" in item.keywords:
+                item.add_marker(skip)
+
+
+@pytest.fixture(autouse=True)
+def _gc_between_tests():
+    gc.collect()
+
+
+@pytest.fixture(scope="session")
+def device(request):
+    from tt_meteor.device import close_device, open_device
+
+    cli = request.config.getoption("--device-id")
+    dev = open_device(int(cli) if cli is not None else None, allow_fallback=False)
+    yield dev
+    close_device(dev)

code/models/common/lightweightmodule.py

@@ -0,0 +1,12 @@
+# SPDX-FileCopyrightText: © 2023 Tenstorrent USA, Inc.
+
+# SPDX-License-Identifier: Apache-2.0
+
+
+class LightweightModule:
+    """Torch modules add a surprising amount of host overhead for attribute
+    access and method calls. This class is a lightweight alternative that
+    just wraps a forward function for now."""
+
+    def __call__(self, *args, **kwargs):
+        return self.forward(*args, **kwargs)

code/scripts/README.md

@@ -0,0 +1,18 @@
+# code/scripts
+
+| script | purpose | device |
+|---|---|---|
+| `fetch_samples.sh` | downloads the public sample data that is not redistributable (sha256-checked) | no |
+| `bench.py` | stage breakdown of warm forwards on several inputs (API split, host_in / H2D / trace / b2b / D2H, e2e p50 / p99, AICLK, the smoke agreement check; `--dispatch` / `--num-cqs` for the D14 matrix, `--rig-switch`): the card / OPT numbers | yes, via `bin/devrun` |
+| `profile_ops.py` | Tracy + device profile: one eager frame with module signposts and `--rounds` signposted traced replays (`python -m tracy -r -p --op-support-count 6000 ...`); `--graph-report DIR` writes a ttnn-visualizer graph report of one eager frame instead | yes, via `bin/devrun` |
+| `profile_summary.py` | summary of a `profile_ops.py` ops CSV: programs, kernel sum, span, op-to-op gaps, per-module / per-op-code tables (eager labels carried to the trace op for op), top ops | no |
+| `container_smoke.sh` | serve ONE serve profile of the built package (`--profile NAME`; default profile otherwise), run `server/smoke_test.py` against it (asserts ETH dispatch, the 12x10 grid and the profile's pins; compares with the stored CPU reference), keep the evidence (container log, `/info`, the `/predict` output and the result, in `logs/smoke/` of the repo or `--log-dir DIR`), always stop it; one `bin/devrun -t 3600 -k 150` window per profile | yes |
+| `make_synthetic_sample.py` | the shipped sample `samples/synthetic_8cam*` (generated, Apache-2.0): `render` (a ray-cast street seen by a generic METEOR-like rig, preset `calib/synthetic_8cam.json`), `optimise` (gradient ascent on the object pixels through the fp32 CPU reference until the 3D heads report the objects with margin), `finalise` (PNGs, request manifest, the stored CPU-reference `/predict` body, the feed sha256 golden) | no (research venv) |
+| `make_sample.py` | builds the PandaSet 019 frame 40 sample (METEOR demo-scene layout, request manifest, preset `pandaset_019`) from the research ship sample; `--dest staging` (default: the git-ignored `staging_samples_pandaset/`, not shipped until the user approves PandaSet samples) or `--dest package` | no |
+| `make_goldens.py` | CPU-reference goldens (taps + outputs of 4 frames under `research/meteor/goldens/`, never shipped), the small decoded golden `tests/goldens/pandaset_019_f40_reference.json` and the reference body `samples/pandaset_019_f40.reference.json` of the PandaSet sample (in the staging dir while it is not shipped); cross-checked against the research ONNX Runtime goldens (research venv) | no |
+| `stress_frames.py` | contained stress of the `frame` trace (the ETH-dispatch hang investigation): `--mode replay` / `rigs` / `segrigs`; every read bit-compared per rig; `--rigs shipped[:N]` needs no golden (rigs from the shipped sample: what the container check runs) | yes, via `bin/devrun` |
+| `run_public_frames.py` | the served model on the public-dataset frames of `research/meteor/public_data` (feed sha256 checked), TT outputs written in the CPU-golden layout for `compare_tt.py` and `make_demo.py` (development workspace only) | yes, via `bin/devrun` |
+| `make_demo.py` | the card's `media/` from TT outputs: cards, camera heads and the animation of the public frames (research renderer: privacy blur, attribution footers), the TT-vs-CPU bird's-eye view, the shipped-sample render, `media/ATTRIBUTION.md` | no (research venv) |
+| `dump_device_outputs.py`, `device_stage_check.py`, `precision_emulation.py`, `repro_*_eth.py` | port tools: raw device outputs of golden frames, eager stage checks, CPU emulation of precision policies, the standalone ETH-hang repros (PORT_LOG 12.7) | partly |
+
+Everything here ships in `code/` on the Hub and inside the image (`source.extra_code` lists `scripts`).

code/scripts/bench.py

@@ -0,0 +1,317 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Stage bench of warm forwards: the numbers OPT_BASELINE.md, OPT_REPORT.md and the card quote.
+
+    bin/devrun -t 1800 -- python code/scripts/bench.py --iters 60 --inputs sample,synthetic --json out.json
+    bin/devrun -t 1800 -- python code/scripts/bench.py --dispatch worker --num-cqs 2 --json ...   # the D14 matrix
+
+Inputs (``--inputs``, comma separated; each one is a request of ``model(...)``, batch 1):
+
+- ``sample``: the PandaSet 019 frame 40 sample ``pandaset_019_f40.json`` (the baseline's sample: the git-ignored
+  ``staging_samples_pandaset/`` copy until the user approves shipping it, else ``samples/``);
+- ``shipped``: the shipped synthetic sample ``samples/synthetic_8cam.json`` (``make_synthetic_sample.py``);
+- ``synthetic``: eight 768x432 uint8 noise images made here (seeded; data generated by us), the ``pandaset_019``
+  calibration (inline from the staged preset; the ``synthetic_8cam`` preset when it is absent), ego speed 10 m/s;
+- ``<name>=<manifest.json>`` or a manifest path: any ``load_sample`` manifest (e.g. a public-dataset frame).
+
+Images are decoded when the request is loaded, before timing (the camera driver hands METEOR decoded images). Per
+input, after ``--warm`` calls (``ttaw.profiling.StageBench``: p50 / p99 / mean / min / max in ms):
+
+1. **plain**: ``model(...)`` exactly as a client calls it: ``e2e`` and the API's own ``timing_ms``
+   (``api.preprocess`` / ``api.device`` / ``api.postprocess``), with ``run_frame`` split by wrapping its parts:
+   ``dev.tables`` (rig check, and the lift-table write when the rig changed), ``dev.image_rows`` (NCHW -> NHWC rows),
+   ``dev.runner`` (upload + replay + segmented read), ``dev.unpack`` (join of the readback segments),
+   ``dev.outputs`` (``outputs_from_device``: device layouts -> the 19 ONNX outputs on the host);
+2. **stages**: ``ttaw.profiling.bench_trace_runner`` on the ``frame`` variant: ``host_in`` (numpy -> ttnn host
+   tensor), ``h2d`` (upload + synchronize), ``trace`` (one replay + synchronize), ``d2h`` (segmented read), ``post``
+   (``unpack``), ``e2e`` (runner call + unpack) and ``b2b`` (back-to-back replays: the device time per frame).
+
+``--rig-switch N``: N calls alternating between the first two inputs (each call writes its rig's lift tables).
+An agreement check of every input with a stored CPU reference (``<stem>.reference.json``, fresh stream, the container
+smoke's ``compare_with_reference``) runs first, so every configuration's numbers come with its accuracy. AICLK /
+power / temperature are sampled during the loops (``AiclkSampler``).
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import platform
+import subprocess
+import sys
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+CODE = HERE.parents[1]
+sys.path.insert(0, str(CODE))
+SAMPLES = CODE / "tt_meteor" / "samples"
+STAGING = CODE.parent / "staging_samples_pandaset"
+ROOT = Path(os.environ.get("TT_MODELS_ROOT", "/home/ubuntu/experiments/tt-models"))
+SAMPLE_NAME = "pandaset_019_f40.json"
+
+
+# ------------------------------------------------------------------------------------------------- inputs
+def sample_path() -> Path:
+    for d in (SAMPLES, STAGING / "samples", STAGING):
+        if (d / SAMPLE_NAME).is_file():
+            return d / SAMPLE_NAME
+    return SAMPLES / SAMPLE_NAME
+
+
+def synthetic_request(seed: int = 1) -> Dict[str, Any]:
+    from tt_meteor.reference import config as C
+
+    rng = np.random.default_rng(seed)
+    images = {cam: rng.integers(0, 256, size=(C.IMG_H, C.IMG_W, 3), dtype=np.uint8) for cam in C.CAMERAS}
+    staged = [d / "pandaset_019.json" for d in (CODE / "tt_meteor" / "calib", STAGING / "calib") if
+              (d / "pandaset_019.json").is_file()]
+    calib = json.loads(staged[0].read_text()) if staged else {"preset": "synthetic_8cam"}
+    return {"images": images, "calibration": calib, "ego_speed": 10.0,
+            "stream": {"id": "synthetic", "pose": [0.0, 0.0, 0.0]}}
+
+
+def load_input(spec: str) -> Tuple[str, Dict[str, Any], Optional[Path]]:
+    from tt_meteor import load_sample
+
+    if spec == "synthetic":
+        return "synthetic", synthetic_request(), None
+    if spec == "sample":
+        path = sample_path()
+        return "sample", load_sample(path), path
+    if spec == "shipped":
+        path = SAMPLES / "synthetic_8cam.json"
+        return "shipped", load_sample(path), path
+    name, _, p = spec.partition("=")
+    path = Path(p or name)
+    return (name if p else path.stem), load_sample(path), path
+
+
+def find_reference(manifest: Optional[Path]) -> Optional[Path]:
+    if manifest is None:
+        return None
+    p = manifest.parent / f"{manifest.stem}.reference.json"
+    return p if p.is_file() else None
+
+
+# --------------------------------------------------------------------------------------------- environment
+def _git(*args: str) -> str:
+    try:
+        return subprocess.run(["git", *args], capture_output=True, text=True,
+                              env=dict(os.environ, GIT_OPTIONAL_LOCKS="0")).stdout.strip()
+    except OSError:
+        return ""
+
+
+def environment() -> Dict[str, Any]:
+    env: Dict[str, Any] = {"host": platform.node(), "python": platform.python_version(),
+                           "loadavg_start": os.getloadavg(), "cpus": os.cpu_count(),
+                           "omp_num_threads": os.environ.get("OMP_NUM_THREADS"),
+                           "env": {k: v for k, v in os.environ.items() if k.startswith("METEOR_")}}
+    try:
+        env["cpu_model"] = next(line.split(":", 1)[1].strip() for line in open("/proc/cpuinfo")
+                                if line.startswith("model name"))
+    except (OSError, StopIteration):
+        pass
+    tm = ROOT / "tt-metal"
+    env["tt_metal_commit"] = _git("-C", str(tm), "rev-parse", "HEAD")
+    try:
+        vend = json.loads((CODE / "tt_meteor/ttaw/VENDORED.json").read_text())
+        env["ttaw"] = vend.get("version")
+        env["ttaw_vendored"] = vend.get("source_commit")
+    except (OSError, ValueError):
+        pass
+    env["bundle_commit"] = _git("-C", str(CODE.parent), "rev-parse", "HEAD")
+    env["bundle_dirty"] = bool(_git("-C", str(CODE.parent), "status", "--porcelain", "code"))
+    return env
+
+
+def cache_entries(device) -> Optional[int]:
+    fn = getattr(device, "num_program_cache_entries", None)
+    try:
+        return int(fn()) if fn else None
+    except Exception:   # noqa: BLE001
+        return None
+
+
+# ------------------------------------------------------------------------------------------------- measure
+class Split:
+    """Times the parts of ``TtMETEOR.run_frame`` (module docstring) while ``bench`` is set."""
+
+    def __init__(self, model):
+        import tt_meteor.tt.model as tmodel
+
+        self.bench = None
+        tt = model.tt
+        self._undo = []
+
+        def timed(name, fn):
+            def wrapper(*a, **k):
+                if self.bench is None:
+                    return fn(*a, **k)
+                t0 = time.perf_counter()
+                try:
+                    return fn(*a, **k)
+                finally:
+                    self.bench.add(name, (time.perf_counter() - t0) * 1e3)
+            return wrapper
+
+        for attr, name in (("set_tables", "dev.tables"), ("image_rows", "dev.image_rows"), ("unpack", "dev.unpack"),
+                           ("runner", "dev.runner")):
+            orig = getattr(tt, attr)
+            setattr(tt, attr, timed(name, orig) if attr != "runner" else _CallTimer(orig, self, name))
+            self._undo.append(lambda tt=tt, attr=attr, orig=orig: setattr(tt, attr, orig))
+        orig_out = tmodel.outputs_from_device
+        tmodel.outputs_from_device = timed("dev.outputs", orig_out)
+        self._undo.append(lambda: setattr(tmodel, "outputs_from_device", orig_out))
+
+    def remove(self) -> None:
+        for fn in reversed(self._undo):
+            fn()
+
+
+class _CallTimer:
+    """Stands in for the TraceRunner inside ``run_frame``: times ``runner(...)``, forwards every attribute."""
+
+    def __init__(self, inner, split: Split, name: str):
+        self._inner, self._split, self._name = inner, split, name
+
+    def __call__(self, *a, **k):
+        if self._split.bench is None:
+            return self._inner(*a, **k)
+        t0 = time.perf_counter()
+        try:
+            return self._inner(*a, **k)
+        finally:
+            self._split.bench.add(self._name, (time.perf_counter() - t0) * 1e3)
+
+    def __getattr__(self, item):
+        return getattr(self._inner, item)
+
+
+def agreement(model, name: str, req: Dict[str, Any], ref_path: Path) -> Dict[str, Any]:
+    sys.path.insert(0, str(CODE / "tt_meteor" / "ttaw" / "server"))
+    import smoke as ttaw_smoke  # noqa: E402
+
+    fresh = dict(req)
+    fresh["stream"] = dict(req.get("stream") or {}, id=f"agreement-{name}-{time.time_ns()}")
+    body = model(**fresh).to_dict()
+    ref = json.loads(ref_path.read_text())
+    metrics, failures = ttaw_smoke.compare_with_reference(body, ref)
+    path_dev = float(np.linalg.norm(np.asarray(body["trajectory"]) - np.asarray(ref["trajectory"]), axis=-1).max())
+    return {"reference": str(ref_path), "metrics": metrics, "failures": failures, "ego.path_dev_m": path_dev,
+            "pass": not failures}
+
+
+def bench_input(model, split: Split, name: str, req: Dict[str, Any], iters: int, warm: int,
+                b2b: int) -> Dict[str, Any]:
+    from tt_meteor.host.inputs import prepare_request
+    from tt_meteor.ttaw.profiling import AiclkSampler, StageBench, bench_trace_runner
+
+    res: Dict[str, Any] = {"loadavg_start": os.getloadavg()}
+    for _ in range(warm):
+        model(**req)
+    plain = StageBench(f"meteor {name} plain")
+    with AiclkSampler(interval_s=0.05) as clk_p:
+        split.bench = plain
+        try:
+            for _ in range(iters):
+                t0 = time.perf_counter()
+                out = model(**req)
+                plain.add("e2e", (time.perf_counter() - t0) * 1e3)
+                for k, v in out.timing_ms.items():
+                    plain.add(f"api.{k}", v)
+        finally:
+            split.bench = None
+    res["plain"] = plain.summary()
+    res["aiclk_plain"] = clk_p.summary()
+    res["boxes3d"] = len(out.boxes3d)
+    print(plain.table(), flush=True)
+    frame = prepare_request(req["images"], req["calibration"], req["ego_speed"], req.get("stream"))
+    model.tt.set_tables(model.calib_cache.get(frame.K, frame.T_cam_ego))
+    tt = model.tt
+    with AiclkSampler(interval_s=0.05) as clk_s:
+        stages = bench_trace_runner(model.runner, "frame", {"imgs": tt.image_rows(frame.imgs)},
+                                    params=tt.frame_params(frame), iters=iters, warmup=warm,
+                                    name=f"meteor {name} stages", post=tt.unpack, b2b_iters=b2b)
+    res["stages"] = stages.summary()
+    res["aiclk_stages"] = clk_s.summary()
+    print(stages.table(), flush=True)
+    res["loadavg_end"] = os.getloadavg()
+    return res
+
+
+def rig_switch(model, split: Split, reqs: List[Tuple[str, Dict[str, Any]]], n: int) -> Dict[str, Any]:
+    from tt_meteor.ttaw.profiling import StageBench
+
+    bench = StageBench("meteor rig switch")
+    split.bench = bench
+    try:
+        for i in range(n):
+            name, req = reqs[i % 2]
+            t0 = time.perf_counter()
+            out = model(**req)
+            bench.add("e2e", (time.perf_counter() - t0) * 1e3)
+            for k, v in out.timing_ms.items():
+                bench.add(f"api.{k}", v)
+    finally:
+        split.bench = None
+    print(bench.table(), flush=True)
+    return {"inputs": [r[0] for r in reqs[:2]], "calls": n, **{"stages": bench.summary()}}
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--inputs", default="sample", help="comma list: sample | synthetic | <name>=<manifest> | path")
+    ap.add_argument("--iters", type=int, default=60)
+    ap.add_argument("--warm", type=int, default=3)
+    ap.add_argument("--b2b", type=int, default=20, help="back-to-back replays of the device-only row")
+    ap.add_argument("--rig-switch", type=int, default=0, help="calls alternating between the first two inputs")
+    ap.add_argument("--dispatch", default=None, choices=["eth", "worker"])
+    ap.add_argument("--num-cqs", type=int, default=None, choices=[1, 2])
+    ap.add_argument("--no-agreement", dest="agreement", action="store_false")
+    ap.add_argument("--json")
+    a = ap.parse_args()
+    from tt_meteor import METEOR
+
+    inputs = [load_input(s.strip()) for s in a.inputs.split(",") if s.strip()]
+    res: Dict[str, Any] = {"args": vars(a), "environment": environment(),
+                           "inputs": {n: str(p) if p else "synthetic" for n, _, p in inputs}}
+    t_load = time.perf_counter()
+    with METEOR.from_pretrained(dispatch=a.dispatch, num_command_queues=a.num_cqs) as model:
+        res["load_s"] = time.perf_counter() - t_load
+        res["warmup_ms"] = model.warmup_ms
+        res["config"] = model.device_info
+        res["program_cache_entries"] = cache_entries(model.device)
+        res["trace"] = model.runner.describe()
+        print("config:", json.dumps(res["config"]), "| load %.1f s" % res["load_s"], "| programs:",
+              res["program_cache_entries"], flush=True)
+        if a.agreement:
+            res["agreement"] = {}
+            for name, req, path in inputs:
+                ref = find_reference(path)
+                if ref is not None:
+                    res["agreement"][name] = agreement(model, name, req, ref)
+                    print("agreement", name, json.dumps(res["agreement"][name], default=str), flush=True)
+        split = Split(model)
+        res["per_input"] = {}
+        for name, req, _ in inputs:
+            res["per_input"][name] = bench_input(model, split, name, req, a.iters, a.warm, a.b2b)
+        if a.rig_switch and len(inputs) >= 2:
+            res["rig_switch"] = rig_switch(model, split, [(n, r) for n, r, _ in inputs], a.rig_switch)
+        split.remove()
+        res["program_cache_entries_end"] = cache_entries(model.device)
+    res["loadavg_end"] = os.getloadavg()
+    text = json.dumps(res, indent=1, default=str)
+    if a.json:
+        Path(a.json).parent.mkdir(parents=True, exist_ok=True)
+        Path(a.json).write_text(text + "\n")
+    print(text)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/container_smoke.sh

@@ -0,0 +1,137 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: Apache-2.0
+# Serve ONE serve profile of the BUILT container package, run the smoke test against it, keep the evidence, and always
+# stop it -- in one device-lock window. Run it once per serve profile (`tt-model profiles
+# <staged dir>/tt_kernel_manifest.json` lists them; without --profile the package's default profile is served):
+#
+#   ROOT=/home/ubuntu/experiments/tt-models
+#   $ROOT/bin/devrun -t 3600 -k 150 -- env -u HF_TOKEN -u HUGGING_FACE_HUB_TOKEN sg docker -c \
+#       "bash code/scripts/container_smoke.sh $ROOT/build/meteor-p150 [port] [--profile NAME] [--log-dir DIR]"
+#
+# The smoke test FAILS unless /info reports ETH dispatch and the 12x10 grid and runs what the package pins for the
+# profile (dispatch, CQs, variant, weights revision); it also compares the output with the stored CPU reference of the
+# sample when one exists (code/tt_meteor/server/smoke_test.py). Exit code: the smoke test's (0 = PASS); 1 when
+# serve fails, 2 on a usage error.
+#
+# Evidence, kept whatever the outcome (--log-dir, default: logs/smoke/ of this repo, next to code/), named
+# <name>[-<profile>]-<UTC time>.*:
+#   .container.log  the container's whole log (boot, requests, shutdown), followed while the container stops
+#   .info.json      GET /info as soon as the server is READY (device: dispatch, grid, cores; pins; versions)
+#   .smoke.json     the smoke test's /predict output (the SMOKE_OUT environment variable overrides this path)
+#   .result.json    profile, port, exit code, start / end times
+#
+# `tt-model serve` returns once the server is READY and leaves the container running, so the stop must happen before
+# the lock is released, also when devrun's timeout TERMs this script: the EXIT trap saves the log, then stops the
+# container cleanly with SIGTERM (120 s grace, hence devrun -k 150), never `docker kill`, which would leave the chip
+# dirty. Needs docker access (sg docker) and python3.
+set -u
+usage() { echo "usage: container_smoke.sh <staged package dir> [port] [--profile NAME] [--log-dir DIR]" >&2; }
+PROFILE=""
+LOG_DIR=""
+POSITIONAL=()
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --profile) [ $# -ge 2 ] || { usage; exit 2; }; PROFILE="$2"; shift 2 ;;
+    --profile=*) PROFILE="${1#--profile=}"; shift ;;
+    --log-dir) [ $# -ge 2 ] || { usage; exit 2; }; LOG_DIR="$2"; shift 2 ;;
+    --log-dir=*) LOG_DIR="${1#--log-dir=}"; shift ;;
+    -h|--help) usage; exit 0 ;;
+    -*) echo "container_smoke.sh: unknown option $1" >&2; usage; exit 2 ;;
+    *) POSITIONAL+=("$1"); shift ;;
+  esac
+done
+[ ${#POSITIONAL[@]} -ge 1 ] && [ ${#POSITIONAL[@]} -le 2 ] || { usage; exit 2; }
+STAGED="${POSITIONAL[0]}"
+PORT="${POSITIONAL[1]:-20000}"
+MANIFEST="$STAGED/tt_kernel_manifest.json"
+[ -f "$MANIFEST" ] || { echo "container_smoke.sh: $MANIFEST not found (run tt-model package first)" >&2; exit 2; }
+HERE="$(cd "$(dirname "$0")" && pwd)"
+PROFILE_ARGS=()
+[ -n "$PROFILE" ] && PROFILE_ARGS=(--profile "$PROFILE")
+
+# The package name and the served profile (tt-model's rule: --profile, else default_profile, else the first serve
+# profile), hence the container name tt-model gives it: tt-model-<name>-<profile> (tt_kernel/container.py).
+read -r NAME PROFILE_NAME < <(python3 - "$MANIFEST" "$PROFILE" <<'EOF'
+import json, sys
+m = json.load(open(sys.argv[1]))
+c = m.get("container") or {}
+profiles = c.get("serve_profiles") or [{}]
+print(m.get("name") or "model", sys.argv[2] or c.get("default_profile") or profiles[0].get("name") or "default")
+EOF
+)
+[ -n "${NAME:-}" ] || { echo "container_smoke.sh: cannot read the package name from $MANIFEST" >&2; exit 2; }
+CONTAINER="tt-model-$NAME-$PROFILE_NAME"
+
+[ -n "$LOG_DIR" ] || LOG_DIR="$(cd "$HERE/../.." && pwd)/logs/smoke"
+if ! mkdir -p "$LOG_DIR" 2>/dev/null || [ ! -w "$LOG_DIR" ]; then
+  echo "container_smoke.sh: cannot write $LOG_DIR; keeping the evidence in ${TMPDIR:-/tmp}" >&2
+  LOG_DIR="${TMPDIR:-/tmp}"
+fi
+LOG_DIR="$(cd "$LOG_DIR" && pwd)"
+STARTED="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
+STEM="$LOG_DIR/$NAME${PROFILE:+-$PROFILE}-$(date -u +%Y%m%dT%H%M%SZ)"
+SMOKE_JSON="${SMOKE_OUT:-$STEM.smoke.json}"
+
+fetch() {  # fetch URL FILE: one GET with a 30 s timeout; the body goes to FILE
+  python3 - "$1" "$2" <<'EOF'
+import sys, urllib.request
+try:
+    with urllib.request.urlopen(sys.argv[1], timeout=30) as r:
+        body = r.read()
+except Exception as e:  # noqa: BLE001 -- best effort: the smoke test reports the server's state
+    sys.exit(f"container_smoke.sh: GET {sys.argv[1]} failed: {e}")
+open(sys.argv[2], "wb").write(body)
+EOF
+}
+
+CHILD=""
+STOPPED=0
+cleanup() {
+  local rc=$?
+  [ "$STOPPED" = 1 ] && return
+  STOPPED=1
+  if [ -n "$CHILD" ]; then kill -TERM "$CHILD" 2>/dev/null; wait "$CHILD" 2>/dev/null; fi
+  # The log is lost with the container: follow it (whole history, then the shutdown lines) while it stops.
+  local logger=""
+  if docker inspect "$CONTAINER" >/dev/null 2>&1; then
+    docker logs --follow "$CONTAINER" > "$STEM.container.log" 2>&1 &
+    logger=$!
+  else
+    tt-model logs "${PROFILE_ARGS[@]}" "$MANIFEST" > "$STEM.container.log" 2>&1 || true
+  fi
+  tt-model stop "${PROFILE_ARGS[@]}" "$MANIFEST" || true
+  if [ -n "$logger" ]; then
+    for _ in $(seq 1 30); do kill -0 "$logger" 2>/dev/null || break; sleep 1; done
+    kill "$logger" 2>/dev/null
+    wait "$logger" 2>/dev/null
+  fi
+  python3 - "$STEM.result.json" "$NAME" "$PROFILE_NAME" "$PORT" "$rc" "$STARTED" "$CONTAINER" "$MANIFEST" <<'EOF'
+import datetime, json, sys
+out, name, profile, port, rc, started, container, manifest = sys.argv[1:]
+ended = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+json.dump({"bundle": name, "profile": profile, "port": int(port) if port.isdigit() else port, "rc": int(rc),
+           "result": "PASS" if rc == "0" else "FAIL",
+           "started": started, "ended": ended, "container": container, "manifest": manifest}, open(out, "w"), indent=1)
+EOF
+  echo "container_smoke.sh: rc=$rc; evidence in $STEM.*"
+}
+trap cleanup EXIT
+trap 'exit 143' TERM
+trap 'exit 130' INT
+
+# Each step runs in the background and is waited for: bash defers a trap while a FOREGROUND command runs, so a TERM
+# sent to this script alone (devrun's timeout signals the whole process group) would otherwise wait for the step.
+step() {
+  "$@" &
+  CHILD=$!
+  wait "$CHILD"
+  local rc=$?
+  CHILD=""
+  return "$rc"
+}
+
+# --port and --profile BEFORE the target: options after it are passed through to the container (tt-model cli rule)
+step tt-model serve --port "$PORT" "${PROFILE_ARGS[@]}" "$MANIFEST" || exit 1
+step fetch "http://127.0.0.1:$PORT/info" "$STEM.info.json"
+step python3 "$HERE/../tt_meteor/server/smoke_test.py" --url "http://127.0.0.1:$PORT" --wait 600 \
+    --manifest "$MANIFEST" "${PROFILE_ARGS[@]}" --out "$SMOKE_JSON"

code/scripts/device_stage_check.py

@@ -0,0 +1,80 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Bring-up check of one device stage, eagerly, against the CPU-reference goldens (needs one p150; run under devrun).
+
+    bin/devrun -t 1800 -- python bundles/meteor-p150/code/scripts/device_stage_check.py --stage image \\
+        --json logs/meteor/stage_image.json
+
+Stages (teacher-forced from ``research/meteor/goldens/<frame>/taps.npz``): ``image`` (the uint8 cameras -> FPN and the
+image heads), ``lift`` (golden ctx / prob -> lift.bev, bev.raw), ``bev`` (golden bev.raw -> trunk and BEV heads),
+``head`` (golden fused / lane.pre / det / risk -> planner, refiners, output tail). Prints and saves the PCC of every tap
+it produces; no gate here (the gates live in ``tests/test_pcc_device.py``, on trace replays). ``--runs 2`` runs the
+stage twice and checks the two eager outputs are bit-identical (PLAN 4.4 bring-up: eager x2 before any trace).
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+sys.path.insert(0, str(HERE.parents[1]))
+
+GOLDENS = Path(os.environ.get("METEOR_GOLDENS", "/home/ubuntu/experiments/tt-models/research/meteor/goldens"))
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--stage", required=True, choices=["image", "lift", "bev", "head"])
+    ap.add_argument("--frame", default="pandaset_019_f40")
+    ap.add_argument("--runs", type=int, default=1)
+    ap.add_argument("--json", default=None)
+    args = ap.parse_args()
+
+    from tt_meteor.device import close_device, describe_device, open_device
+    from tt_meteor.tt.debug import STAGES, compare_taps_to_golden
+    from tt_meteor.tt.params import MeteorParams
+
+    gold = np.load(GOLDENS / args.frame / "taps.npz")
+    t0 = time.perf_counter()
+    params = MeteorParams.load()
+    dev = open_device(allow_fallback=False)
+    info = describe_device(dev)
+    print("device:", {k: info[k] for k in ("dispatch", "grid", "num_command_queues")}, flush=True)
+    rows = {}
+    try:
+        stage = STAGES[args.stage](dev, params)
+        prev = None
+        for run in range(args.runs):
+            t1 = time.perf_counter()
+            taps = stage.run(gold)
+            dt = time.perf_counter() - t1
+            print(f"run {run}: {dt:.1f} s", flush=True)
+            if prev is not None:
+                for k in taps:
+                    same = np.array_equal(np.asarray(taps[k]), np.asarray(prev[k]), equal_nan=True)
+                    rows.setdefault("eager_x2_bit_identical", {})[k] = bool(same)
+                    if not same:
+                        print(f"  NOT bit-identical across eager runs: {k}", flush=True)
+            prev = taps
+        rows.update(compare_taps_to_golden(prev, gold))
+        stage.release()
+    finally:
+        close_device(dev)
+    for k, v in rows.items():
+        if k != "eager_x2_bit_identical":
+            print(f"  {k:24s} {v}")
+    print(f"total {time.perf_counter() - t0:.1f} s")
+    if args.json:
+        Path(args.json).parent.mkdir(parents=True, exist_ok=True)
+        Path(args.json).write_text(json.dumps({"stage": args.stage, "frame": args.frame, "device": info,
+                                               "rows": rows}, indent=1, default=str))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/dump_device_outputs.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Run golden frames' graph inputs through the served model and save the 19 raw device outputs (npz) for offline
+analysis against the goldens (e.g. which boxes a set-based gate misses and why).
+
+    bin/devrun -t 900 -- python code/scripts/dump_device_outputs.py --out DIR meteor_valday_f040 ...
+"""
+from __future__ import annotations
+
+import argparse
+import os
+from pathlib import Path
+
+import numpy as np
+
+from tt_meteor import METEOR
+from tt_meteor.host.preprocess import MeteorFrame
+
+GOLDENS = Path(os.environ.get("METEOR_GOLDENS", "/home/ubuntu/experiments/tt-models/research/meteor/goldens"))
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("frames", nargs="+")
+    ap.add_argument("--out", required=True)
+    ap.add_argument("--repeat", type=int, default=2, help="runs per frame (outputs must be bit-identical)")
+    a = ap.parse_args()
+    out = Path(a.out)
+    out.mkdir(parents=True, exist_ok=True)
+    with METEOR.from_pretrained() as model:
+        for name in a.frames:
+            with np.load(GOLDENS / name / "taps.npz") as z:
+                g = {k: z[k] for k in ("input.imgs", "input.K", "input.T_cam_ego", "input.v0", "input.present")}
+            fr = MeteorFrame(g["input.imgs"], g["input.K"], g["input.T_cam_ego"], g["input.v0"], g["input.present"])
+            runs = [{k: np.array(v) for k, v in model._forward({"frame": fr}).items()} for _ in range(a.repeat)]
+            for r in runs[1:]:
+                for k in r:
+                    assert np.array_equal(r[k], runs[0][k]), f"{name}: {k} differs between runs"
+            np.savez(out / f"{name}.npz", **runs[0])
+            print(f"{name}: saved {len(runs[0])} outputs, {a.repeat} runs bit-identical")
+
+
+if __name__ == "__main__":
+    main()

code/scripts/fetch_samples.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: Apache-2.0
+# Download the public sample data that meteor-p150 may NOT redistribute (license unstated or non-commercial)
+# into ${METEOR_SAMPLES:-$HOME/.cache/tt_meteor/samples}, sha256-checked. Tests and benchmarks read it from there.
+# Nothing downloaded here is executed; parse it with numpy / sqlite3 only.
+set -euo pipefail
+DEST="${METEOR_SAMPLES:-$HOME/.cache/tt_meteor/samples}"
+mkdir -p "$DEST"
+
+fetch() {  # fetch <url> <sha256> <file name>
+  local url="$1" sum="$2" name="$3"
+  if [ -f "$DEST/$name" ] && echo "$sum  $DEST/$name" | sha256sum -c --status; then
+    echo "ok      $name"; return
+  fi
+  curl -fL --retry 3 -o "$DEST/$name.part" "$url"
+  echo "$sum  $DEST/$name.part" | sha256sum -c --status || { echo "sha256 mismatch: $name" >&2; exit 1; }
+  mv "$DEST/$name.part" "$DEST/$name"; echo "fetched $name"
+}
+
+# Example (Autoware demo rosbag; pinned in autoware/ansible/roles/demo_artifacts/tasks/main.yaml:48-53):
+# fetch https://autoware-files.s3.us-west-2.amazonaws.com/recordings/bags/demos/sample-rosbag.zip \
+#       5f9d36353393b3d249212153c19049822b1298db56512aa045b4f7f6fc37cf88 sample-rosbag.zip
+# METEOR demo scenes (research / demonstration use only) and nuScenes-derived inputs (CC BY-NC-SA 4.0) are never shipped;
+# fetch AutowareFoundation/meteor-demo-scenes at the pinned revision bd5f94dc79a0fbbd1836ad5e7198b1feb1b614d3 (research/meteor/SPEC.md section 7)
+echo "samples in $DEST"

code/scripts/make_demo.py

@@ -0,0 +1,322 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""The card's demo media (``media/``) from outputs of this port on the p150. Development tool of the workspace (CPU
+only; research venv: numpy, OpenCV, PIL), run after the device jobs that produce the TT outputs:
+
+    PY=/home/ubuntu/experiments/tt-models/tools/research-venv/bin/python
+    $PY code/scripts/make_demo.py --tt-public logs/public_tt --tt-sample logs/meteor/shipped_sample_tt.json \\
+        [--compare logs/public_tt/compare]
+
+- ``--tt-public``: ``code/scripts/run_public_frames.py``'s output (TT outputs of the public frames in the CPU-golden
+  layout). Renders, with the research renderer ``research/meteor/public_data/scripts/render_media.py`` (privacy
+  blur, dataset attribution in every footer), the card and the camera-head images of PandaSet 019 frame 40,
+  PandaSet 090 frame 40 and nuScenes scene-0103 key-frame 9 (non-commercial, labelled ``_NC``), the PandaSet 019
+  animation, and a bird's-eye TT-vs-CPU comparison of PandaSet 019 frame 40 (no camera pixels).
+- ``--tt-sample``: the TT ``/predict`` body of the shipped synthetic sample (``tests/test_e2e_device.py`` keeps it);
+  renders the eight synthetic cameras with the TT 2D boxes and the TT and CPU-reference bird's-eye views side by side.
+- ``media/ATTRIBUTION.md``: every file, its source frames, licence and changes.
+"""
+from __future__ import annotations
+
+import argparse
+import base64
+import io
+import json
+import math
+import os
+import sys
+from pathlib import Path
+
+import numpy as np
+
+BUNDLE = Path(__file__).resolve().parents[2]
+ROOT = Path(os.environ.get("TT_MODELS_ROOT", "/home/ubuntu/experiments/tt-models"))
+PUBLIC = ROOT / "research" / "meteor" / "public_data"
+sys.path.insert(0, str(PUBLIC / "scripts"))
+sys.path.insert(0, str(ROOT / "research" / "meteor" / "scripts"))
+MEDIA = BUNDLE / "media"
+PKG = BUNDLE / "code" / "tt_meteor"
+TT_LABEL = "Tenstorrent p150 output: meteor-p150 port (Blackhole, ETH dispatch 12x10, 1 CQ, one metal trace)"
+CPU_LABEL = "fp32 CPU reference (the same network, ONNX Runtime parity)"
+CARDS = [("ps019", 20), ("ps090", 20), ("ns0103", 9)]
+
+
+def _rm():
+    import render_media as rm  # research renderer (privacy blur, footers)
+
+    return rm
+
+
+def tag_of(gid: str, i: int, F) -> str:
+    return f"meteor_{gid}_f{i:04d}" + ("_NC" if F.dataset == "nuScenes" else "")
+
+
+def save_jpeg(path: Path, rgb: np.ndarray, q: int = 88) -> None:
+    from PIL import Image
+
+    Image.fromarray(rgb).save(path, quality=q, optimize=True)
+
+
+def save_png(path: Path, rgb: np.ndarray) -> None:
+    from PIL import Image
+
+    Image.fromarray(rgb).save(path, optimize=True)
+
+
+def public_renders(tt_dir: Path, entries: list) -> None:
+    rm = _rm()
+    for gid, i in CARDS:
+        F = rm.Frame(gid, i, str(tt_dir / gid))
+        tag = tag_of(gid, i, F)
+        lic = "CC BY 4.0 + PandaSet Dataset Terms" if F.dataset == "PandaSet" else "**CC BY-NC-SA 4.0, non-commercial**"
+        rgb, nb = rm.card(F, TT_LABEL)
+        save_jpeg(MEDIA / f"{tag}_card_tt.jpg", rgb)
+        entries.append((f"{tag}_card_tt.jpg", F.dataset, F.desc, lic, "TT outputs: 8-slot camera mosaic with 2D boxes; "
+                        "BEV lanes / risk / 3D boxes / futures / plan vs the logged path; front view with projected "
+                        "3D boxes and the planned path", rgb.shape))
+        hd = rm.heads(F, TT_LABEL)
+        save_jpeg(MEDIA / f"{tag}_heads_tt.jpg", hd)
+        entries.append((f"{tag}_heads_tt.jpg", F.dataset, F.desc, lic, "TT outputs: per-camera seg2d overlay and "
+                        "depth (linear bins), 8 slots", hd.shape))
+        print("rendered", tag, flush=True)
+    frames = sorted(int(p.stem[1:]) for p in (tt_dir / "ps019").glob("f[0-9][0-9][0-9][0-9].json"))
+    p = MEDIA / "meteor_ps019_seq_tt.gif"
+    Fs, dur = rm.animation("ps019", frames, str(p), TT_LABEL, golden_dir=str(tt_dir / "ps019"))
+    from PIL import Image
+
+    with Image.open(p) as im:
+        shape = (im.height, im.width, 3)
+    entries.append((p.name, "PandaSet", f"{Fs[0].desc} ... {Fs[-1].desc} (every 4th frame, {len(frames)} frames, "
+                    f"{dur} ms each: real time)", "CC BY 4.0 + PandaSet Dataset Terms", "TT outputs: the card as an "
+                    "animation (128-colour GIF)", shape))
+    print("rendered", p.name, flush=True)
+
+
+def _text(draw, xy, s, size=12, fill=(235, 235, 235), bold=False, anchor="la"):
+    _rm().text(draw, xy, s, size, fill, bold, anchor)
+
+
+def bev_compare(tt_dir: Path, compare: dict, entries: list) -> None:
+    """PandaSet 019 frame 40: CPU BEV | TT BEV | where the two lane maps differ (no camera pixels)."""
+    import cv2
+    from PIL import Image, ImageDraw
+
+    rm = _rm()
+    gid, i = "ps019", 20
+    Fc = rm.Frame(gid, i, None)
+    Ft = rm.Frame(gid, i, str(tt_dir / gid))
+    w, h = 300, 450
+    a, b = rm.bev_panel(Fc, w, h), rm.bev_panel(Ft, w, h)
+    view_f, view_r, yh = 50.0, 25.0, 25.0
+    r0, r1 = int((80 - view_f) / 0.2), int((80 + view_r) / 0.2)
+    c0, c1 = int((50 - yh) / 0.2), int((50 + yh) / 0.2)
+    lc, lt = Fc.out["lane"][0][r0:r1, c0:c1], Ft.out["lane"][0][r0:r1, c0:c1]
+    diff = np.full(lc.shape + (3,), 30, np.uint8)
+    diff[lc > 0] = (70, 70, 75)
+    diff[lc != lt] = (255, 60, 60)
+    diff = cv2.resize(diff, (w, h), interpolation=cv2.INTER_NEAREST)
+    H = 44 + 16 + h + 92
+    canvas = np.full((H, 3 * w + 4 * 8, 3), rm.BG, np.uint8)
+    for k, img in enumerate((a, b, diff)):
+        rm.paste(canvas, img, 8 + k * (w + 8), 60)
+    im = Image.fromarray(canvas)
+    d = ImageDraw.Draw(im)
+    _text(d, (8, 6), f"METEOR v1.0 on {Fc.desc}: Tenstorrent p150 vs the fp32 CPU reference (bird's-eye view)", 14, bold=True)
+    _text(d, (8, 26), "Pixel-free BEV panels: lane classes, risk heat, 3D boxes, futures, the 3 ego paths (selected: green); "
+          "white circles: logged path", 10, fill=rm.TEXT2)
+    for k, lab in enumerate(("fp32 CPU reference", "Tenstorrent p150 (this port)", "lane cells that differ (red)")):
+        _text(d, (8 + k * (w + 8), 44), lab, 12, bold=True)
+    r = compare.get(f"{gid}/f{i:04d}", {})
+    lines = []
+    if r:
+        lines.append(f"Agreement on this frame: lane {r['lane_agree']:.4f}, seg2d {r['seg2d_agree']:.4f}, depth "
+                     f"{r['depth_agree']:.4f} (argmax maps); hm / reg / stationary / risk PCC {r['hm_pcc']:.5f} / "
+                     f"{r['reg_pcc']:.5f} / {r['stationary_pcc']:.5f} / {r['risk_pcc']:.5f};")
+        lines.append(f"3D boxes matched {r['det3d_matched']:.3f} (same class, <= 1 m); selected ego path mean deviation "
+                     f"{r['ego_path_dev_m']:.3f} m, same mode: {r['ego_mode_equal']}; traffic light equal: {r['tl_equal']}.")
+    lines += ["Contains data from PandaSet (Scale AI and Hesai), https://pandaset.org, CC BY 4.0 + PandaSet Dataset Terms "
+              "(outputs only; no camera pixels).", "Scale AI and Hesai do not endorse this work."]
+    for k, line in enumerate(lines):
+        _text(d, (8, 60 + h + 8 + 15 * k), line, 10, fill=rm.TEXT1 if k < 2 else rm.TEXT2)
+    out = np.asarray(im)
+    path = MEDIA / "meteor_ps019_f0020_bev_tt_vs_cpu.png"
+    save_png(path, out)
+    entries.append((path.name, "PandaSet", Fc.desc, "CC BY 4.0 + PandaSet Dataset Terms", "BEV of the fp32 CPU "
+                    "reference and of the TT output side by side, and the lane cells that differ (no camera pixels)",
+                    out.shape))
+    print("rendered", path.name, flush=True)
+
+
+# ------------------------------------------------------------------------------------------- shipped sample
+def _decode_lane(body: dict) -> np.ndarray:
+    from PIL import Image
+
+    lane = body["lane"]
+    a = np.asarray(Image.open(io.BytesIO(base64.b64decode(lane["data"]))))
+    return a.reshape(lane["shape"])
+
+
+def body_bev(body: dict, w: int, h: int, view_f=50.0, view_r=25.0, yh=25.0) -> np.ndarray:
+    import cv2
+
+    rm = _rm()
+    lane = _decode_lane(body)
+    r0, r1 = int((80 - view_f) / 0.2), int((80 + view_r) / 0.2)
+    c0, c1 = int((50 - yh) / 0.2), int((50 + yh) / 0.2)
+    img = cv2.resize(np.asarray(rm.LANE_RGB, np.uint8)[lane[r0:r1, c0:c1]], (w, h), interpolation=cv2.INTER_NEAREST)
+    s = h / (view_f + view_r)
+
+    def px(x, y):
+        return (yh - y) * s, (view_f - x) * s
+
+    for dd in range(-int(view_r // 10) * 10, int(view_f) + 1, 10):
+        yy = int(round((view_f - dd) * s))
+        cv2.line(img, (0, yy), (w, yy), (55, 55, 60), 1)
+        if dd:
+            cv2.putText(img, f"{dd}m", (3, yy - 3), cv2.FONT_HERSHEY_SIMPLEX, 0.32, (150, 150, 150), 1, cv2.LINE_AA)
+    for det in body["detections"]:
+        (x, y), (L, W), yaw = det["center"], det["size"], det["yaw"]
+        col = rm.STAT_RGB if det.get("stationary") else (rm.VEH_RGB if det["label_id"] == 0 else rm.VRU_RGB)
+        cors = rm.box_corners(x, y, 0, L, W, 0, yaw)[:4]
+        p = np.array([px(cx, cy) for cx, cy, _ in cors]).round().astype(np.int32)
+        cv2.polylines(img, [p], True, col, 2, cv2.LINE_AA)
+        cx, cy = px(x, y)
+        fx, fy = px(x + 0.5 * L * math.cos(yaw), y + 0.5 * L * math.sin(yaw))
+        cv2.line(img, (int(cx), int(cy)), (int(fx), int(fy)), col, 2, cv2.LINE_AA)
+        if det.get("future") and not det.get("stationary") and det["label_id"] == 0:
+            q = np.array([[cx, cy]] + [px(a, b) for a, b in det["future"]]).round().astype(np.int32)
+            cv2.polylines(img, [q], False, col, 1, cv2.LINE_AA)
+    ex, ey = px(0, 0)
+    sel = body["plan"]["mode"]
+    for k, path in enumerate(body["plan"]["paths"]):
+        q = np.array([[ex, ey]] + [px(a, b) for a, b in path]).round().astype(np.int32)
+        cv2.polylines(img, [q], False, rm.PLAN_RGB if k == sel else (0, 150, 60), 2 if k == sel else 1, cv2.LINE_AA)
+        if k == sel:
+            for pt in q[1:]:
+                cv2.circle(img, tuple(int(v) for v in pt), 3, rm.PLAN_RGB, -1, cv2.LINE_AA)
+    tri = np.array([[ex, ey - 9], [ex - 6, ey + 7], [ex + 6, ey + 7]]).round().astype(np.int32)
+    cv2.fillPoly(img, [tri], (255, 255, 255))
+    return img
+
+
+def sample_render(tt_body: dict, entries: list) -> None:
+    import cv2
+    from PIL import Image, ImageDraw
+
+    rm = _rm()
+    ref = json.loads((PKG / "samples" / "synthetic_8cam.reference.json").read_text())
+    spec = json.loads((PKG / "samples" / "synthetic_8cam.json").read_text())
+    tw, th, gap, x0, top, lab = 233, 131, 6, 4, 44, 14
+    bw, bh = 300, 450
+    y_bot = top + 2 * (lab + th) + 12
+    H = y_bot + 18 + bh + 70
+    canvas = np.full((H, 960, 3), rm.BG, np.uint8)
+    for k, cam in enumerate(rm.TILE_ORDER):
+        img = np.asarray(Image.open(PKG / "samples" / spec["images"][cam]).convert("RGB")).copy()
+        for det in tt_body["detections_2d"].get(cam, []):
+            a, b, c, e = (int(round(v)) for v in det["box_xyxy"])
+            cv2.rectangle(img, (a, b), (c, e), rm.DET10_RGB[det["label_id"] % 10], 2)
+        img = cv2.resize(img, (tw, th), interpolation=cv2.INTER_AREA)
+        rm.paste(canvas, img, x0 + (k % 4) * (tw + gap), top + (k // 4) * (lab + th) + lab)
+    rm.paste(canvas, body_bev(ref, bw, bh), x0, y_bot + 18)
+    rm.paste(canvas, body_bev(tt_body, bw, bh), x0 + bw + 8, y_bot + 18)
+    im = Image.fromarray(canvas)
+    d = ImageDraw.Draw(im)
+    _text(d, (8, 6), "METEOR v1.0 on the shipped synthetic sample (samples/synthetic_8cam): Tenstorrent p150 vs the fp32 CPU "
+          "reference", 14, bold=True)
+    _text(d, (8, 26), "Synthetic test frame generated by this repository (Apache-2.0): a ray-cast street; the object "
+          "pixels were optimised against the CPU reference", 10, fill=rm.WARN)
+    for k, cam in enumerate(rm.TILE_ORDER):
+        n2d = len(tt_body["detections_2d"].get(cam, []))
+        _text(d, (x0 + (k % 4) * (tw + gap), top + (k // 4) * (lab + th) + 1), f"{cam[4:]}  (TT 2D boxes: {n2d})",
+              10, fill=rm.TEXT2)
+    _text(d, (x0, y_bot), "BEV: fp32 CPU reference (stored)", 12, bold=True)
+    _text(d, (x0 + bw + 8, y_bot), "BEV: Tenstorrent p150", 12, bold=True)
+    ix = x0 + 2 * (bw + 8) + 4
+    lines = [f"3D boxes: TT {tt_body['num_detections']}, reference {ref['num_detections']}",
+             f"plan mode: TT {tt_body['plan']['mode']} (p={tt_body['plan']['mode_probs'][tt_body['plan']['mode']]:.2f}), "
+             f"reference {ref['plan']['mode']}",
+             "path end (m): TT ({:.2f}, {:.2f}), reference ({:.2f}, {:.2f})".format(*tt_body["trajectory"][-1],
+                                                                                  *ref["trajectory"][-1]),
+             f"traffic light: TT {tt_body['traffic_light']['state']}, reference {ref['traffic_light']['state']}",
+             f"2D boxes: TT {sum(len(v) for v in tt_body['detections_2d'].values())}, reference "
+             f"{sum(len(v) for v in ref['detections_2d'].values())}"]
+    lane_t, lane_r = _decode_lane(tt_body), _decode_lane(ref)
+    lines.append(f"lane map agreement: {float((lane_t == lane_r).mean()):.4f}")
+    by_label: dict = {}
+    for det in tt_body["detections"]:
+        by_label.setdefault(det["label"], []).append(round(det["score"], 2))
+    for k, v in sorted(by_label.items()):
+        lines.append(f"TT {k} scores: " + ", ".join(f"{x:.2f}" for x in v))
+    for k, line in enumerate(lines):
+        _text(d, (ix, y_bot + 22 + 17 * k), line, 11)
+    ly = y_bot + 22 + 17 * len(lines) + 10
+    for name, colr in (("vehicle", rm.VEH_RGB), ("VRU", rm.VRU_RGB), ("stationary", rm.STAT_RGB), ("plan", rm.PLAN_RGB)):
+        d.rectangle([ix, ly + 3, ix + 8, ly + 11], outline=colr, width=2)
+        _text(d, (ix + 12, ly), name, 10, fill=rm.TEXT2)
+        ly += 15
+    _text(d, (8, H - 18), "Data generated by this repository (code/scripts/make_synthetic_sample.py), Apache-2.0. "
+          "No third-party pixels.", 10, fill=rm.TEXT2)
+    out = np.asarray(im)
+    path = MEDIA / "meteor_synthetic_8cam_tt_vs_cpu.png"
+    save_png(path, out)
+    entries.append((path.name, "synthetic (this repository)", "code/tt_meteor/samples/synthetic_8cam", "Apache-2.0",
+                    "the shipped sample's eight cameras with the TT 2D boxes; BEV of the stored CPU reference and of "
+                    "the TT output", out.shape))
+    print("rendered", path.name, flush=True)
+
+
+def write_attribution(entries: list) -> None:
+    rm = _rm()
+    C = rm.C
+    lines = ["# media/: sources and licences", "",
+             "Every model output drawn here was computed by **this port on one Tenstorrent Blackhole p150** (ETH "
+             "dispatch, 12x10 grid, 1 command queue, the `frame` metal trace), unless a panel says \"CPU reference\" "
+             "(the port's fp32 CPU reference of the same network, which matches ONNX Runtime on "
+             "`meteor_v157c3Z.onnx`). Renderer: `code/scripts/make_demo.py` (camera mosaics, heads and animation through "
+             "`research/meteor/public_data/scripts/render_media.py` of the development workspace).", "",
+             "METEOR was trained on TIER IV recordings only (no paper, no public training split). PandaSet and nuScenes "
+             "are out of its training domain (another camera rig, country and ISP), so the detections show the domain "
+             "gap, not the port. No METEOR demo-scene frame appears in any image.", "",
+             "| file | data | frames | content | licence | size (px) |", "|---|---|---|---|---|---|"]
+    for f, ds, frames, lic, content, shape in entries:
+        lines.append(f"| `{f}` | {ds} | {frames} | {content} | {lic} | {shape[1]}x{shape[0]} |")
+    lines += ["", "Changes to the dataset images: resized to the 768x432 METEOR inputs (and a centre crop for the "
+              "virtual FRONT_NARROW camera), downscaled for display (<= 960 px wide), heads of annotated pedestrians "
+              "(<= 30 m) and plate areas of annotated vehicles (<= 25 m) blurred, model outputs drawn. The BACK_NARROW "
+              "tile is black because that camera is absent (the model gets a zero image plus the BACK_WIDE pose, its "
+              "trained 7-camera configuration).", "",
+              "Attribution texts (also printed in each image footer):", "",
+              "- PandaSet: " + C.PS_ATTRIBUTION,
+              "- nuScenes (files marked `_NC`, **non-commercial**): " + C.NS_ATTRIBUTION,
+              "- Synthetic sample: data generated by this repository (`code/scripts/make_synthetic_sample.py`), "
+              "Apache-2.0.", "",
+              "Rules (research/DATASETS.md section 2): the nuScenes renders are **non-commercial (CC BY-NC-SA 4.0, "
+              "ShareAlike)** and keep that label next to the image wherever they are shown; the PandaSet renders are "
+              "CC BY 4.0 + the PandaSet Dataset Terms (do not use them to identify people; no use of the licensors' "
+              "names or logos beyond the attribution). No raw dataset file is in this directory."]
+    (MEDIA / "ATTRIBUTION.md").write_text("\n".join(lines) + "\n")
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--tt-public", type=Path, required=True)
+    ap.add_argument("--tt-sample", type=Path, help="TT /predict body of the shipped sample (omit: no sample render)")
+    ap.add_argument("--compare", type=Path, help="compare_tt.py JSON files (<id>.json) of the TT run vs the goldens")
+    a = ap.parse_args()
+    MEDIA.mkdir(exist_ok=True)
+    compare = {}
+    if a.compare:
+        for p in sorted(a.compare.glob("*.json")):
+            for r in json.loads(p.read_text()).get("rows", []):
+                compare[f"{p.stem}/f{r['frame']:04d}"] = r
+    entries: list = []
+    public_renders(a.tt_public, entries)
+    bev_compare(a.tt_public, compare, entries)
+    if a.tt_sample:
+        sample_render(json.loads(a.tt_sample.read_text()), entries)
+    write_attribution(entries)
+
+
+if __name__ == "__main__":
+    main()

code/scripts/make_goldens.py

@@ -0,0 +1,237 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Goldens of the METEOR port from the bundle's fp32 CPU reference (``tt_meteor.reference``), cross-checked against
+the research ONNX Runtime goldens of the same frames.
+
+Run in the research venv (torch, onnx; no device)::
+
+    cd bundles/meteor-p150 && OMP_NUM_THREADS=4 PYTHONPATH=$PWD/code \\
+        /home/ubuntu/experiments/tt-models/tools/research-venv/bin/python code/scripts/make_goldens.py [--frames ...]
+
+Writes (large files never go into the bundle):
+
+- ``research/meteor/goldens/<frame>/taps.npz``: the reference's taps (``ttaw.golden`` format, JSON ``__meta__``): every
+  tap for the shipped sample, the gate taps for the other frames; float tensors above 1 M elements as float16, the
+  rest float32, the 19 outputs in their ONNX dtypes, the feed (``input.*``) and the lift geometry tables.
+- ``research/meteor/goldens/report.json``: the cross-check of every frame against its ORT golden (PCC, agreement).
+- ``tests/goldens/pandaset_019_f40_reference.json``: the decoded reference result of the PandaSet sample (small; the
+  host / e2e tests and the card compare against it);
+- ``samples/pandaset_019_f40.reference.json``: the ``/predict`` body of the reference on the PandaSet sample request
+  (``tests/test_e2e_device.py`` compares the served body with it).
+  Both under ``staging_samples_pandaset/`` (the PandaSet sample does not ship until the user approves it; once it is
+  swapped into ``code/tt_meteor/`` they are written there). The shipped synthetic sample's reference is written by
+  ``code/scripts/make_synthetic_sample.py``.
+- ``--variants``: ``research/meteor/goldens/<variant>/taps.npz`` (gate taps) of the LOAD-knob variants of the graph
+  (``VARIANTS``: the D12 ``input_norm=imagenet`` option with ``depth_mean_bins=linear``, on the shipped sample with
+  CAM_FRONT_NARROW absent: zero image + the donor's calibration, so the ``present`` mask matters; plus its
+  ``_unzeroed`` counter-example, the mask not applied) for ``tests/test_variants_device.py``. The imagenet reference is itself checked against ONNX Runtime
+  (``test_reference_cpu.py::test_imagenet_variant``).
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+BUNDLE = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(BUNDLE / "code"))
+
+from tt_meteor import __version__  # noqa: E402
+from tt_meteor.api import load_sample  # noqa: E402
+from tt_meteor.host.calib import lift_geometry  # noqa: E402
+from tt_meteor.host.postprocess import PostConfig  # noqa: E402
+from tt_meteor.host.preprocess import MeteorFrame, load_scene_frame, scene_dir_of  # noqa: E402
+from tt_meteor.host.result import build_output  # noqa: E402
+from tt_meteor.reference import config as C  # noqa: E402
+from tt_meteor.reference.model import OUTPUT_NAMES  # noqa: E402
+from tt_meteor.reference.pipeline import MeteorReference  # noqa: E402
+from tt_meteor.ttaw.golden import TapRegistry, save_goldens  # noqa: E402
+from tt_meteor.ttaw.metrics import pcc  # noqa: E402
+
+ROOT = Path("/home/ubuntu/experiments/tt-models")
+R = ROOT / "research" / "meteor"
+OUT = R / "goldens"
+_PKG = BUNDLE / "code" / "tt_meteor"
+_ROOT = _PKG if (_PKG / "samples" / "pandaset_019_f40.json").is_file() else BUNDLE / "staging_samples_pandaset"
+SHIP = _ROOT / "samples" / "pandaset_019_f40"
+GATE_TAPS = ["img.fpn", "seg2d.logits", "depth.logits", "ctx.painted", "lift.weight", "lift.bev", "bev.raw",
+             "bev.fused", "lane.pre", "det.feat", "det.hm_pre", "det.reg_pre", "traj.agent_delta", "ego.*",
+             "bev.fused_mean", "refiner.e2e_ln", "refiner.e2e_res", "lift.valid", "lift.grid"] + list(OUTPUT_NAMES)
+EXACT_TAPS = {"lift.grid"}       # kept float32 whatever their size (bit-exact geometry)
+# frame id -> (scene dir or None, frame index, ORT golden, all taps?, licence note)
+FRAMES = {
+    "pandaset_019_f40": (SHIP, 0, R / "public_data/golden/ps019/full_f0020.npz", True,
+                         "PandaSet CC BY 4.0 + Terms (the shipped sample)"),
+    "pandaset_090_f40": (R / "public_data/inputs/ps090", 20, R / "public_data/golden/ps090/full_f0020.npz", False,
+                         "PandaSet CC BY 4.0 + Terms"),
+    "nuscenes_0103_kf09": (R / "public_data/inputs/ns0103", 9, R / "public_data/golden/ns0103/full_f0009.npz", False,
+                           "nuScenes CC BY-NC-SA 4.0: internal validation only, never shipped"),
+    "meteor_valday_f040": (ROOT / "assets/meteor/hf_meteor-demo-scenes/valday", 40, R / "golden/valday_f040_ort.npz",
+                           False, "METEOR demo scenes: research / demonstration use only, never shipped"),
+}
+
+# variant id -> (base frame, MeteorConfig overrides, absent cameras, taps, present flags of the absent cameras)
+# ``_unzeroed``: the same zero images but every camera flagged present, i.e. what the graph computes when the imagenet
+# present mask is NOT applied (absent camera = the normalised zero image -mean / std): the counter-example that shows
+# the device's absent-camera outputs come from the zeroed input (``tests/test_variants_device.py``).
+_IMAGENET_LINEAR = {"input_norm": "imagenet", "depth_mean_bins": "linear"}
+VARIANTS = {
+    "pandaset_019_f40_imagenet_linear": ("pandaset_019_f40", _IMAGENET_LINEAR, ("CAM_FRONT_NARROW",), GATE_TAPS,
+                                         False),
+    "pandaset_019_f40_imagenet_linear_unzeroed": ("pandaset_019_f40", _IMAGENET_LINEAR, ("CAM_FRONT_NARROW",),
+                                                  list(OUTPUT_NAMES), True),
+}
+
+
+def sha(a) -> str:
+    return hashlib.sha256(np.ascontiguousarray(a).tobytes()).hexdigest()
+
+
+def store(taps: dict) -> dict:
+    out = {}
+    for k, v in taps.items():
+        v = np.asarray(v)
+        if k in OUTPUT_NAMES or v.dtype in (np.bool_, np.uint8, np.float16) or v.dtype.kind in "iu":
+            out[k] = v
+        elif k in EXACT_TAPS:
+            out[k] = v.astype(np.float32)
+        elif v.size > (1 << 20):
+            out[k] = v.astype(np.float16)
+        else:
+            out[k] = v.astype(np.float32)
+    return out
+
+
+def compare(out: dict, gold) -> dict:
+    rows = {}
+    for k in OUTPUT_NAMES:
+        a, b = np.asarray(out[k]), np.asarray(gold[k])
+        if b.dtype == np.uint8:
+            rows[k] = {"agreement": float((a == b).mean())}
+        else:
+            rows[k] = {"pcc": pcc(a.astype(np.float64), b.astype(np.float64)),
+                       "max_abs": float(np.abs(a.astype(np.float64) - b.astype(np.float64)).max())}
+    return rows
+
+
+def summary(res, frame: MeteorFrame, out: dict) -> dict:
+    return {"ego": np.asarray(out["ego"], np.float64).reshape(-1).tolist(),
+            "tl": np.asarray(out["tl"], np.float64).reshape(-1).tolist(),
+            "boxes3d": [b.to_dict() for b in sorted(res.boxes3d, key=lambda z: -z.score)],
+            "boxes2d": {cam: [b.to_dict() for b in bs] for cam, bs in zip(C.CAMERAS, res.boxes2d)},
+            "unknown": res.unknown, "plan_mode": int(res.plan["mode"]), "stationary_healthy": res.stationary_healthy,
+            "argmax_sha256": {k: sha(out[k]) for k in ("lane", "seg2d", "depth")},
+            "input_sha256": frame.sha256()}
+
+
+def make_variant(vid: str, threads: int) -> None:
+    """Goldens of one LOAD-knob variant (``VARIANTS``): the base frame with the absent cameras zeroed and given their
+    donor's calibration (``host.preprocess.assemble_frame``'s rule), run by the reference with the variant config."""
+    base, overrides, absent, tap_names, flag_present = VARIANTS[vid]
+    scene, idx = FRAMES[base][:2]
+    frame = load_scene_frame(scene_dir_of(scene) if (Path(scene) / "scenes.txt").is_file() else scene, idx)
+    imgs, K, T, present = frame.imgs.copy(), frame.K.copy(), frame.T_cam_ego.copy(), frame.present.copy()
+    for name in absent:
+        i, d = C.CAMERAS.index(name), C.CAMERAS.index(C.DONOR[name])
+        imgs[0, i] = 0
+        K[0, i], T[0, i] = K[0, d], T[0, d]
+        present[i] = False
+    if flag_present:
+        present[:] = True
+    frame = MeteorFrame(imgs, K, T, frame.v0, present, frame.pose, dict(frame.meta))
+    ref = MeteorReference(threads=threads, cfg=C.default_config(**overrides))
+    taps = TapRegistry(include=list(tap_names))
+    t1 = time.time()
+    ref.run_frame(frame, taps)
+    dt = time.time() - t1
+    geom = lift_geometry(frame.K, frame.T_cam_ego, points=ref.weights.lift_ground_points()[0])
+    arrays = store(taps.to_dict())
+    arrays.update({f"input.{k}": v for k, v in frame.feed().items()})
+    arrays["input.present"] = frame.present
+    meta = {"generator": "code/scripts/make_goldens.py --variants", "bundle_version": __version__, "frame": vid,
+            "base_frame": base, "absent": list(absent), "scene": str(scene), "index": idx,
+            "onnx_sha256": ref.weights.sha256, "licence": FRAMES[base][4], "input_norm": ref.cfg.input_norm,
+            "depth_mean_bins": ref.cfg.depth_mean_bins, "flag_present": bool(flag_present),
+            "taps": "gate" if tap_names is GATE_TAPS else "outputs", "seconds": round(dt, 1),
+            "lift": geom.stats()}
+    path = save_goldens(OUT / vid / "taps.npz", arrays, meta)
+    print(f"{vid}: {dt:.1f} s forward, {path.stat().st_size / 2 ** 20:.1f} MB", flush=True)
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--frames", nargs="*", default=list(FRAMES))
+    ap.add_argument("--variants", nargs="*", default=None, help="LOAD-knob variants (VARIANTS) instead of frames")
+    ap.add_argument("--threads", type=int, default=4)
+    a = ap.parse_args()
+    t0 = time.time()
+    if a.variants is not None:
+        for vid in a.variants or list(VARIANTS):
+            make_variant(vid, a.threads)
+        print(f"done in {time.time() - t0:.0f} s")
+        return
+    ref = MeteorReference(threads=a.threads)
+    report = json.loads((OUT / "report.json").read_text()) if (OUT / "report.json").is_file() else {}
+    for fid in a.frames:
+        scene, idx, golden, full, note = FRAMES[fid]
+        if not Path(scene).is_dir():
+            print(f"skip {fid}: {scene} missing")
+            continue
+        frame = load_scene_frame(scene_dir_of(scene) if (Path(scene) / "scenes.txt").is_file() else scene, idx)
+        taps = TapRegistry(include=["*"] if full else GATE_TAPS)
+        t1 = time.time()
+        out = ref.run_frame(frame, taps)
+        dt = time.time() - t1
+        geom = lift_geometry(frame.K, frame.T_cam_ego, points=ref.weights.lift_ground_points()[0])
+        arrays = store(taps.to_dict())
+        arrays.update({f"input.{k}": v for k, v in frame.feed().items()})
+        arrays["input.present"] = frame.present
+        arrays.update({"geom.b0": geom.b0.astype(np.uint8), "geom.fr": geom.fr})
+        rows = {}
+        if golden.is_file():
+            g = np.load(golden)
+            same_feed = all(np.array_equal(g[f"in_{k}"], v) for k, v in frame.feed().items())
+            rows = compare(out, g)
+            rows["_same_feed_as_ort_golden"] = bool(same_feed)
+        meta = {"generator": "code/scripts/make_goldens.py", "bundle_version": __version__, "frame": fid,
+                "scene": str(scene), "index": idx, "onnx_sha256": ref.weights.sha256, "licence": note,
+                "input_norm": ref.cfg.input_norm, "depth_mean_bins": ref.cfg.depth_mean_bins,
+                "taps": "all" if full else "gate", "ort_golden": str(golden), "seconds": round(dt, 1),
+                "lift": geom.stats()}
+        path = save_goldens(OUT / fid / "taps.npz", arrays, meta)
+        report[fid] = {"meta": meta, "vs_ort": rows, "size_mb": round(path.stat().st_size / 2 ** 20, 1)}
+        res = build_output(out, frame, PostConfig(), None)
+        report[fid]["decode"] = {"boxes3d": len(res.boxes3d), "boxes2d": [len(b) for b in res.boxes2d],
+                                 "unknown": len(res.unknown), "mode": int(res.plan["mode"]),
+                                 "tl": res.traffic_light["state"]}
+        worst = min((r.get("pcc", r.get("agreement", 1.0)) for k, r in rows.items() if isinstance(r, dict)),
+                    default=float("nan"))
+        print(f"{fid}: {dt:.1f} s forward, {report[fid]['size_mb']} MB, worst vs ORT {worst:.10f}, "
+              f"decode {report[fid]['decode']}", flush=True)
+        if fid == "pandaset_019_f40":
+            small = summary(res, frame, out)
+            small["_doc"] = ("Decoded fp32 CPU reference (tt_meteor.reference, PCC 1.0 vs ONNX Runtime) on the shipped "
+                             "sample PandaSet 019 frame 40, stateless host decode (PostConfig defaults); "
+                             "code/scripts/make_goldens.py.")
+            (_ROOT / "tests/goldens/pandaset_019_f40_reference.json").write_text(
+                json.dumps(small, indent=1, default=float) + "\n")
+    (OUT / "report.json").write_text(json.dumps(report, indent=1, default=float) + "\n")
+    if "pandaset_019_f40" in a.frames:
+        # the /predict body of the reference on the shipped sample request (fresh stream, as the smoke test sends it)
+        request = load_sample(SHIP.parent / "pandaset_019_f40.json")
+        body = MeteorReference(weights=ref.weights, threads=a.threads)(**request)
+        d = body.to_dict()
+        d["timing_ms"] = {}
+        (SHIP.parent / "pandaset_019_f40.reference.json").write_text(json.dumps(d) + "\n")
+        print("wrote samples/pandaset_019_f40.reference.json:", d["num_detections"], "detections, mode",
+              d["plan"]["mode"])
+    print(f"done in {time.time() - t0:.0f} s")
+
+
+if __name__ == "__main__":
+    main()

code/scripts/make_sample.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Build the PandaSet sample of meteor-p150 from the research ship sample (PandaSet 019 frame 40, CC BY 4.0 + the
+PandaSet Dataset Terms; research/meteor/public_data/ship_sample/pandaset_019_f40, made by its make_ship_sample.py).
+It does NOT ship until the user approves PandaSet-derived samples (PLAN.md 6.3, research/DATASETS.md N5): by default
+(``--dest staging``) it is written to the git-ignored ``staging_samples_pandaset/`` (laid out like ``code/tt_meteor/``,
+where the tests find it, ``tests/paths.py``); ``--dest package`` writes it into ``code/tt_meteor/`` (the swap-in of
+``staging_samples_pandaset/README.md``). The shipped sample is the synthetic ``samples/synthetic_8cam.json``
+(``code/scripts/make_synthetic_sample.py``). Paths below are relative to the destination:
+
+- ``samples/pandaset_019_f40/``: the METEOR demo-scene layout (scenes.txt, manifest.json, the 8 camera
+  JPEGs incl. the all-zero absent BACK_NARROW, ego_motion.npz, gt_boxes.json), LICENSE.txt, README.md, SHA256SUMS;
+- ``calib/pandaset_019.json``: the rig as a calibration preset (K at 768x432, T_ref_from_camera =
+  camera optical frame -> ego; BACK_NARROW carries its donor's calibration, as the manifest does);
+- ``samples/pandaset_019_f40.json``: the request manifest (``model(**load_sample(path))``).
+
+Usage: python code/scripts/make_sample.py [--src <ship sample dir>] [--dest staging|package]   (numpy; no device)
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import shutil
+from pathlib import Path
+
+import numpy as np
+
+BUNDLE = Path(__file__).resolve().parents[2]
+PKG = BUNDLE / "code" / "tt_meteor"
+SRC = Path("/home/ubuntu/experiments/tt-models/research/meteor/public_data/ship_sample/pandaset_019_f40")
+CAMERAS = ("CAM_FRONT_WIDE", "CAM_FRONT_LEFT", "CAM_FRONT_RIGHT", "CAM_BACK_WIDE", "CAM_BACK_LEFT", "CAM_BACK_RIGHT",
+           "CAM_FRONT_NARROW", "CAM_BACK_NARROW")
+SCENE = "pandaset_019_f40"
+
+
+def sha256(p: Path) -> str:
+    return hashlib.sha256(p.read_bytes()).hexdigest()
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--src", type=Path, default=SRC)
+    ap.add_argument("--dest", choices=("staging", "package"), default="staging")
+    a = ap.parse_args()
+    root = PKG if a.dest == "package" else BUNDLE / "staging_samples_pandaset"
+    dst = root / "samples" / SCENE
+    if dst.exists():
+        shutil.rmtree(dst)
+    (dst / SCENE / "img").mkdir(parents=True)
+    imgs = [f"{SCENE}/img/{p.name}" for p in sorted((a.src / SCENE / "img").iterdir())]
+    for rel in ["scenes.txt", "LICENSE.txt", f"{SCENE}/manifest.json", f"{SCENE}/ego_motion.npz",
+                f"{SCENE}/gt_boxes.json"] + imgs:
+        shutil.copyfile(a.src / rel, dst / rel)
+    m = json.loads((a.src / SCENE / "manifest.json").read_text())
+    ego = np.load(a.src / SCENE / "ego_motion.npz")
+    readme = (a.src / "README.md").read_text()
+    readme = readme.replace("## Run\n", "## Run (this bundle)\n\n```python\nfrom tt_meteor import METEOR, load_sample\n"
+                            "kwargs = load_sample('code/tt_meteor/samples/pandaset_019_f40.json')\n"
+                            "```\n\nThe CPU reference of the bundle (`tt_meteor.reference.pipeline.MeteorReference`) "
+                            "and the research tools read the scene layout directly:\n\n", 1)
+    readme = readme.replace("## Expected outputs (`expected/`)", "## Expected outputs (research `expected/`, not "
+                            "copied here; the bundle's own reference output is "
+                            "`../pandaset_019_f40.reference.json`)", 1)
+    (dst / "README.md").write_text(readme)
+    files = sorted(p for p in dst.rglob("*") if p.is_file())
+    (dst / "SHA256SUMS").write_text("".join(f"{sha256(p)}  {p.relative_to(dst).as_posix()}\n" for p in files))
+
+    cams = {}
+    for c in CAMERAS:
+        cc = m["cams"][c]
+        cams[c] = {"intrinsics": cc["K"], "T_ref_from_camera": cc["T_ego_cam"], "image_size": [768, 432]}
+        if not cc.get("present", True):
+            cams[c]["absent"] = f"zero image + donor {cc.get('donor')} calibration (bevlane/dataset.py:131-145)"
+    preset = {"frame_id": "base_link",
+              "_source": f"research/meteor/public_data/ship_sample/{SCENE}/{SCENE}/manifest.json (PandaSet 019, "
+                         "CC BY 4.0 + PandaSet Dataset Terms; rig derivation: research/meteor/public_data/README.md)",
+              "_ego_frame": m.get("ego_frame"),
+              "cameras": cams}
+    (root / "calib").mkdir(parents=True, exist_ok=True)
+    (root / "calib" / "pandaset_019.json").write_text(json.dumps(preset, indent=1) + "\n")
+    fr = m["frames"][0]
+    req = {"images": {c: f"{SCENE}/{SCENE}/{fr['imgs'][c]}" for c in CAMERAS},
+           "calibration": {"preset": "pandaset_019"},
+           "ego_speed": float(ego["v0"][0]),
+           "stream": {"id": "pandaset_019", "T_world_from_ego": dict(zip(("x", "y", "yaw"), (float(v) for v in
+                                                                                    ego["pose"][0])),
+                                                                        z=0.0, roll=0.0, pitch=0.0)},
+           "_source": "PandaSet 019 frame 40 (Scale AI and Hesai, https://pandaset.org), CC BY 4.0 + PandaSet Dataset "
+                      f"Terms ({SCENE}/LICENSE.txt); METEOR 8-slot layout, BACK_NARROW absent (all-zero image)"}
+    (root / "samples" / f"{SCENE}.json").write_text(json.dumps(req, indent=1) + "\n")
+    print(f"wrote {dst} ({sum(p.stat().st_size for p in files) / 1e6:.2f} MB), calib/pandaset_019.json, "
+          f"samples/{SCENE}.json")
+
+
+if __name__ == "__main__":
+    main()

code/scripts/make_synthetic_sample.py

@@ -0,0 +1,508 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""The shipped sample of meteor-p150: a synthetic eight-camera frame rendered here (data generated by this repository,
+Apache-2.0; no third-party pixels or calibration). CPU only, research venv (torch, OpenCV)::
+
+    cd bundles/meteor-p150; PY=/home/ubuntu/experiments/tt-models/tools/research-venv/bin/python
+    OMP_NUM_THREADS=6 PYTHONPATH=$PWD/code $PY code/scripts/make_synthetic_sample.py render     # ~3 min
+    OMP_NUM_THREADS=6 PYTHONPATH=$PWD/code $PY code/scripts/make_synthetic_sample.py optimise   # ~15 s per step
+    OMP_NUM_THREADS=6 PYTHONPATH=$PWD/code $PY code/scripts/make_synthetic_sample.py finalise   # PNGs + reference
+
+(``--work`` holds the intermediate arrays, default ``logs/synthetic_sample/``; ``optimise`` resumes from its
+``images_f32.npy``.) The shipped sample came from ``render``, ``optimise --steps 80`` (stopped after 28 steps to drop
+the pedestrian beyond 20 m from the targets), ``optimise --steps 60 --lr 0.015`` and ``finalise``.
+
+A generic METEOR-like rig (``calib/synthetic_8cam.json``: the eight slots in METEOR's order, a 98 deg wide camera
+front and back, four corner cameras pitched 25 deg down, two 30.4 deg narrow cameras; fy = 0.87 fx like METEOR's
+vertically squashed training images; mounted 1.9 m above the road) looks at a procedural street scene that is
+ray-cast per pixel (2x2 supersampled, INTER_AREA down to 768x432): a straight two-lane road with lane lines and a
+stop line, kerbs, pavements, building facades, box-shaped vehicles and pedestrians. All eight cameras are present.
+Plain renders give the network too little to detect (the strongest 3D heatmap cell was 0.22 < the 0.35 vehicle
+threshold), so ``optimise`` then changes the object pixels only (gradient ascent through the fp32 CPU reference: image
+encoder, lift, BEV detector, box refiner; the image rounded to uint8 by a straight-through estimator) until every
+target object within range (:data:`DROP_FAR`) clears :data:`GOAL` at its cell and nothing else clears :data:`CEIL`:
+a stable smoke reference whose published boxes sit far from the thresholds.
+
+Writes:
+
+- ``code/tt_meteor/calib/synthetic_8cam.json``: the rig as a calibration preset (served in ``/info``);
+- ``code/tt_meteor/samples/synthetic_8cam/<CAM>.png``: the eight 768x432 RGB images (lossless, so every decoder
+  gives the same pixels);
+- ``code/tt_meteor/samples/synthetic_8cam.json``: the request manifest (``model(**load_sample(path))``);
+- ``code/tt_meteor/samples/synthetic_8cam.reference.json``: the ``/predict`` body of the fp32 CPU reference
+  (``tt_meteor.reference.pipeline.MeteorReference``, PCC 1.0 vs ONNX Runtime) on that request as a fresh stream,
+  the container smoke's stored reference (``--no-reference`` skips it: 40-60 s of CPU);
+- ``code/tt_meteor/tests/goldens/synthetic_8cam_inputs.json``: sha256 of the graph feed (``imgs``, ``K``,
+  ``T_cam_ego``, ``v0``) the request produces, so a host test notices any change of the decoding / resize path.
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import math
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+BUNDLE = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(BUNDLE / "code"))
+PKG = BUNDLE / "code" / "tt_meteor"
+NAME = "synthetic_8cam"
+W, H, SS = 768, 432, 2                  # network input size, supersampling factor
+EGO_SPEED = 8.0                         # m/s
+SEED = 20261010
+
+# slot -> (x, y, z of the camera in base_link, yaw deg CCW from +x, pitch deg down, horizontal FOV deg)
+RIG = {
+    "CAM_FRONT_WIDE": (2.10, 0.00, 1.90, 0.0, 2.0, 98.0),
+    "CAM_FRONT_LEFT": (1.90, 0.85, 1.90, 60.0, 25.0, 98.0),
+    "CAM_FRONT_RIGHT": (1.90, -0.85, 1.90, -60.0, 25.0, 98.0),
+    "CAM_BACK_WIDE": (-0.90, 0.00, 1.90, 180.0, 2.0, 98.0),
+    "CAM_BACK_LEFT": (-0.70, 0.85, 1.90, 120.0, 25.0, 98.0),
+    "CAM_BACK_RIGHT": (-0.70, -0.85, 1.90, -120.0, 25.0, 98.0),
+    "CAM_FRONT_NARROW": (2.10, 0.00, 1.88, 0.0, 1.0, 30.4),
+    "CAM_BACK_NARROW": (-0.90, 0.00, 1.88, 180.0, 2.0, 30.4),
+}
+FY_OVER_FX = 0.87                       # METEOR's training images are squashed vertically (2880x1860 -> 768x432)
+
+# the street (base_link: x forward, y left, z up, origin on the road below the rear axle; left-hand traffic, ego in
+# the left lane): lane edges and lines (y, half width, kind)
+ROAD_L, ROAD_R = 1.75, -5.25            # carriageway edges (ego lane 1.75 .. -1.75, oncoming lane -1.75 .. -5.25)
+KERB = 0.15                             # kerb height
+WALK_L, WALK_R = 5.0, -8.5              # pavement outer edges = building facades
+# boxes: (x, y, yaw deg, length, width, height, rgb, kind)
+OBJECTS = [
+    (11.0, 0.0, 0.0, 4.4, 1.8, 1.5, (180, 30, 35), "car"),          # car ahead in the ego lane
+    (19.0, -3.5, 180.0, 4.6, 1.85, 1.55, (40, 70, 160), "car"),     # oncoming cars
+    (29.0, -3.4, 180.0, 4.5, 1.8, 1.5, (150, 150, 155), "car"),
+    (-7.0, -3.5, 180.0, 4.4, 1.8, 1.5, (200, 170, 40), "car"),
+    (42.0, 0.1, 0.0, 8.5, 2.4, 3.0, (225, 225, 220), "truck"),      # truck further ahead
+    (-13.0, 0.0, 0.0, 4.3, 1.75, 1.45, (220, 220, 225), "car"),     # car behind
+    (6.0, 3.2, 0.0, 4.2, 1.75, 1.45, (60, 60, 65), "car"),          # parked car on the left (on the pavement edge)
+    (-4.0, -6.8, 90.0, 0.5, 0.6, 1.7, (40, 40, 90), "ped"),         # pedestrian on the right pavement
+    (9.0, 4.0, 0.0, 0.5, 0.6, 1.75, (120, 40, 40), "ped"),          # pedestrian on the left pavement
+    (22.0, -7.2, 0.0, 0.5, 0.6, 1.65, (30, 90, 60), "ped"),
+]
+STOP_LINE_X = 34.0                      # stop line across the ego lane
+CROSSWALK_X = (36.0, 40.0)              # zebra across the road
+
+
+def object_parts():
+    """Every object as oriented boxes: (centre x, y, z, yaw deg, half extents (l, w, h), rgb, material)."""
+    out = []
+    for (bx, by, yaw, L, Wd, Hh, col, kind) in OBJECTS:
+        z0 = KERB if (by > ROAD_L or by < ROAD_R) else 0.0                 # on the pavement or on the road
+        c, s = math.cos(math.radians(yaw)), math.sin(math.radians(yaw))
+
+        def at(dx, dy, dz, half, rgb, mat, yaw_=yaw):
+            out.append((bx + c * dx - s * dy, by + s * dx + c * dy, z0 + dz, yaw_, half, rgb, mat))
+
+        if kind == "car":
+            hb = 0.55 * Hh                                                   # lower body 0.25 .. 0.25 + hb
+            at(0, 0, 0.25 + hb / 2, (L / 2, Wd / 2, hb / 2), col, "paint")
+            at(-0.15 * L, 0, 0.25 + hb + (Hh - 0.25 - hb) / 2, (0.28 * L, 0.44 * Wd, (Hh - 0.25 - hb) / 2),
+               (55, 65, 80), "glass")                                        # cabin
+            at(-0.15 * L, 0, Hh - 0.02, (0.26 * L, 0.42 * Wd, 0.03), col, "paint")   # roof
+            for sx in (0.32, -0.32):
+                for sy in (1, -1):
+                    at(sx * L, sy * (Wd / 2 - 0.12), 0.32, (0.32, 0.13, 0.32), (22, 22, 24), "rubber")
+            for sy in (1, -1):                                               # lamps
+                at(L / 2 - 0.02, sy * 0.33 * Wd, 0.25 + 0.7 * hb, (0.04, 0.16, 0.06), (240, 240, 215), "lamp")
+                at(-L / 2 + 0.02, sy * 0.33 * Wd, 0.25 + 0.7 * hb, (0.04, 0.16, 0.06), (210, 25, 25), "lamp")
+        elif kind == "truck":
+            cab = 1.9
+            at(L / 2 - cab / 2, 0, 0.35 + (Hh - 0.6 - 0.35) / 2, (cab / 2, Wd / 2, (Hh - 0.6 - 0.35) / 2), (40, 90, 160), "paint")
+            at(L / 2 - 0.05, 0, Hh - 1.3, (0.06, 0.45 * Wd, 0.35), (55, 65, 80), "glass")
+            at(-cab / 2, 0, 0.6 + (Hh - 0.6) / 2, ((L - cab) / 2 - 0.05, Wd / 2, (Hh - 0.6) / 2), col, "paint")
+            for sx in (0.38, 0.0, -0.33):
+                for sy in (1, -1):
+                    at(sx * L, sy * (Wd / 2 - 0.15), 0.45, (0.45, 0.15, 0.45), (22, 22, 24), "rubber")
+        else:                                                                # pedestrian
+            at(0, 0, 0.42, (0.12, 0.2, 0.42), (40, 40, 55), "cloth")
+            at(0, 0, 0.84 + (Hh - 1.07) / 2, (0.14, 0.24, (Hh - 1.07) / 2), col, "cloth")
+            at(0, 0, Hh - 0.115, (0.1, 0.09, 0.115), (215, 170, 140), "skin")
+    return out
+
+
+def shadow(px: np.ndarray, py: np.ndarray) -> np.ndarray:
+    """Soft contact shadows of the objects on the ground: a multiplicative factor in (0, 1]."""
+    f = np.ones(px.shape, np.float32)
+    for (bx, by, yaw, L, Wd, Hh, col, kind) in OBJECTS:
+        c, s = math.cos(math.radians(yaw)), math.sin(math.radians(yaw))
+        dx, dy = px - bx, py - by
+        lx, ly = np.abs(c * dx + s * dy) - L / 2, np.abs(-s * dx + c * dy) - Wd / 2
+        out = np.hypot(np.maximum(lx, 0), np.maximum(ly, 0)) + np.minimum(np.maximum(lx, ly), 0)
+        f *= (1.0 - 0.75 * np.clip(1.0 - (out + 0.15) / 0.6, 0, 1)).astype(np.float32)
+    return f
+
+
+def rotation(yaw_deg: float, pitch_deg: float) -> np.ndarray:
+    """camera optical frame (x right, y down, z forward) -> base_link rotation."""
+    p, y = math.radians(pitch_deg), math.radians(yaw_deg)
+    f = np.array([math.cos(p) * math.cos(y), math.cos(p) * math.sin(y), -math.sin(p)])
+    r = np.array([math.sin(y), -math.cos(y), 0.0])
+    d = np.cross(f, r)
+    return np.stack([r, d, f], axis=1)
+
+
+def intrinsics(hfov_deg: float, w: int = W, h: int = H) -> np.ndarray:
+    fx = (w / 2.0) / math.tan(math.radians(hfov_deg) / 2.0)
+    return np.array([[fx, 0.0, w / 2.0], [0.0, FY_OVER_FX * fx, h / 2.0], [0.0, 0.0, 1.0]])
+
+
+def value_noise(x: np.ndarray, y: np.ndarray, cell: float, seed: int) -> np.ndarray:
+    """Smooth hash noise in [0, 1) on a ground-plane lattice of ``cell`` metres."""
+    gx, gy = np.floor(x / cell), np.floor(y / cell)
+    fx, fy = x / cell - gx, y / cell - gy
+
+    def h(ix, iy):
+        v = (ix.astype(np.int64) * 73856093) ^ (iy.astype(np.int64) * 19349663) ^ seed
+        v = (v * 2654435761) & 0xFFFFFFFF
+        return (v % 10007) / 10007.0
+
+    sx, sy = fx * fx * (3 - 2 * fx), fy * fy * (3 - 2 * fy)
+    a, b, c, d = h(gx, gy), h(gx + 1, gy), h(gx, gy + 1), h(gx + 1, gy + 1)
+    return (a * (1 - sx) + b * sx) * (1 - sy) + (c * (1 - sx) + d * sx) * sy
+
+
+def ground_colour(px: np.ndarray, py: np.ndarray, dist: np.ndarray) -> np.ndarray:
+    n = 0.6 * value_noise(px, py, 0.35, SEED) + 0.4 * value_noise(px, py, 2.0, SEED + 1)
+    rgb = np.empty(px.shape + (3,), np.float32)
+    road = (py <= ROAD_L) & (py >= ROAD_R)
+    asphalt = 78 + 34 * n
+    rgb[...] = np.stack([asphalt, asphalt, asphalt + 4], -1)[...]
+    walk = ~road
+    tile = 150 + 30 * n + 12 * ((np.floor(px / 0.6) + np.floor(py / 0.6)) % 2)
+    rgb[walk] = np.stack([tile, tile - 6, tile - 14], -1)[walk]
+    # markings on the carriageway: edge lines (solid), centre line (dashed 5 m / 5 m), stop line, zebra
+    white = np.zeros(px.shape, bool)
+    white |= road & (np.abs(py - (ROAD_L - 0.2)) < 0.075)
+    white |= road & (np.abs(py - (ROAD_R + 0.2)) < 0.075)
+    white |= (np.abs(py + 1.75) < 0.075) & (np.mod(px, 10.0) < 5.0)
+    white |= (py <= 1.6) & (py >= -1.75) & (np.abs(px - STOP_LINE_X) < 0.225)
+    zebra = road & (px >= CROSSWALK_X[0]) & (px <= CROSSWALK_X[1]) & (np.mod(py - ROAD_R, 0.9) < 0.45)
+    white |= zebra
+    wv = 215 + 25 * n
+    rgb[white] = np.stack([wv, wv, wv - 5], -1)[white]
+    rgb *= shadow(px, py)[..., None]
+    # haze with distance
+    fog = np.clip(dist / 120.0, 0, 0.6)[..., None]
+    return rgb * (1 - fog) + np.array([170, 180, 195], np.float32) * fog
+
+
+def render_camera(R: np.ndarray, t: np.ndarray, K: np.ndarray):
+    """Ray-cast the street for one camera; returns (uint8 (H*SS, W*SS, 3), bool object mask (H*SS, W*SS))."""
+    w, h = W * SS, H * SS
+    Ks = K.copy()
+    Ks[:2] *= SS
+    u, v = np.meshgrid(np.arange(w) + 0.5, np.arange(h) + 0.5)
+    rays_c = np.stack([(u - Ks[0, 2]) / Ks[0, 0], (v - Ks[1, 2]) / Ks[1, 1], np.ones_like(u)], -1)
+    d = rays_c @ R.T
+    d /= np.linalg.norm(d, axis=-1, keepdims=True)
+    best = np.full(d.shape[:2], np.inf)
+    rgb = np.zeros(d.shape, np.float32)
+    obj = np.zeros(d.shape[:2], bool)
+    # sky
+    up = np.clip(d[..., 2], -1, 1)
+    sky = np.stack([120 + 60 * (1 - up), 160 + 50 * (1 - up), 225 + 20 * (1 - up)], -1)
+    rgb[...] = sky
+    # ground (z = 0; pavements at kerb height)
+    with np.errstate(divide="ignore", invalid="ignore"):
+        for z0, keep in ((0.0, lambda py: (py <= ROAD_L) & (py >= ROAD_R)),
+                         (KERB, lambda py: (py > ROAD_L) | (py < ROAD_R))):
+            tg = (z0 - t[2]) / d[..., 2]
+            ok = (tg > 0) & np.isfinite(tg)
+            px, py = t[0] + tg * d[..., 0], t[1] + tg * d[..., 1]
+            ok &= keep(py) & (py <= WALK_L) & (py >= WALK_R)
+            ok &= tg < best
+            col = ground_colour(px, py, tg)
+            rgb[ok], best[ok] = col[ok], tg[ok]
+        # kerb faces (vertical planes y = ROAD_L / ROAD_R, 0 .. KERB)
+        for yk in (ROAD_L, ROAD_R):
+            tk = (yk - t[1]) / d[..., 1]
+            zk = t[2] + tk * d[..., 2]
+            ok = (tk > 0) & (zk >= 0) & (zk <= KERB) & (tk < best)
+            rgb[ok], best[ok] = np.array([185, 180, 170], np.float32), tk[ok]
+        # facades (y = WALK_L / WALK_R), 6-14 m tall blocks with windows
+        for yw, sd in ((WALK_L, 3), (WALK_R, 4)):
+            tw = (yw - t[1]) / d[..., 1]
+            xw, zw = t[0] + tw * d[..., 0], t[2] + tw * d[..., 2]
+            block = np.floor(xw / 12.0)
+            hb = 6 + 8 * value_noise(block * 12.0, np.zeros_like(block), 12.0, SEED + sd)
+            ok = (tw > 0) & (zw >= KERB) & (zw <= hb) & (tw < best)
+            base = 0.55 + 0.45 * value_noise(block * 12.0, np.full_like(block, 5.0), 12.0, SEED + 10 + sd)
+            wall = np.stack([200 * base, 170 * base, 140 * base], -1)
+            win = (np.mod(xw, 3.0) > 0.9) & (np.mod(zw - 0.8, 3.2) < 1.6) & (zw > 3.0)
+            wall[win] = np.array([70, 90, 115], np.float32)
+            door = (zw < 2.4) & (np.mod(xw, 12.0) > 5.0) & (np.mod(xw, 12.0) < 6.6)
+            wall[door] = np.array([60, 45, 35], np.float32)
+            fog = np.clip(tw / 120.0, 0, 0.6)[..., None]
+            wall = wall * (1 - fog) + np.array([170, 180, 195], np.float32) * fog
+            rgb[ok], best[ok] = wall[ok], tw[ok]
+    # objects: every part is an oriented box (slab test in the part frame)
+    light = np.array([0.4, 0.3, 0.87])
+    light /= np.linalg.norm(light)
+    for (cx, cy, cz, yaw, half, col, mat) in object_parts():
+        c, s = math.cos(math.radians(yaw)), math.sin(math.radians(yaw))
+        Rb = np.array([[c, -s, 0], [s, c, 0], [0, 0, 1.0]])
+        o = Rb.T @ (t - np.array([cx, cy, cz]))
+        dl = d @ Rb
+        half = np.asarray(half, np.float64)
+        with np.errstate(divide="ignore", invalid="ignore"):
+            t1 = (-half - o) / dl
+            t2 = (half - o) / dl
+        tmin = np.nanmax(np.minimum(t1, t2), axis=-1)
+        tmax = np.nanmin(np.maximum(t1, t2), axis=-1)
+        hit = (tmax >= tmin) & (tmin > 0) & (tmin < best)
+        if not hit.any():
+            continue
+        p = o + tmin[..., None] * dl                                     # hit point, part frame
+        ax = np.argmax(np.abs(p) / half, axis=-1)
+        nrm_l = np.zeros_like(p)
+        np.put_along_axis(nrm_l, ax[..., None], np.sign(np.take_along_axis(p, ax[..., None], -1)), -1)
+        nrm = nrm_l @ Rb.T
+        lam = np.clip(nrm @ light, 0, 1)
+        base = np.broadcast_to(np.array(col, np.float32), p.shape).copy()
+        if mat == "glass":                                                # sky reflection on upward-tilted views
+            refl = np.clip(0.35 + 0.65 * np.abs(nrm[..., 2]), 0, 1)
+            shade = 0.5 + 0.3 * refl
+        elif mat == "paint":
+            shade = 0.35 + 0.65 * lam + 0.08 * (value_noise(p[..., 0], p[..., 2], 0.4, SEED + 7) - 0.5)
+        else:
+            shade = 0.4 + 0.6 * lam
+        cshade = base * shade[..., None]
+        rgb[hit], best[hit] = cshade[hit], tmin[hit]
+        obj |= hit
+    obj &= np.isfinite(best)
+    return np.clip(rgb, 0, 255).astype(np.uint8), obj
+
+
+def build_rig():
+    cams = {}
+    for name, (x, y, z, yaw, pitch, fov) in RIG.items():
+        R = rotation(yaw, pitch)
+        T = np.eye(4)
+        T[:3, :3], T[:3, 3] = R, (x, y, z)
+        cams[name] = {"intrinsics": intrinsics(fov).round(4).tolist(), "T_ref_from_camera": T.round(9).tolist(),
+                      "image_size": [W, H]}
+    return cams
+
+
+def hm_cell(x: float, y: float):
+    """base_link (x, y) -> (row, col) of the 400x250 detection grid (rows x = 79.8 .. -79.8, cols y = 49.8 .. -49.8)."""
+    return int(round((79.8 - x) / 0.4)), int(round((49.8 - y) / 0.4))
+
+
+# what the optimisation asks of the 3D heads: every OBJECTS entry is a target (vehicle: car / truck, VRU: ped)
+GOAL = {0: 0.70, 1: 0.50}               # sigmoid at the target cell (thresholds: vehicle 0.35, VRU 0.15)
+CEIL = {0: 0.20, 1: 0.07}               # sigmoid ceiling everywhere else (outside 1.6 m of a target of that class)
+DROP_FAR = {0: 40.0, 1: 20.0}           # targets beyond this range are scenery (pushed below the ceiling instead)
+
+
+def targets():
+    out = []
+    for (bx, by, yaw, L, Wd, Hh, col, kind) in OBJECTS:
+        cls = 0 if kind in ("car", "truck") else 1
+        if math.hypot(bx, by) <= DROP_FAR[cls]:
+            out.append((cls, bx, by))
+    return out
+
+
+def differentiable_hm(net, x, K, T):
+    """The 3D heatmap logits [1, 2, 400, 250] of the fp32 reference from float images x [8, 3, 432, 768] in [0, 1]
+    (the graph's /255 input normalisation; the same modules as ``MeteorNet.forward``, with gradients)."""
+    f = net.image_encoder(x)
+    seg = net.seg2d_head(f)
+    _, dprob = net.depth_head(f)
+    ctx = net.context(f, seg)
+    raw = net.lift(ctx, dprob, K, T)
+    _, hm_pre, reg_pre = net.det_stem(raw)
+    hm, _ = net.box_refiner(hm_pre, reg_pre)
+    return hm
+
+
+def stage_render(work: Path) -> None:
+    import cv2
+
+    t0 = time.time()
+    cams = build_rig()
+    imgs, masks = [], []
+    for name in RIG:
+        c = cams[name]
+        T = np.asarray(c["T_ref_from_camera"])
+        img, obj = render_camera(T[:3, :3], T[:3, 3], np.asarray(c["intrinsics"]))
+        imgs.append(cv2.resize(img, (W, H), interpolation=cv2.INTER_AREA))
+        m = cv2.resize(obj.astype(np.uint8) * 255, (W, H), interpolation=cv2.INTER_AREA) > 0
+        masks.append(cv2.dilate(m.astype(np.uint8), np.ones((9, 9), np.uint8)) > 0)
+    work.mkdir(parents=True, exist_ok=True)
+    np.save(work / "base_u8.npy", np.stack(imgs))                      # [8, H, W, 3] RGB
+    np.save(work / "mask.npy", np.stack(masks))                        # [8, H, W]
+    print(f"render: {time.time() - t0:.0f} s, object pixels {np.stack(masks).mean():.3%}", flush=True)
+
+
+def _load_net(weights_dir, threads):
+    import torch
+
+    from tt_meteor.reference.pipeline import MeteorReference
+    from tt_meteor.reference.weights import MeteorWeights
+
+    torch.set_num_threads(threads)
+    return MeteorReference(weights=MeteorWeights(weights_dir), threads=threads)   # plain tensors: no weight grads
+
+
+def _calib_tensors():
+    import torch
+
+    from tt_meteor.host.preprocess import invert_extrinsics
+
+    cams = build_rig()
+    K = np.stack([np.asarray(cams[n]["intrinsics"], np.float32) for n in RIG])[None]
+    T = np.stack([invert_extrinsics(cams[n]["T_ref_from_camera"]) for n in RIG])[None]
+    return torch.from_numpy(K), torch.from_numpy(T)
+
+
+def stage_optimise(work: Path, weights_dir, threads: int, steps: int, lr: float) -> None:
+    """Gradient ascent on the object pixels (and only those) until every target clears its goal on the uint8-rounded
+    images and nothing else clears its ceiling."""
+    import torch
+
+    ref = _load_net(weights_dir, threads)
+    K, T = _calib_tensors()
+    base = torch.from_numpy(np.load(work / "base_u8.npy").astype(np.float32) / 255.0).permute(0, 3, 1, 2)
+    mask = torch.from_numpy(np.load(work / "mask.npy")).float()[:, None]
+    start = work / "images_f32.npy"
+    x0 = torch.from_numpy(np.load(start)) if start.is_file() else base.clone()
+    delta = (x0 - base).clone().requires_grad_(True)
+    opt = torch.optim.Adam([delta], lr=lr)
+    tg = targets()
+    logit = lambda p: math.log(p / (1 - p))
+    keep = torch.ones(1, 2, 400, 250, dtype=torch.bool)
+    rr, cc = torch.meshgrid(torch.arange(400), torch.arange(250), indexing="ij")
+    for cls, bx, by in tg:
+        r, c = hm_cell(bx, by)
+        keep[0, cls] &= (rr - r) ** 2 + (cc - c) ** 2 > 16              # 1.6 m around a target of its class
+    ceil = torch.tensor([logit(CEIL[0]), logit(CEIL[1])]).reshape(1, 2, 1, 1)
+    log = open(work / "optimise.log", "a")
+    for step in range(steps):
+        t0 = time.time()
+        x = (base + delta * mask).clamp(0, 1)
+        xq = x + ((x * 255).round() / 255 - x).detach()                  # straight-through uint8 rounding
+        hm = differentiable_hm(ref.net, xq, K, T)
+        z = []
+        loss = torch.zeros(())
+        for cls, bx, by in tg:
+            r, c = hm_cell(bx, by)
+            zt = hm[0, cls, r - 1:r + 2, c - 1:c + 2].max()             # the 3x3 peak at the target
+            z.append(float(torch.sigmoid(zt)))
+            loss = loss + torch.nn.functional.softplus(logit(GOAL[cls]) + 0.3 - zt) * 2.0
+        over = torch.relu(hm - ceil)[keep]
+        loss = loss + (over ** 2).sum() * 0.5
+        opt.zero_grad()
+        loss.backward()
+        opt.step()
+        with torch.no_grad():
+            delta.clamp_(-0.6, 0.6)
+        n_over = int((over > 0).sum())
+        ok = all(p >= GOAL[cls] for p, (cls, _, _) in zip(z, tg)) and n_over == 0
+        line = (f"step {step}: loss {float(loss):.4f}, targets {[round(p, 3) for p in z]}, cells over the ceiling "
+                f"{n_over}, max other {float(torch.sigmoid(hm.detach()[keep]).max()):.3f}, {time.time() - t0:.0f} s")
+        print(line, flush=True)
+        log.write(line + "\n")
+        log.flush()
+        np.save(work / "images_f32.npy", x.detach().numpy())
+        if ok:
+            print("all targets met", flush=True)
+            break
+
+
+def stage_finalise(work: Path, weights_dir, threads: int, reference: bool) -> None:
+    import cv2
+
+    t0 = time.time()
+    cams = build_rig()
+    preset = {"frame_id": "base_link",
+              "_source": "generated by code/scripts/make_synthetic_sample.py (a generic METEOR-like 8-camera rig of "
+                         "round numbers; no third-party data), Apache-2.0",
+              "_rig": {k: dict(zip(("x", "y", "z", "yaw_deg", "pitch_down_deg", "hfov_deg"), v)) for k, v in RIG.items()},
+              "cameras": cams}
+    (PKG / "calib" / f"{NAME}.json").write_text(json.dumps(preset, indent=1) + "\n")
+    src = work / "images_f32.npy"
+    if src.is_file():
+        imgs = np.round(np.load(src) * 255).clip(0, 255).astype(np.uint8).transpose(0, 2, 3, 1)
+    else:
+        imgs = np.load(work / "base_u8.npy")
+    out = PKG / "samples" / NAME
+    out.mkdir(parents=True, exist_ok=True)
+    for name, img in zip(RIG, imgs):
+        cv2.imwrite(str(out / f"{name}.png"), np.ascontiguousarray(img[:, :, ::-1]), [cv2.IMWRITE_PNG_COMPRESSION, 9])
+    req = {"images": {name: f"{NAME}/{name}.png" for name in RIG},
+           "calibration": {"preset": NAME},
+           "ego_speed": EGO_SPEED,
+           "stream": {"id": NAME, "reset": True, "T_world_from_ego": {"x": 0.0, "y": 0.0, "z": 0.0, "roll": 0.0,
+                                                                        "pitch": 0.0, "yaw": 0.0}},
+           "_source": "synthetic street scene rendered and optimised by code/scripts/make_synthetic_sample.py (data "
+                      "generated by this repository, Apache-2.0); calibration preset calib/synthetic_8cam.json; all "
+                      "eight cameras present"}
+    (PKG / "samples" / f"{NAME}.json").write_text(json.dumps(req, indent=1) + "\n")
+    size = sum(p.stat().st_size for p in out.iterdir()) / 1e6
+    print(f"wrote samples/{NAME}/ ({size:.2f} MB), samples/{NAME}.json, calib/{NAME}.json", flush=True)
+
+    from tt_meteor.api import load_sample
+    from tt_meteor.host.inputs import prepare_request
+
+    kw = load_sample(PKG / "samples" / f"{NAME}.json")
+    frame = prepare_request(kw["images"], kw["calibration"], kw["ego_speed"], kw.get("stream"))
+    assert np.array_equal(frame.imgs[0].transpose(0, 2, 3, 1), imgs), "the PNGs do not read back bit-exactly"
+    feed_sha = {k: hashlib.sha256(np.ascontiguousarray(v).tobytes()).hexdigest() for k, v in frame.feed().items()}
+    (PKG / "tests" / "goldens" / f"{NAME}_inputs.json").write_text(json.dumps(
+        {"_doc": f"sha256 of the graph feed of samples/{NAME}.json (code/scripts/make_synthetic_sample.py)",
+         "input_sha256": feed_sha, "present": frame.present.astype(int).tolist()}, indent=1) + "\n")
+    if not reference:
+        return
+    ref = _load_net(weights_dir, threads)
+    t1 = time.time()
+    body = ref(**kw).to_dict()
+    body["timing_ms"] = {}
+    (PKG / "samples" / f"{NAME}.reference.json").write_text(json.dumps(body) + "\n")
+    labels = {}
+    for d in body["detections"]:
+        labels[d["label"]] = labels.get(d["label"], 0) + 1
+    n2d = sum(len(v) for v in body["detections_2d"].values())
+    print(f"reference ({time.time() - t1:.0f} s): {body['num_detections']} 3D boxes {labels} scores "
+          f"{[round(d['score'], 3) for d in body['detections']]}, {n2d} 2D boxes, mode {body['plan']['mode']} "
+          f"(probs {[round(v, 3) for v in body['plan']['mode_probs']]}), TL {body['traffic_light']['state']}, "
+          f"path end {[round(v, 2) for v in body['trajectory'][-1]]}; total {time.time() - t0:.0f} s", flush=True)
+
+
+def default_weights_dir():
+    snaps = Path.home() / ".cache/huggingface/hub/models--AutowareFoundation--meteor/snapshots"
+    for cand in sorted(snaps.glob("*")):
+        if (cand / "meteor_v157c3Z.onnx").is_file():
+            return cand
+    return Path("/home/ubuntu/experiments/tt-models/assets/meteor/hf_meteor")
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("stage", choices=("render", "optimise", "finalise", "all"))
+    ap.add_argument("--work", type=Path, default=BUNDLE / "logs" / "synthetic_sample")
+    ap.add_argument("--threads", type=int, default=4)
+    ap.add_argument("--steps", type=int, default=60)
+    ap.add_argument("--lr", type=float, default=0.02)
+    ap.add_argument("--no-reference", action="store_true")
+    ap.add_argument("--weights-dir", type=Path, default=None)
+    a = ap.parse_args()
+    wd = a.weights_dir or default_weights_dir()
+    if a.stage in ("render", "all"):
+        stage_render(a.work)
+    if a.stage in ("optimise", "all"):
+        stage_optimise(a.work, wd, a.threads, a.steps, a.lr)
+    if a.stage in ("finalise", "all"):
+        stage_finalise(a.work, wd, a.threads, not a.no_reference)
+
+
+if __name__ == "__main__":
+    main()

code/scripts/precision_emulation.py

@@ -0,0 +1,182 @@
+# SPDX-License-Identifier: Apache-2.0
+"""CPU emulation of the device's activation precision on the class decisions (PORT_LOG.md E20; no device).
+
+    cd bundles/meteor-p150 && OMP_NUM_THREADS=4 PYTHONPATH=$PWD/code \\
+        /home/ubuntu/experiments/tt-models/tools/research-venv/bin/python code/scripts/precision_emulation.py image bb tt Bt
+    ... precision_emulation.py bev
+    ... precision_emulation.py det meteor_valday_f040 fff bff fbf ffb bbb nbb bFb nFF
+    ... precision_emulation.py absent ff ft fT TT tt bb
+
+``image``: the CPU reference's image branch with every conv / resize output rounded to bf16 (``b``), TF32 (``t``: 10
+explicit mantissa bits, what the device's fp32 conv path computes: its unpacker rounds fp32 activations to TF32) or
+kept fp32 (``f``), per (encoder, heads) pair; ``B`` = the stem output alone in bf16 (``ttnn.max_pool2d`` is bf16
+only). Prints the depth / seg2d argmax agreement with the fp32 reference (PLAN 2.13 gate: >= 0.99) and the depth
+top-2 logit margins. ``bev``: the lane decoder + seg refiner from the golden raw BEV with bf16 activations (the convs
+whose outputs the device keeps fp32 stay fp32) -> lane agreement. ``det FRAME RDB ...`` (PORT_LOG.md E22): the det
+stem + merged heads (``D``) and the box refiner (``B``) from the golden raw BEV (``R``) of FRAME, each letter one of
+``f`` (fp32), ``b`` (bf16 conv / resize outputs), ``t`` (TF32), ``F`` (fp32 convs = three bf16 terms, TF32 resizes:
+the device's terms3 path), and for ``R`` also ``n`` (bf16 after 0.3 % relative noise: the chained device raw) -> the
+strict det3d recall / precision of the decoded boxes vs the golden outputs (``tests/box_agreement.py``) and the hm
+error. ``absent CR ...`` (PORT_LOG.md E23 / I7): the image branch on an absent camera (the all-zero normalised input
+of ``input_norm=imagenet``) with conv outputs ``C`` and resize outputs ``R`` rounded as above, plus ``T`` (TF32 by
+TRUNCATION) -> seg2d / depth agreement with the variant golden's absent camera and with the device's argmax maps
+(``logs/meteor/variants_device_absent_argmax.npz``, written by ``tests/test_variants_device.py``).
+Goldens: ``research/meteor/goldens``.
+Measured (PandaSet 019 f40): image bb 0.9819 / 0.9908, bt 0.9856 / 0.9948, tb 0.9913 / 0.9917, tt 0.9981 / 0.9988,
+Bt 0.9874 / 0.9951; bev lane 0.99966 (bf16) / 0.99996 (fp32). det (valday #40, strict recall / precision): fff
+1.0 / 1.0, bff 1.0 / 1.0, fbf 0.971 / 1.0, ffb 1.0 / 1.0, bbb 0.914 / 1.0, nbb 0.914 / 1.0, bFb 1.0 / 1.0, nFF
+1.0 / 1.0 (nuScenes nbb 0.821 / 0.885, nFF 1.0 / 1.0): the bf16 det stem is the loss. absent (seg2d / depth vs the
+reference; vs the device): ff 1.0 / 1.0; fT 0.9975 / 0.9993; tt 0.9966 / 0.9981; TT 0.9856 / 0.9901 (0.9927 / 0.9933
+vs the device); bb 0.9724 / 0.9800; the device 0.9832 / 0.9913 (camera 6): it behaves like TF32-truncated conv outputs.
+"""
+from __future__ import annotations
+
+import os
+import sys
+from pathlib import Path
+
+import numpy as np
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+GOLD = Path(os.environ.get("METEOR_GOLDENS", "/home/ubuntu/experiments/tt-models/research/meteor/goldens"))
+
+FP32_OUT_BEV = {"dec/out/out.3", "lane_branch/lane_branch.6", "refiner/seg/out/out.3"}
+
+
+def rnd(t, m):
+    import torch
+
+    if m == "f":
+        return t
+    if m == "T":                                                 # TF32 by truncation
+        a = t.contiguous().numpy().view(np.uint32) & np.uint32(0xFFFFE000)
+        return torch.from_numpy(a.view(np.float32).copy())
+    if m == "b":
+        return t.to(torch.bfloat16).to(torch.float32)
+    a = t.contiguous().numpy().view(np.uint32).astype(np.uint64)
+    a = ((a + 0x1000) & 0xFFFFE000).astype(np.uint32)            # round to 10 explicit mantissa bits (TF32)
+    return torch.from_numpy(a.view(np.float32))
+
+
+def main() -> None:
+    import torch
+
+    from tt_meteor.reference.model import MeteorNet
+    from tt_meteor.reference.weights import MeteorWeights
+
+    torch.set_num_threads(int(os.environ.get("OMP_NUM_THREADS", "4")))
+    what = sys.argv[1] if len(sys.argv) > 1 else "image"
+    net = MeteorNet(MeteorWeights(), check_graph=False)
+    z = np.load(GOLD / "pandaset_019_f40" / "taps.npz")
+    cur = {"m": "b", "stem": None}
+    conv0, resize0 = net.conv, net.resize
+
+    def conv(x, module, relu=False):
+        y = conv0(x, module, relu)
+        if what == "bev" and module in FP32_OUT_BEV:
+            return y
+        if module == "stem/stem.0" and cur["stem"]:
+            return rnd(y, cur["stem"])
+        return rnd(y, cur["m"])
+
+    if what == "det":
+        return det(net, sys.argv[2], sys.argv[3:])
+    if what == "absent":
+        return absent(net, sys.argv[2:])
+    net.conv = conv
+    net.resize = lambda x, hw: rnd(resize0(x, hw), cur["m"])
+    with torch.inference_mode():
+        if what == "bev":
+            raw = rnd(torch.from_numpy(z["bev.raw"].astype(np.float32)), "b")
+            for m in ("b", "f"):
+                cur["m"] = m
+                ll = net.seg_refiner(net.lane_decoder(raw))
+                lane = torch.argmax(ll, dim=1).numpy().astype(np.uint8)
+                print(f"bev {m}: lane agreement {(lane == z['lane']).mean():.5f}")
+            return
+        x = net.normalize(torch.from_numpy(z["input.imgs"]))
+        for cfg in sys.argv[2:] or ["bb", "tt"]:
+            enc, head = cfg[0], cfg[1]
+            cur["stem"] = "b" if enc == "B" else None
+            cur["m"] = "t" if enc == "B" else enc
+            f = net.image_encoder(x)
+            cur["m"], cur["stem"] = head, None
+            dl, _ = net.depth_head(f)
+            seg = net.seg2d_head(f)
+            d = torch.argmax(dl.reshape(1, 8, 64, 108, 192), dim=2).numpy()
+            s = torch.argmax(torch.clamp(seg, -30, 30).reshape(1, 8, 21, 108, 192), dim=2).numpy()
+            print(f"encoder {enc} heads {head}: depth {(d == z['depth']).mean():.4f} seg2d {(s == z['seg2d']).mean():.4f}",
+                  flush=True)
+        srt = np.sort(z["depth.logits"].astype(np.float32), axis=1)
+        margin = srt[:, -1] - srt[:, -2]
+        print("depth top-2 margin quantiles (0.05, 0.1, 0.25, 0.5):",
+              np.round(np.quantile(margin, [0.05, 0.1, 0.25, 0.5]), 4).tolist())
+
+
+def det(net, frame: str, configs) -> None:
+    """``det`` mode (module docstring)."""
+    import torch
+
+    from tt_meteor.host.postprocess import PostConfig
+    from tt_meteor.tests.box_agreement import box_agreement, decode_3d
+
+    g = np.load(GOLD / frame / "taps.npz")
+    conv0, resize0 = net.conv, net.resize
+    cur = {"m": "f"}
+    conv_mode = lambda m: "f" if m == "F" else m          # noqa: E731
+    resize_mode = lambda m: "t" if m == "F" else m        # noqa: E731
+    net.conv = lambda x, module, relu=False: rnd(conv0(x, module, relu), conv_mode(cur["m"]))
+    net.resize = lambda x, hw: rnd(resize0(x, hw), resize_mode(cur["m"]))
+    cfg = PostConfig()
+    ref = decode_3d({"hm": g["hm"], "reg": g["reg"]}, cfg)
+    torch.manual_seed(0)
+    for c in configs or ["bbb", "bFF"]:
+        r_m, d_m, b_m = c
+        with torch.inference_mode():
+            raw = torch.from_numpy(g["bev.raw"].astype(np.float32))
+            raw = rnd(raw + torch.randn_like(raw) * raw.abs() * 0.003, "b") if r_m == "n" else rnd(raw, r_m)
+            cur["m"] = d_m
+            _, hm_pre, reg_pre = net.det_stem(raw)
+            cur["m"] = b_m
+            hm, reg = net.box_refiner(hm_pre, reg_pre)
+        r = box_agreement(decode_3d({"hm": hm.numpy(), "reg": reg.numpy()}, cfg), ref, cfg)
+        err = np.abs(hm.numpy() - g["hm"]).ravel()
+        print(f"{frame} raw {r_m} det {d_m} refiner {b_m}: recall {r['recall']:.4f} precision {r['precision']:.4f} "
+              f"hm max abs {err.max():.4f}", flush=True)
+
+
+def absent(net, configs) -> None:
+    """``absent`` mode (module docstring)."""
+    import torch
+
+    g = np.load(GOLD / "pandaset_019_f40_imagenet_linear" / "taps.npz")
+    cam = int(np.flatnonzero(~np.asarray(g["input.present"], bool))[0])
+    ref_s, ref_d = g["seg2d"][0][cam], g["depth"][0][cam]
+    dev_path = Path(os.environ.get("TT_MODELS_ROOT", "/home/ubuntu/experiments/tt-models")) / "logs" / "meteor" / \
+        "variants_device_absent_argmax.npz"
+    dev = None
+    if dev_path.is_file():
+        with np.load(dev_path) as d:
+            i = int(np.flatnonzero(d["absent"] == cam)[0])
+            dev = {"seg2d": d["seg2d"][i], "depth": d["depth"][i]}
+    conv0, resize0 = net.conv, net.resize
+    cur = {"c": "f", "r": "f"}
+    net.conv = lambda x, module, relu=False: rnd(conv0(x, module, relu), cur["c"])
+    net.resize = lambda x, hw: rnd(resize0(x, hw), cur["r"])
+    x = torch.zeros(1, 3, *g["input.imgs"].shape[-2:])                # imagenet: absent = zero AFTER normalising
+    for c in configs or ["ff", "TT"]:
+        cur["c"], cur["r"] = c[0], c[1]
+        with torch.inference_mode():
+            f = net.image_encoder(x)
+            dl, _ = net.depth_head(f)
+            seg = net.seg2d_head(f)
+        s = torch.argmax(torch.clamp(seg, -30, 30), dim=1).numpy()[0]
+        d = torch.argmax(dl.reshape(1, -1, *s.shape), dim=1).numpy()[0]
+        msg = f"absent cam {cam} conv {c[0]} resize {c[1]}: seg2d {(s == ref_s).mean():.4f} depth {(d == ref_d).mean():.4f}"
+        if dev is not None:
+            msg += f" | vs device seg2d {(s == dev['seg2d']).mean():.4f} depth {(d == dev['depth']).mean():.4f}"
+        print(msg, flush=True)
+
+
+if __name__ == "__main__":
+    main()

code/scripts/profile_ops.py

@@ -0,0 +1,231 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Device profile of meteor-p150: one eager frame with module signposts and ``--rounds`` traced replays of the
+``frame`` trace, between Tracy signposts (OPT_BASELINE.md).
+
+    ROOT=/home/ubuntu/experiments/tt-models
+    $ROOT/bin/devrun -t 3600 -- python -m tracy -r -p -v --no-web-server --op-support-count 6000 \\
+        -o $ROOT/generated/profiler/meteor_baseline code/scripts/profile_ops.py --rounds 2
+    tt-perf-report <ops_perf_results_*.csv> --start-signpost frame --end-signpost frame_end --tracing-mode
+    python code/scripts/profile_summary.py $ROOT/generated/profiler/meteor_baseline --json s.json --md s.md
+
+ttnn-visualizer memory + graph report of one eager frame (slow; never used for timing):
+
+    $ROOT/bin/devrun -t 3600 -- python code/scripts/profile_ops.py --graph-report $ROOT/generated/ttnn_reports/meteor_baseline
+    ttnn-visualizer --profiler-path $ROOT/generated/ttnn_reports/meteor_baseline/frame \\
+        --performance-path $ROOT/generated/profiler/meteor_baseline/reports/<ts>
+
+The model is loaded like the API but without the trace capture (``warmup_variants="none"``), the rig's lift tables
+are written, and one eager frame compiles every program. Then, with the device profiler buffer flushed
+(``ttnn.ReadDeviceProfiler``) before and after each section:
+
+- ``eager_frame`` .. ``eager_frame_end``: one eager frame (program cache hot) with a signpost per module (``img.*``,
+  ``lift``, ``lift.resize``, ``bev.*``, ``head.*``, ``out.pack``). Ops issued inline after a module call (argmaxes,
+  the depth softmax, the ``to_rm`` conversions, planner adds) belong to the module signposted last. An eager frame
+  BEFORE the capture is the documented order (``TtMETEOR.run_frame_eager``: no allocation next to a live trace);
+- the trace is captured (``TtMETEOR.capture``) and replayed once (not profiled);
+- ``frame`` .. ``frame_end`` (and ``frame2`` ... for ``--rounds``): one replay each, the served device path.
+
+Through the identical op order the replay's ops inherit the eager attribution (``profile_summary.py``). One frame is
+about 2,000 programs, above the profiler's default buffer of ~1,000, hence ``--op-support-count 6000``. The output
+folder must be absolute and outside the bundle (PLAN.md 5.1).
+"""
+from __future__ import annotations
+
+import argparse
+import contextlib
+import json
+import os
+import sys
+import threading
+import time
+from pathlib import Path
+from typing import Callable, List
+
+HERE = Path(__file__).resolve()
+sys.path.insert(0, str(HERE.parent))
+sys.path.insert(0, str(HERE.parents[1]))
+
+IMAGE_PARTS = (("normalize", "img.normalize"), ("stem_conv", "img.stem"), ("stages", "img.resnet"),
+               ("fuse", "img.fpn_fuse"), ("seg_logits", "img.seg2d"), ("depth_logits", "img.depth"),
+               ("depth_mean", "img.depth_mean"), ("context", "img.ctx_paint"), ("det2d", "img.det2d"),
+               ("tl", "img.tl"))
+BEV_PARTS = ("fuse", "lane", "det", "feat_bf16", "occupancy", "stationary", "traj", "risk")
+HEAD_PARTS = ("ego_stem", "ego_mlp", "ego_attn", "sem", "risk_sample", "decoder_wp", "e2e", "seg_refine",
+              "box_refine")
+
+
+class _Signposted:
+    """A callable stand-in that emits a signpost (optionally one after the call too), then calls the inner object."""
+
+    def __init__(self, inner, label: str, signpost, after: str = ""):
+        self.inner, self._label, self._signpost, self._after = inner, label, signpost, after
+
+    def __call__(self, *args, **kwargs):
+        if self._label:
+            self._signpost(self._label)
+        out = self.inner(*args, **kwargs)
+        if self._after:
+            self._signpost(self._after)
+        return out
+
+    def __getattr__(self, name):
+        return getattr(self.inner, name)
+
+
+def module_signposts(tt, signpost) -> Callable[[], None]:
+    """Wrap the stage modules of a ``TtMETEOR`` with signposting stand-ins; returns the function that restores
+    them."""
+    undo: List[Callable[[], None]] = []
+
+    def swap(obj, attr, label, after=""):
+        had = attr in obj.__dict__
+        old = obj.__dict__.get(attr)
+        inner = getattr(obj, attr)
+        setattr(obj, attr, _Signposted(inner, label, signpost, after))
+        undo.append(lambda: setattr(obj, attr, old) if had else obj.__dict__.pop(attr, None))
+
+    img = tt.image
+    for attr, label in IMAGE_PARTS:
+        swap(img, attr, label)
+    lat0 = img.lat[0]
+    img.lat[0] = _Signposted(lat0, "img.fpn", signpost)
+    undo.append(lambda: img.lat.__setitem__(0, lat0))
+    swap(tt.lift, "resize", "lift.resize")
+    swap(tt, "lift", "lift")
+    for attr in BEV_PARTS:
+        swap(tt.bev, attr, f"bev.{attr}")
+    for attr in HEAD_PARTS:
+        swap(tt.head, attr, f"head.{attr}")
+    swap(tt, "head", "", after="out.pack")
+
+    def restore() -> None:
+        for fn in reversed(undo):
+            fn()
+
+    return restore
+
+
+def rss_guard(limit_gb: float, stop: threading.Event) -> None:
+    """The graph capture lives in host RAM: never let it take the shared host down."""
+    page = os.sysconf("SC_PAGE_SIZE")
+    while not stop.wait(1.0):
+        with open("/proc/self/statm") as fh:
+            rss = int(fh.read().split()[1]) * page
+        if rss > limit_gb * 2 ** 30:
+            print(f"graph capture: host RSS {rss / 2 ** 30:.1f} GB > {limit_gb} GB, abort", flush=True)
+            os._exit(3)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--input", default="sample", help="bench.py input spec (sample | synthetic | name=manifest)")
+    ap.add_argument("--rounds", type=int, default=2)
+    ap.add_argument("--no-eager", dest="eager", action="store_false")
+    ap.add_argument("--graph-report", default=None, help="write a ttnn-visualizer graph report of one eager frame to "
+                                                         "<dir>/frame instead of profiling")
+    ap.add_argument("--max-rss-gb", type=float, default=24.0)
+    ap.add_argument("--dispatch", default=None, choices=["eth", "worker"])
+    ap.add_argument("--num-cqs", type=int, default=None)
+    ap.add_argument("--json", default=None, help="write the run's metadata (config, trace description)")
+    a = ap.parse_args()
+    import ttnn
+
+    from bench import cache_entries, load_input
+    from tt_meteor import METEOR
+    from tt_meteor.host.inputs import prepare_request
+    from tt_meteor.ttaw.profiling import read_device_profiler, signpost
+
+    name, req, path = load_input(a.input)
+    t0 = time.perf_counter()
+    model = METEOR.from_pretrained(dispatch=a.dispatch, num_command_queues=a.num_cqs, warmup_variants="none")
+    meta = {"config": model.device_info, "input": str(path or name), "load_s": round(time.perf_counter() - t0, 1)}
+    print("config:", json.dumps(meta["config"]), flush=True)
+    dev, tt = model.device, model.tt
+    sync = lambda: ttnn.synchronize_device(dev)  # noqa: E731
+    graph_out = None
+    try:
+        frame = prepare_request(req["images"], req["calibration"], req["ego_speed"], req.get("stream"))
+        geom = model.calib_cache.get(frame.K, frame.T_cam_ego)
+        t1 = time.perf_counter()
+        tt.run_frame_eager(frame, geom)                       # compile + constants
+        sync()
+        meta["first_eager_s"] = round(time.perf_counter() - t1, 1)
+        meta["program_cache_entries"] = cache_entries(dev)
+        print("first eager frame %.1f s, programs %s" % (meta["first_eager_s"], meta["program_cache_entries"]),
+              flush=True)
+        if a.graph_report:
+            out_dir = Path(a.graph_report).resolve() / "frame"
+            out_dir.mkdir(parents=True, exist_ok=True)
+            json_path = out_dir / "graph_capture.json"
+            stop = threading.Event()
+            threading.Thread(target=rss_guard, args=(a.max_rss_gb, stop), daemon=True).start()
+            prev = ttnn.CONFIG.enable_fast_runtime_mode
+            ttnn.CONFIG.enable_fast_runtime_mode = False
+            t1 = time.perf_counter()
+            try:
+                ttnn.graph.begin_graph_capture()
+                tt.run_frame_eager(frame, geom)
+                sync()
+                ttnn.graph.end_graph_capture_to_file(str(json_path))
+            finally:
+                ttnn.CONFIG.enable_fast_runtime_mode = prev
+                stop.set()
+            meta["graph_capture_s"] = round(time.perf_counter() - t1, 1)
+            meta["graph_json_mb"] = round(json_path.stat().st_size / 1e6, 1)
+            print("graph captured:", meta["graph_capture_s"], "s,", meta["graph_json_mb"], "MB", flush=True)
+            graph_out = (out_dir, json_path)
+        else:
+            read_device_profiler(dev)                          # drop the load / compile data
+            if a.eager:
+                restore = module_signposts(tt, signpost)
+                try:
+                    signpost("eager_frame")
+                    tt.run_frame_eager(frame, geom)
+                    sync()
+                    signpost("eager_frame_end")
+                finally:
+                    restore()
+                read_device_profiler(dev)
+            t1 = time.perf_counter()
+            tt.capture()
+            meta["capture_s"] = round(time.perf_counter() - t1, 1)
+            tt.run_frame(frame, geom)                          # one served frame (tables current, inputs uploaded)
+            sync()
+            read_device_profiler(dev)
+            for k in range(1, a.rounds + 1):
+                tag = "" if k == 1 else str(k)
+                signpost(f"frame{tag}")
+                tt.runner.replay("frame")
+                sync()
+                signpost(f"frame{tag}_end")
+                read_device_profiler(dev)
+        meta["trace"] = tt.describe()
+        meta["program_cache_entries_end"] = cache_entries(dev)
+    finally:
+        model.close()
+    if graph_out is not None:
+        out_dir, json_path = graph_out
+        t1 = time.perf_counter()
+        try:
+            db = ttnn.graph_report.import_report(json_path, out_dir)
+        finally:
+            json_path.unlink(missing_ok=True)
+        ttnn.save_config_to_json_file(out_dir / "config.json")
+        cfg = json.loads((out_dir / "config.json").read_text())
+        cfg.update({"enable_fast_runtime_mode": False, "enable_graph_report": True,
+                    "enable_detailed_buffer_report": False, "root_report_path": str(out_dir.parent),
+                    "report_name": out_dir.name})
+        (out_dir / "config.json").write_text(json.dumps(cfg, indent=4) + "\n")
+        meta["graph_report"] = {"dir": str(out_dir), "db": str(db), "import_s": round(time.perf_counter() - t1, 1),
+                                "db_mb": round(Path(db).stat().st_size / 1e6, 1)}
+        print("graph report:", json.dumps(meta["graph_report"]), flush=True)
+    if a.json:
+        Path(a.json).parent.mkdir(parents=True, exist_ok=True)
+        Path(a.json).write_text(json.dumps(meta, indent=1, default=str) + "\n")
+    print(json.dumps(meta, indent=1, default=str))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/profile_summary.py

@@ -0,0 +1,288 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Summary of a ``profile_ops.py`` device profile (the ``ops_perf_results_<ts>.csv`` of ``python -m tracy -r``).
+
+    python code/scripts/profile_summary.py $ROOT/generated/profiler/meteor_baseline --json out.json --md out.md
+
+- per traced segment (the ``frame`` signpost range, and ``frame2`` ... of later rounds):
+  programs, kernel sum, FW sum, op-to-op gaps (sum over the segment, its first op excluded) and the span (first FW
+  start to last FW end, at the CHIP_FREQ of ``profile_log_device.csv``, else 1350 MHz), per op code;
+- per module stage: the eager run's module signposts (``img.normalize`` ... ``out.pack``) label the eager op sequence;
+  when it equals the traced segment's op sequence op for op, the labels are carried over to the traced ops, and the
+  table reports the traced kernel times per stage, per block (``img`` / ``lift`` / ``bev`` / ``head`` / ``out``)
+  and per (stage, op code);
+- the top ops by traced kernel time and the largest op-to-op gaps.
+
+csv + json only (no ttnn): runs anywhere.
+"""
+from __future__ import annotations
+
+import argparse
+import csv
+import glob
+import json
+import os
+import sys
+from collections import OrderedDict, defaultdict
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+SEGMENTS = ("frame",)
+KERNEL = "DEVICE KERNEL DURATION [ns]"
+FW = "DEVICE FW DURATION [ns]"
+GAP = "OP TO OP LATENCY [ns]"
+
+
+def find_csv(path: Path) -> Path:
+    if path.is_file():
+        return path
+    found = sorted(glob.glob(str(path / "**" / "ops_perf_results*.csv"), recursive=True), key=os.path.getmtime)
+    if not found:
+        raise FileNotFoundError(f"no ops_perf_results*.csv under {path}")
+    return Path(found[-1])
+
+
+def chip_freq_mhz(csv_path: Path, root: Path) -> Optional[float]:
+    cands = [csv_path.parent / "profile_log_device.csv", root / ".logs" / "profile_log_device.csv"]
+    cands += [Path(p) for p in glob.glob(str(root / "**" / "profile_log_device.csv"), recursive=True)]
+    for c in cands:
+        try:
+            head = c.read_text(errors="ignore").splitlines()[0]
+        except (OSError, IndexError):
+            continue
+        for part in head.split(","):
+            if "CHIP_FREQ" in part and ":" in part:
+                try:
+                    return float(part.split(":")[1].strip())
+                except ValueError:
+                    pass
+    return None
+
+
+def num(row: Dict[str, str], key: str) -> float:
+    v = (row.get(key) or "").strip()
+    try:
+        return float(v) if v else 0.0
+    except ValueError:
+        return 0.0
+
+
+def is_signpost(row: Dict[str, str]) -> bool:
+    return (row.get("OP TYPE") or "").strip() == "signpost"
+
+
+def window(rows: List[Dict[str, str]], name: str) -> Optional[List[Tuple[str, Dict[str, str]]]]:
+    """Rows between the first signpost ``name`` and the next ``name_end`` as ``(label, row)``; ``label`` is the
+    latest inner signpost (``""`` before the first one). None when the signpost is absent."""
+    out, on, label = [], False, ""
+    for r in rows:
+        if is_signpost(r):
+            code = r.get("OP CODE", "")
+            if on and code == f"{name}_end":
+                return out
+            if not on and code == name:
+                on = True
+                continue
+            if on:
+                label = code
+            continue
+        if on:
+            out.append((label, r))
+    return out if on else None
+
+
+def tensor_desc(row: Dict[str, str], prefix: str) -> str:
+    """``INPUT_0`` / ``OUTPUT_0`` columns -> ``[W, Z, Y, X] LAYOUT DTYPE MEMORY`` (logical dims), '' when absent."""
+    dims = []
+    for d in "WZYX":
+        v = (row.get(f"{prefix}_{d}_PAD[LOGICAL]") or "").strip()
+        if not v:
+            return ""
+        dims.append(v.split("[")[1].rstrip("]") if "[" in v else v)
+    mem = (row.get(f"{prefix}_MEMORY") or "").replace("DEV_1_", "").replace("DEV_0_", "")
+    return f"[{','.join(dims)}] {row.get(f'{prefix}_LAYOUT', '')} {row.get(f'{prefix}_DATATYPE', '')} {mem}".strip()
+
+
+def io_desc(row: Dict[str, str], prefix: str, n: int = 3) -> str:
+    if (row.get(prefix + "S") or "").strip():          # older reports: one INPUTS / OUTPUTS column
+        text = " ".join(row[prefix + "S"].split())
+        return text if len(text) <= 160 else text[:157] + "..."
+    parts = [tensor_desc(row, f"{prefix}_{i}") for i in range(n)]
+    return "; ".join(p for p in parts if p)
+
+
+def group_of(label: str) -> str:
+    """``img.resnet`` -> ``img``, ``bev.fuse`` -> ``bev``, ``lift.resize`` -> ``lift``: the block of a stage."""
+    return label.partition(".")[0]
+
+
+def summarize(ops: List[Dict[str, str]], freq: float) -> Dict[str, Any]:
+    kernel = [num(r, KERNEL) / 1e3 for r in ops]
+    fw = [num(r, FW) / 1e3 for r in ops]
+    gaps = [num(r, GAP) / 1e3 for r in ops]
+    starts = [num(r, "DEVICE FW START CYCLE") for r in ops]
+    ends = [num(r, "DEVICE FW END CYCLE") for r in ops]
+    valid = [s for s in starts if s > 0]
+    span = (max(ends) - min(valid)) / freq if ops and valid and max(ends) > 0 else None
+    by_op: Dict[str, List[float]] = defaultdict(lambda: [0, 0.0])
+    fid: Dict[str, int] = defaultdict(int)
+    for r, k in zip(ops, kernel):
+        by_op[r.get("OP CODE", "?")][0] += 1
+        by_op[r.get("OP CODE", "?")][1] += k
+        f = (r.get("MATH FIDELITY") or "").strip()
+        if f:
+            fid[f] += 1
+    total = sum(kernel) or 1.0
+    traced = sum(1 for r in ops if (r.get("METAL TRACE ID") or "").strip())
+    return {"ops": len(ops), "traced_ops": traced, "kernel_us": round(sum(kernel), 1), "fw_us": round(sum(fw), 1),
+            "op2op_us": round(sum(gaps[1:]), 1), "span_us": None if span is None else round(span, 1),
+            "by_op": [{"op": op, "count": int(c), "kernel_us": round(t, 1), "share": round(t / total, 4)}
+                      for op, (c, t) in sorted(by_op.items(), key=lambda kv: -kv[1][1])],
+            "fidelity": dict(fid)}
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("source", type=Path, help="tracy output dir (-o) or an ops_perf_results CSV")
+    ap.add_argument("--freq-mhz", type=float, default=None)
+    ap.add_argument("--top", type=int, default=15)
+    ap.add_argument("--json", type=Path, default=None)
+    ap.add_argument("--md", type=Path, default=None)
+    a = ap.parse_args()
+    path = find_csv(a.source)
+    with open(path, newline="") as f:
+        rows = list(csv.DictReader(f))
+    root = a.source if a.source.is_dir() else path.parent
+    freq = a.freq_mhz or chip_freq_mhz(path, root) or 1350.0
+    res: Dict[str, Any] = {"csv": str(path), "freq_mhz": freq, "rows": len(rows),
+                           "signposts": [r.get("OP CODE") for r in rows if is_signpost(r)], "segments": OrderedDict(),
+                           "eager": OrderedDict()}
+    traced_ops: Dict[str, List[Dict[str, str]]] = {}
+    for k in ("", "2", "3"):
+        for seg in SEGMENTS:
+            w = window(rows, f"{seg}{k}")
+            if w is None:
+                continue
+            ops = [r for _, r in w]
+            traced_ops[f"{seg}{k}"] = ops
+            res["segments"][f"{seg}{k}"] = summarize(ops, freq)
+    steady = [s for s in SEGMENTS if s in res["segments"]]
+    if steady:
+        seg = res["segments"]
+        res["steady_frame"] = {key: round(sum(seg[s][key] or 0 for s in steady), 1)
+                               for key in ("ops", "kernel_us", "fw_us", "op2op_us", "span_us")}
+        res["steady_frame"]["segments"] = steady
+    # eager windows with stage labels; carry the labels over to the traced ops when the sequences match
+    labels: Dict[str, List[str]] = {}
+    groups: "OrderedDict[str, Dict[str, Any]]" = OrderedDict()
+    for seg in SEGMENTS:
+        w = window(rows, f"eager_{seg}")
+        if w is None:
+            continue
+        ops = [r for _, r in w]
+        res["eager"][seg] = summarize(ops, freq)
+        eager_codes = [r.get("OP CODE") for r in ops]
+        traced_codes = [r.get("OP CODE") for r in traced_ops.get(seg, [])]
+        match = eager_codes == traced_codes
+        res["eager"][seg]["sequence_matches_trace"] = match
+        if not match:
+            res["eager"][seg]["mismatch"] = {"eager_ops": len(eager_codes), "traced_ops": len(traced_codes),
+                                             "first_diff": next((i for i, (x, y) in enumerate(
+                                                 zip(eager_codes, traced_codes)) if x != y), None)}
+        labels[seg] = [lab or f"{seg[:3]}.begin" for lab, _ in w]
+        # per stage: eager kernel sums (always) and traced kernel sums (when aligned)
+        stages: "OrderedDict[str, Dict[str, Any]]" = OrderedDict()
+        for i, (lab, r) in enumerate(w):
+            st = stages.setdefault(lab or f"{seg[:3]}.begin", {"ops": 0, "eager_kernel_us": 0.0, "eager_op2op_us": 0.0,
+                                                               "kernel_us": 0.0, "op2op_us": 0.0,
+                                                               "by_op": defaultdict(float)})
+            st["ops"] += 1
+            st["eager_kernel_us"] += num(r, KERNEL) / 1e3
+            st["eager_op2op_us"] += num(r, GAP) / 1e3 if i else 0.0
+            src = traced_ops[seg][i] if match else r
+            st["by_op"][src.get("OP CODE", "?")] += num(src, KERNEL) / 1e3
+            if match:
+                st["kernel_us"] += num(src, KERNEL) / 1e3
+                st["op2op_us"] += num(src, GAP) / 1e3 if i else 0.0
+            g = group_of(lab or f"{seg[:3]}.begin")
+            go = groups.setdefault(g, {"ops": 0, "kernel_us": 0.0, "by_op": defaultdict(lambda: [0, 0.0])})
+            go["ops"] += 1
+            go["kernel_us"] += num(src, KERNEL) / 1e3
+            go["by_op"][src.get("OP CODE", "?")][0] += 1
+            go["by_op"][src.get("OP CODE", "?")][1] += num(src, KERNEL) / 1e3
+            go["traced"] = match
+        for v in stages.values():
+            v["by_op"] = {op: round(t, 1) for op, t in sorted(v["by_op"].items(), key=lambda kv: -kv[1])}
+        res["eager"][seg]["stages"] = {k: {kk: round(vv, 1) if isinstance(vv, float) else vv for kk, vv in v.items()}
+                                       for k, v in stages.items()}
+    res["groups"] = {g: {"ops": v["ops"], "kernel_us": round(v["kernel_us"], 1), "traced": v.get("traced", False),
+                         "by_op": [{"op": op, "count": c, "kernel_us": round(t, 1)}
+                                   for op, (c, t) in sorted(v["by_op"].items(), key=lambda kv: -kv[1][1])]}
+                     for g, v in groups.items()}
+    # top ops by traced kernel time (all segments of the first round)
+    flat = []
+    for seg in SEGMENTS:
+        for i, r in enumerate(traced_ops.get(seg, [])):
+            lab = labels.get(seg, [None] * (i + 1))[i] if (seg in labels and res["eager"].get(seg, {}).get(
+                "sequence_matches_trace")) else None
+            flat.append({"segment": seg, "index": i, "stage": lab, "op": r.get("OP CODE"),
+                         "kernel_us": round(num(r, KERNEL) / 1e3, 1), "fw_us": round(num(r, FW) / 1e3, 1),
+                         "op2op_us": round(num(r, GAP) / 1e3, 2), "cores": r.get("CORE COUNT"),
+                         "fidelity": r.get("MATH FIDELITY"), "pm_ideal_us": round(num(r, "PM IDEAL [ns]") / 1e3, 1),
+                         "dram_bw_util": r.get("DRAM BW UTIL (%)"), "fpu_util": r.get("PM FPU UTIL (%)"),
+                         "inputs": io_desc(r, "INPUT"), "outputs": io_desc(r, "OUTPUT", 1)})
+    res["top_ops"] = sorted(flat, key=lambda d: -d["kernel_us"])[: a.top]
+    res["top_gaps"] = sorted(flat, key=lambda d: -d["op2op_us"])[: a.top]
+    if a.json:
+        a.json.parent.mkdir(parents=True, exist_ok=True)
+        a.json.write_text(json.dumps(res, indent=1) + "\n")
+    lines = [f"csv `{path}` ({len(rows)} rows), CHIP_FREQ {freq:.0f} MHz", "",
+             "| segment | programs | kernel us | FW us | op-to-op us | span us |", "|---|---:|---:|---:|---:|---:|"]
+    for k, v in res["segments"].items():
+        lines.append(f"| {k} | {v['ops']} | {v['kernel_us']} | {v['fw_us']} | {v['op2op_us']} | {v['span_us']} |")
+    if "steady_frame" in res:
+        s = res["steady_frame"]
+        lines.append(f"| **steady frame** ({' + '.join(s['segments'])}) | {s['ops']} | {s['kernel_us']} | {s['fw_us']} | "
+                     f"{s['op2op_us']} | {s['span_us']} |")
+    for seg, e in res["eager"].items():
+        lines += ["", f"stages of `{seg}` (labels from the eager run; traced times "
+                      f"{'aligned op for op' if e['sequence_matches_trace'] else 'NOT aligned: eager times only'})", "",
+                  "| stage | ops | traced kernel us | traced op-to-op us | eager kernel us | eager op-to-op us | "
+                  "top op codes (us) |",
+                  "|---|---:|---:|---:|---:|---:|---|"]
+        for name, st in e["stages"].items():
+            tops = ", ".join(f"{op} {t}" for op, t in list(st["by_op"].items())[:4])
+            lines.append(f"| {name} | {st['ops']} | {st['kernel_us']} | {st['op2op_us']} | {st['eager_kernel_us']} | "
+                         f"{st['eager_op2op_us']} | {tops} |")
+    if res.get("groups"):
+        lines += ["", "stage groups (traced kernel time when the eager sequence matches the trace)", "",
+                  "| group | ops | kernel us | top op codes (us) |", "|---|---:|---:|---|"]
+        for g, v in res["groups"].items():
+            tops = ", ".join(f"{d['op']} {d['count']}x {d['kernel_us']}" for d in v["by_op"][:5])
+            lines.append(f"| {g} | {v['ops']} | {v['kernel_us']} | {tops} |")
+    for seg in SEGMENTS:
+        if seg in res["segments"]:
+            lines += ["", f"op codes of `{seg}` (traced)", "", "| op code | count | kernel us | share |",
+                      "|---|---:|---:|---:|"]
+            for b in res["segments"][seg]["by_op"][:12]:
+                lines.append(f"| {b['op']} | {b['count']} | {b['kernel_us']} | {b['share']:.1%} |")
+    lines += ["", f"top {a.top} ops by traced kernel time", "",
+              "| # | segment | stage | op | kernel us | cores | fidelity | DRAM BW % | FPU % | inputs | output |",
+              "|---:|---|---|---|---:|---:|---|---:|---:|---|---|"]
+    for i, d in enumerate(res["top_ops"], 1):
+        lines.append(f"| {i} | {d['segment']}[{d['index']}] | {d['stage']} | {d['op']} | {d['kernel_us']} | {d['cores']} | "
+                     f"{d['fidelity']} | {d['dram_bw_util']} | {d['fpu_util']} | {d['inputs']} | {d['outputs']} |")
+    lines += ["", f"largest {a.top} op-to-op gaps (traced)", "", "| # | segment | stage | op | gap us | kernel us |",
+              "|---:|---|---|---|---:|---:|"]
+    for i, d in enumerate(res["top_gaps"], 1):
+        lines.append(f"| {i} | {d['segment']}[{d['index']}] | {d['stage']} | {d['op']} | {d['op2op_us']} | {d['kernel_us']} |")
+    text = "\n".join(lines) + "\n"
+    if a.md:
+        a.md.parent.mkdir(parents=True, exist_ok=True)
+        a.md.write_text(text)
+    print(text)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/repro_grid_sample_eth.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Minimal standalone repro: ``ttnn.grid_sample`` at the METEOR lift shapes, replayed in a trace loop (no METEOR code).
+
+    bin/devrun -t 900 -- python -u repro_grid_sample_eth.py --dispatch eth --iters 2000 --jsonl out.jsonl
+    bin/devrun -t 900 -- python -u repro_grid_sample_eth.py --dispatch worker ...          # A/B
+
+One trace = ``--per-trace`` x (grid_sample of x64 [8, 108, 192, 64] and of x96 [8, 108, 192, 96], bf16 ROW_MAJOR,
+with one fp32 grid [8, 400, 250, 2]; bilinear, zeros, align_corners=False; DRAM outputs [8, 400, 250, C]).
+The grid is ``--grid file.npy`` (e.g. a real METEOR lift grid: normalised coords in [-2, 2], ~half out of image)
+or uniform random in [-1.2, 1.2]. Each replay is followed by ``synchronize_device`` and a fsync'd JSON heartbeat;
+every ``--check-every`` replays the outputs are read back and hashed (must not change). faulthandler dumps the Python
+stack when a replay stalls for ``--stall-s``. Nothing arms tt-triage or an operation timeout.
+"""
+from __future__ import annotations
+
+import argparse
+import faulthandler
+import hashlib
+import json
+import os
+import sys
+import time
+
+import numpy as np
+
+
+def log(fh, **rec):
+    rec["t"] = round(time.time(), 3)
+    line = json.dumps(rec)
+    print(line, flush=True)
+    if fh:
+        fh.write(line + "\n")
+        fh.flush()
+        os.fsync(fh.fileno())
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--dispatch", choices=["eth", "worker"], default="eth")
+    ap.add_argument("--cqs", type=int, default=1)
+    ap.add_argument("--iters", type=int, default=2000)
+    ap.add_argument("--per-trace", type=int, default=1, help="grid_sample pairs per trace")
+    ap.add_argument("--grid", help=".npy fp32 [8, 400, 250, 2] grid (default: random)")
+    ap.add_argument("--check-every", type=int, default=100)
+    ap.add_argument("--stall-s", type=float, default=60.0)
+    ap.add_argument("--jsonl")
+    a = ap.parse_args()
+    faulthandler.enable()
+    fh = open(a.jsonl, "a") if a.jsonl else None
+    faulthandler.dump_traceback_later(600, repeat=True)
+
+    import torch
+    import ttnn
+
+    core = ttnn.DispatchCoreType.ETH if a.dispatch == "eth" else ttnn.DispatchCoreType.WORKER
+    dev = ttnn.open_device(device_id=0, dispatch_core_config=ttnn.DispatchCoreConfig(core),
+                           num_command_queues=a.cqs, l1_small_size=32768, trace_region_size=64 << 20)
+    rc = 0
+    try:
+        g = dev.compute_with_storage_grid_size()
+        log(fh, ev="open", dispatch=a.dispatch, cqs=a.cqs, grid=f"{g.x}x{g.y}", per_trace=a.per_trace,
+            grid_file=a.grid)
+        rng = np.random.default_rng(0)
+        if a.grid:
+            grid_np = np.load(a.grid).astype(np.float32).reshape(8, 400, 250, 2)
+        else:
+            grid_np = rng.uniform(-1.2, 1.2, (8, 400, 250, 2)).astype(np.float32)
+        mk = lambda arr, dt: ttnn.from_torch(torch.from_numpy(arr), dtype=dt, layout=ttnn.ROW_MAJOR_LAYOUT,  # noqa: E731
+                                            device=dev, memory_config=ttnn.DRAM_MEMORY_CONFIG)
+        x64 = mk(rng.standard_normal((8, 108, 192, 64)).astype(np.float32), ttnn.bfloat16)
+        x96 = mk(rng.standard_normal((8, 108, 192, 96)).astype(np.float32), ttnn.bfloat16)
+        grid = mk(grid_np, ttnn.float32)
+
+        def body():
+            outs = []
+            for _ in range(a.per_trace):
+                for x in (x64, x96):
+                    outs.append(ttnn.grid_sample(x, grid, mode="bilinear", padding_mode="zeros",
+                                                 align_corners=False, memory_config=ttnn.DRAM_MEMORY_CONFIG))
+            return outs
+
+        warm = body()                                           # compile; then free, capture
+        ttnn.synchronize_device(dev)
+        for t in warm:
+            ttnn.deallocate(t)
+        tid = ttnn.begin_trace_capture(dev, cq_id=0)
+        outs = body()
+        ttnn.end_trace_capture(dev, tid, cq_id=0)
+        ttnn.synchronize_device(dev)
+        log(fh, ev="captured")
+        ref = None
+        for i in range(a.iters):
+            faulthandler.dump_traceback_later(a.stall_s, repeat=True)
+            t0 = time.perf_counter()
+            ttnn.execute_trace(dev, tid, cq_id=0, blocking=False)
+            ttnn.synchronize_device(dev)
+            rec = {"ev": "iter", "i": i, "ms": round((time.perf_counter() - t0) * 1e3, 2)}
+            if i % a.check_every == 0 or i == a.iters - 1:
+                h = hashlib.sha256()
+                for t in outs:
+                    h.update(ttnn.to_torch(t).float().numpy().tobytes())
+                d = h.hexdigest()[:16]
+                rec["digest"] = d
+                ref = ref or d
+                if d != ref:
+                    rec["MISMATCH"] = ref
+                    rc = 1
+            log(fh, **rec)
+        faulthandler.cancel_dump_traceback_later()
+        log(fh, ev="done", iters=a.iters, rc=rc)
+        ttnn.release_trace(dev, tid)
+    finally:
+        faulthandler.dump_traceback_later(300, repeat=True)
+        ttnn.close_device(dev)
+        faulthandler.cancel_dump_traceback_later()
+        log(fh, ev="closed")
+    return rc
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/repro_resize2d_eth.py

@@ -0,0 +1,223 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Minimal standalone repro (plain ttnn, no METEOR code): the separable bilinear x2 resize of the METEOR lift,
+captured in one trace and replayed in a tight loop. On a Blackhole p150 with ETH dispatch the replay intermittently
+never completes (logs/meteor/hangfix/ETH_DISPATCH_HANG_ISSUE.md).
+
+    bin/devrun -t 600 -- python -u repro_resize2d_eth.py --dispatch eth --iters 3000 --jsonl out.jsonl
+    bin/devrun -t 600 -- python -u repro_resize2d_eth.py --dispatch worker ...        # A/B
+
+Op chain (= ttaw.ops.upsample.Resize2d, in (400, 250) -> out (800, 500), C = 96, N = 1, fp32 operands, HiFi4 + fp32
+accumulation), on an input x [1, 1, 100000, 96] bf16 TILE in DRAM:
+
+    RM -> reshape [1, 400, 250, 96] -> TILE -> typecast fp32 -> transpose(-2, -1) [1, 400, 96, 250]
+    -> matmul A_w^T [250, 500] -> transpose(-2, -1) [1, 400, 500, 96] -> RM -> reshape [1, 1, 400, 48000] -> TILE
+    -> matmul A_h [800, 400] @ [1, 1, 400, 48000] -> [1, 1, 800, 48000] fp32 -> typecast bf16 -> RM
+    -> reshape [1, 1, 400000, 96] -> TILE
+
+``--part w`` stops after the W pass, ``--part h`` runs only the H-pass matmul (on a persistent [1, 1, 400, 48000]
+fp32 TILE input). Each replay is followed by ``synchronize_device`` and a fsync'd JSON heartbeat; outputs are hashed
+every ``--check-every`` replays; faulthandler dumps the stack when a replay stalls. No tt-triage, no op timeout.
+"""
+from __future__ import annotations
+
+import argparse
+import faulthandler
+import hashlib
+import json
+import os
+import sys
+import time
+
+import numpy as np
+
+H, W, H2, W2 = 400, 250, 800, 500
+
+
+def interp(n_in: int, n_out: int) -> np.ndarray:
+    """1-D bilinear, align_corners=False (half-pixel, clamped at 0), as F.interpolate."""
+    s = n_in / n_out
+    src = np.maximum((np.arange(n_out) + 0.5) * s - 0.5, 0.0)
+    i0 = np.minimum(np.floor(src).astype(np.int64), n_in - 1)
+    i1 = np.minimum(i0 + 1, n_in - 1)
+    lam = src - i0
+    a = np.zeros((n_out, n_in))
+    np.add.at(a, (np.arange(n_out), i0), 1.0 - lam)
+    np.add.at(a, (np.arange(n_out), i1), lam)
+    return a.astype(np.float32)
+
+
+def log(fh, **rec):
+    rec["t"] = round(time.time(), 3)
+    line = json.dumps(rec)
+    print(line, flush=True)
+    if fh:
+        fh.write(line + "\n")
+        fh.flush()
+        os.fsync(fh.fileno())
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--dispatch", choices=["eth", "worker"], default="eth")
+    ap.add_argument("--cqs", type=int, default=1)
+    ap.add_argument("--iters", type=int, default=3000)
+    ap.add_argument("--per-trace", type=int, default=1, help="resize chains per trace")
+    ap.add_argument("--part", choices=["full", "w", "h", "w_rm", "wh", "tail", "mid", "mid_h", "h_tail",
+                                           "t_typecast", "t_untilize", "t_reshape", "t_tilize", "t_reshape_tile"], default="full",
+                    help="w: up to the W-pass transpose; w_rm: + RM/reshape/TILE; wh: + H matmul (no tail); "
+                         "tail: typecast/RM/reshape/TILE of a persistent z; mid: RM/reshape/TILE of a persistent "
+                         "[1,400,500,C] y; mid_h: mid + H matmul; h_tail: H matmul + tail; t_*: ONE tail op on a "
+                         "persistent input (typecast fp32->bf16 TILE [800, 500C]; untilize bf16 [800, 500C]; RM "
+                         "reshape [800, 500C] -> [400000, C]; tilize RM [400000, C])")
+    ap.add_argument("--channels", type=int, default=96, help="C (the H-pass matmul N is 500 * C; 96 -> 48000)")
+    ap.add_argument("--dtype", choices=["float32", "bfloat16"], default="float32", help="matmul operand dtype")
+    ap.add_argument("--fidelity", choices=["HiFi4", "HiFi3", "HiFi2", "LoFi"], default="HiFi4")
+    ap.add_argument("--no-fp32-acc", action="store_true", help="fp32_dest_acc_en=False")
+    ap.add_argument("--dest-cols", type=int, default=0,
+                    help="t_reshape: destination last dim (default C: [800, 500C] -> [400000, C]); dest page = 2 B x it")
+    ap.add_argument("--reshape", choices=["rm", "tile", "tail_tile"], default="rm",
+                    help="rm: TILE -> RM -> reshape -> TILE (as Resize2d); tile: ttnn.reshape on the TILE tensor")
+    ap.add_argument("--check-every", type=int, default=200)
+    ap.add_argument("--stall-s", type=float, default=60.0)
+    ap.add_argument("--jsonl")
+    a = ap.parse_args()
+    faulthandler.enable()
+    fh = open(a.jsonl, "a") if a.jsonl else None
+    faulthandler.dump_traceback_later(600, repeat=True)
+
+    import torch
+    import ttnn
+
+    core = ttnn.DispatchCoreType.ETH if a.dispatch == "eth" else ttnn.DispatchCoreType.WORKER
+    dev = ttnn.open_device(device_id=0, dispatch_core_config=ttnn.DispatchCoreConfig(core),
+                           num_command_queues=a.cqs, l1_small_size=32768, trace_region_size=64 << 20)
+    rc = 0
+    try:
+        g = dev.compute_with_storage_grid_size()
+        C = a.channels
+        dt = ttnn.float32 if a.dtype == "float32" else ttnn.bfloat16
+        log(fh, ev="open", dispatch=a.dispatch, cqs=a.cqs, grid=f"{g.x}x{g.y}", part=a.part, per_trace=a.per_trace,
+            channels=C, n=W2 * C, dtype=a.dtype, fidelity=a.fidelity, fp32_acc=not a.no_fp32_acc,
+            reshape=a.reshape, dest_cols=a.dest_cols or None)
+        cfg = ttnn.init_device_compute_kernel_config(dev.arch(), math_fidelity=getattr(ttnn.MathFidelity, a.fidelity),
+                                                     fp32_dest_acc_en=not a.no_fp32_acc, packer_l1_acc=False,
+                                                     math_approx_mode=False)
+        to_dev = lambda arr, dt, lay: ttnn.from_torch(torch.from_numpy(np.ascontiguousarray(arr)), dtype=dt,  # noqa
+                                                      layout=lay, device=dev, memory_config=ttnn.DRAM_MEMORY_CONFIG)
+        rng = np.random.default_rng(0)
+        aw_t = to_dev(interp(W, W2).T, dt, ttnn.TILE_LAYOUT)                     # [250, 500]
+        ah = to_dev(interp(H, H2), dt, ttnn.TILE_LAYOUT)                         # [800, 400]
+        x_in = to_dev(rng.standard_normal((1, 1, H * W, C)).astype(np.float32), ttnn.bfloat16, ttnn.TILE_LAYOUT)
+        y_in = to_dev(rng.standard_normal((1, 1, H, W2 * C)).astype(np.float32), dt, ttnn.TILE_LAYOUT)
+
+        z_in = to_dev(rng.standard_normal((1, 1, H2, W2 * C)).astype(np.float32), dt, ttnn.TILE_LAYOUT) \
+            if a.part == "tail" else None
+        yw_in = to_dev(rng.standard_normal((1, H, W2, C)).astype(np.float32), dt, ttnn.TILE_LAYOUT) \
+            if a.part in ("mid", "mid_h") else None
+
+        one = None
+        if a.part.startswith("t_"):
+            z32 = rng.standard_normal((1, 1, H2, W2 * C)).astype(np.float32)
+            one = {"t_typecast": lambda: to_dev(z32, ttnn.float32, ttnn.TILE_LAYOUT),
+                   "t_untilize": lambda: to_dev(z32, ttnn.bfloat16, ttnn.TILE_LAYOUT),
+                   "t_reshape": lambda: to_dev(z32, ttnn.bfloat16, ttnn.ROW_MAJOR_LAYOUT),
+                   "t_tilize": lambda: to_dev(z32.reshape(1, 1, H2 * W2, C), ttnn.bfloat16, ttnn.ROW_MAJOR_LAYOUT),
+                   "t_reshape_tile": lambda: to_dev(z32, ttnn.bfloat16, ttnn.TILE_LAYOUT),
+                   }[a.part]()
+
+        def tail(z):
+            if z.dtype != ttnn.bfloat16:
+                z = ttnn.typecast(z, ttnn.bfloat16)
+            if a.reshape in ("tile", "tail_tile"):
+                return ttnn.reshape(z, (1, 1, H2 * W2, C))
+            z = ttnn.to_layout(z, ttnn.ROW_MAJOR_LAYOUT)
+            z = ttnn.reshape(z, (1, 1, H2 * W2, C))
+            return ttnn.to_layout(z, ttnn.TILE_LAYOUT)
+
+        def mid(y):
+            if a.reshape == "tile":
+                return ttnn.reshape(y, (1, 1, H, W2 * C))
+            y = ttnn.to_layout(y, ttnn.ROW_MAJOR_LAYOUT)
+            y = ttnn.reshape(y, (1, 1, H, W2 * C))
+            return ttnn.to_layout(y, ttnn.TILE_LAYOUT)
+
+        def chain():
+            if a.part == "t_typecast":
+                return ttnn.typecast(one, ttnn.bfloat16)
+            if a.part == "t_untilize":
+                return ttnn.to_layout(one, ttnn.ROW_MAJOR_LAYOUT)
+            if a.part == "t_reshape":
+                d = a.dest_cols or C
+                return ttnn.reshape(one, (1, 1, H2 * W2 * C // d, d))
+            if a.part == "t_reshape_tile":
+                return ttnn.reshape(one, (1, 1, H2 * W2, C))
+            if a.part == "t_tilize":
+                return ttnn.to_layout(one, ttnn.TILE_LAYOUT)
+            if a.part == "h":
+                return ttnn.matmul(ah, y_in, compute_kernel_config=cfg)
+            if a.part == "h_tail":
+                return tail(ttnn.matmul(ah, y_in, compute_kernel_config=cfg))
+            if a.part == "tail":
+                return tail(z_in)
+            if a.part == "mid":
+                return mid(yw_in)
+            if a.part == "mid_h":
+                return ttnn.matmul(ah, mid(yw_in), compute_kernel_config=cfg)
+            x = ttnn.to_layout(x_in, ttnn.ROW_MAJOR_LAYOUT)
+            x = ttnn.reshape(x, (1, H, W, C))
+            x = ttnn.to_layout(x, ttnn.TILE_LAYOUT)
+            if dt != ttnn.bfloat16:
+                x = ttnn.typecast(x, dt)
+            xt = ttnn.transpose(x, -2, -1)
+            y = ttnn.matmul(xt, aw_t, compute_kernel_config=cfg)
+            y = ttnn.transpose(y, -2, -1)
+            if a.part == "w":
+                return y
+            y = mid(y)
+            if a.part == "w_rm":
+                return y
+            z = ttnn.matmul(ah, y, compute_kernel_config=cfg)
+            if a.part == "wh":
+                return z
+            return tail(z)
+
+        warm = [chain() for _ in range(a.per_trace)]          # compile, then capture
+        ttnn.synchronize_device(dev)
+        del warm
+        tid = ttnn.begin_trace_capture(dev, cq_id=0)
+        outs = [chain() for _ in range(a.per_trace)]
+        ttnn.end_trace_capture(dev, tid, cq_id=0)
+        ttnn.synchronize_device(dev)
+        log(fh, ev="captured")
+        ref = None
+        for i in range(a.iters):
+            faulthandler.dump_traceback_later(a.stall_s, repeat=True)
+            t0 = time.perf_counter()
+            ttnn.execute_trace(dev, tid, cq_id=0, blocking=False)
+            ttnn.synchronize_device(dev)
+            rec = {"ev": "iter", "i": i, "ms": round((time.perf_counter() - t0) * 1e3, 2)}
+            if i % a.check_every == 0 or i == a.iters - 1:
+                h = hashlib.sha256()
+                for t in outs:
+                    h.update(ttnn.to_torch(t).float().numpy().tobytes())
+                d = h.hexdigest()[:16]
+                rec["digest"] = d
+                ref = ref or d
+                if d != ref:
+                    rec["MISMATCH"] = ref
+                    rc = 1
+            log(fh, **rec)
+        faulthandler.cancel_dump_traceback_later()
+        log(fh, ev="done", iters=a.iters, rc=rc)
+        ttnn.release_trace(dev, tid)
+    finally:
+        faulthandler.dump_traceback_later(300, repeat=True)
+        ttnn.close_device(dev)
+        faulthandler.cancel_dump_traceback_later()
+        log(fh, ev="closed")
+    return rc
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/run_public_frames.py

@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Run the served model (``METEOR.from_pretrained``: the ``frame`` trace, ETH 12x10) on the public-dataset frames of
+``research/meteor/public_data`` and write the TT outputs in the layout of the CPU goldens (``fNNNN.npz`` compact +
+``fNNNN.json`` decoded, ``run_public_reference.save_frame``), so ``compare_tt.py`` scores them against the CPU
+goldens and ``make_demo.py`` renders them. Development tool of the workspace (needs the research scripts and the
+converted public inputs, which never ship; nuScenes is CC BY-NC-SA: local validation only).
+
+    bin/devrun -t 1800 -- python -u code/scripts/run_public_frames.py --out logs/public_tt [--ids ps019,ps090,ns0103]
+
+Every frame's graph feed is checked against the golden's ``input_sha256`` first (the same pixels, K, T and v0 as the
+CPU reference). Frames are processed per sequence (one rig each: the lift tables are written once per sequence).
+Also writes ``<out>/run.json`` (versions, device, per-frame seconds) and, with ``--full`` frames, the 19 outputs.
+"""
+from __future__ import annotations
+
+import argparse
+import glob
+import json
+import os
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+ROOT = Path(os.environ.get("TT_MODELS_ROOT", "/home/ubuntu/experiments/tt-models"))
+PUBLIC = ROOT / "research" / "meteor" / "public_data"
+sys.path.insert(0, str(PUBLIC / "scripts"))
+sys.path.insert(0, str(ROOT / "research" / "meteor" / "scripts"))
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--ids", default="ps019,ps090,ns0103")
+    ap.add_argument("--full", default="ps019:20,ps090:20,ns0103:9", help="id:frame pairs that also get full_fNNNN.npz")
+    a = ap.parse_args()
+    import me_public_common as C  # noqa: F401  (research decode, metrics)
+    import meteor_io
+    from run_public_reference import save_frame
+
+    from tt_meteor import METEOR, __version__
+    from tt_meteor.host.preprocess import MeteorFrame
+
+    full = {tuple(p.split(":")) for p in a.full.split(",") if p}
+    jobs = []
+    for gid in a.ids.split(","):
+        gdir = PUBLIC / "golden" / gid
+        frames = sorted(int(Path(f).stem[1:]) for f in glob.glob(str(gdir / "f[0-9][0-9][0-9][0-9].json")))
+        jobs.append((gid, frames))
+    run = {"bundle_version": __version__, "frames": {}, "started": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())}
+    t_load = time.time()
+    with METEOR.from_pretrained() as model:
+        run["load_s"] = round(time.time() - t_load, 1)
+        run["device"] = {k: model.device_info.get(k) for k in ("dispatch", "grid", "num_command_queues", "cores")}
+        print("device", run["device"], "load", run["load_s"], "s", flush=True)
+        for gid, frames in jobs:
+            root = PUBLIC / "inputs" / gid
+            sd = meteor_io.scene_dir_of(str(root)) if hasattr(meteor_io, "scene_dir_of") else \
+                str(root / (root / "scenes.txt").read_text().split()[0])
+            man = meteor_io.load_manifest(sd)
+            ego_gt = dict(np.load(os.path.join(sd, "ego_motion.npz")))
+            present = np.asarray(man.get("present_mask", [1] * 8), bool)
+            out_dir = a.out / gid
+            out_dir.mkdir(parents=True, exist_ok=True)
+            for i in frames:
+                feed = meteor_io.load_frame(sd, i, man)
+                gold = json.loads((PUBLIC / "golden" / gid / f"f{i:04d}.json").read_text())
+                sha = {k: C.sha256_array(v) for k, v in feed.items()}
+                if sha != gold["input_sha256"]:
+                    raise SystemExit(f"{gid} f{i}: the feed differs from the golden's input_sha256")
+                fr = MeteorFrame(feed["imgs"], feed["K"], feed["T_cam_ego"], feed["v0"], present)
+                t0 = time.time()
+                o = {k: np.asarray(v) for k, v in model._forward({"frame": fr}).items()}
+                dt = time.time() - t0
+                save_frame(str(out_dir), i, o, feed, man, ego_gt, full=(gid, str(i)) in full)
+                run["frames"][f"{gid}/f{i:04d}"] = round(dt, 3)
+                print(f"{gid} f{i:04d}: {dt * 1e3:.0f} ms", flush=True)
+    run["finished"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
+    (a.out / "run.json").write_text(json.dumps(run, indent=1) + "\n")
+
+
+if __name__ == "__main__":
+    main()

code/scripts/stress_frames.py

@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Contained stress runs of the ``frame`` trace (the METEOR hang investigation, RESET_INVESTIGATION.md section 6).
+
+    METEOR_DEVICE_OK=1 bin/devrun -t 900 -- python -u code/scripts/stress_frames.py --mode replay --frames 200
+    METEOR_DEVICE_OK=1 bin/devrun -t 1500 -- python -u code/scripts/stress_frames.py --mode rigs --frames 100
+
+``--mode replay`` (R1): the model is built with the sample rig's tables and cameras as its warm-up data, captured,
+then the frame trace is replayed ``--frames`` times with NO upload at all, reading back every ``--read-every``-th
+replay. ``--mode rigs`` (R2): the model is built like the API (zero tables), then every frame cycles to the next of
+the golden rigs: table write (when the rig changes) + camera upload + replay + segmented readback.
+
+Every read is checked: bit-identical to the first read of the same rig (corruption / non-determinism shows up as a
+mismatch) and agreement with the CPU golden (``lane`` argmax, ``hm`` PCC).
+
+``--rigs shipped[:N]`` needs no golden (for the container, which has none): N rigs (default 4) made from the shipped
+synthetic sample, rig k with every camera raised by 5k cm (a different calibration, so different lift tables, and the
+same cameras); only the bit-identity per rig is checked. Inside the container image (``/opt/tt-metal``)::
+
+    python -u /opt/tt-metal/scripts/stress_frames.py --mode rigs --rigs shipped --frames 200 One heartbeat line per frame goes to
+stdout (run with ``python -u``) and one JSON line to ``--jsonl`` (fsync'd); ``faulthandler`` dumps the Python stack
+when no frame finishes for ``--stall-s`` seconds (a hang leaves evidence of where it stopped, without touching the
+device). Nothing here arms tt-triage or an operation timeout.
+"""
+from __future__ import annotations
+
+import argparse
+import faulthandler
+import hashlib
+import json
+import os
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+GOLDENS = Path(os.environ.get("METEOR_GOLDENS", "/home/ubuntu/experiments/tt-models/research/meteor/goldens"))
+RIGS = ["pandaset_019_f40", "pandaset_090_f40", "nuscenes_0103_kf09", "meteor_valday_f040"]
+INPUT_KEYS = ("input.imgs", "input.K", "input.T_cam_ego", "input.v0", "input.present")
+CHECK_KEYS = ("lane", "hm", "ego")
+
+
+def log(fh, **rec) -> None:
+    rec["t"] = round(time.time(), 3)
+    line = json.dumps(rec, default=str)
+    print(line, flush=True)
+    if fh is not None:
+        fh.write(line + "\n")
+        fh.flush()
+        os.fsync(fh.fileno())
+
+
+def digest(out) -> str:
+    h = hashlib.sha256()
+    for k in sorted(out):
+        a = np.ascontiguousarray(out[k])
+        h.update(k.encode())
+        h.update(a.tobytes())
+    return h.hexdigest()[:16]
+
+
+def checks(out, gold) -> dict:
+    from tt_meteor.ttaw.metrics import pcc
+
+    lane = float((np.asarray(out["lane"]).reshape(-1) == np.asarray(gold["lane"]).reshape(-1)).mean())
+    hm = float(pcc(np.asarray(out["hm"], np.float64), np.asarray(gold["hm"], np.float64)))
+    return {"lane_agree": round(lane, 5), "hm_pcc": round(hm, 6)}
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--mode", choices=["replay", "rigs", "segrigs"], required=True)
+    ap.add_argument("--frames", type=int, default=100)
+    ap.add_argument("--read-every", type=int, default=1)
+    ap.add_argument("--rigs", default=",".join(RIGS))
+    ap.add_argument("--dispatch", default="eth", choices=["eth", "worker"])
+    ap.add_argument("--open-cqs", type=int, default=None, help="HW command queues to open (default: DEVICE_DEFAULTS)")
+    ap.add_argument("--runner-cqs", type=int, default=None, help="CQs the TraceRunner uses (default: as opened)")
+    ap.add_argument("--lift-reps", type=int, default=1,
+                    help="segrigs: replay the lift segment(s) this many times per frame (sync + heartbeat after each)")
+    ap.add_argument("--split-lift", nargs="?", const=True, default=False,
+                    help="segrigs: lift as lift_gs (grid_samples) + lift_rest; 'fine': lift_rest as 4 op groups")
+    ap.add_argument("--no-tables", action="store_true", help="segrigs: one rig only, tables written once")
+    ap.add_argument("--stall-s", type=float, default=120.0)
+    ap.add_argument("--jsonl")
+    a = ap.parse_args()
+
+    faulthandler.enable()
+    fh = open(a.jsonl, "a") if a.jsonl else None
+    rigs = a.rigs.split(",")
+    faulthandler.dump_traceback_later(900, repeat=True)          # build / warm-up / capture
+    log(fh, ev="start", mode=a.mode, frames=a.frames, read_every=a.read_every, rigs=rigs, pid=os.getpid())
+
+    import ttnn
+
+    from tt_meteor.device import close_device, describe_device, open_device
+    from tt_meteor.host.calib import lift_geometry
+    from tt_meteor.host.preprocess import MeteorFrame
+    from tt_meteor.tt.lift import lift_tables
+    from tt_meteor.tt.model import TtMETEOR
+    from tt_meteor.tt.params import MeteorParams
+    from tt_meteor.tt.unpack import outputs_from_device
+
+    t0 = time.perf_counter()
+    params = MeteorParams.load()
+    data = {}
+    if len(rigs) == 1 and rigs[0].startswith("shipped"):
+        from tt_meteor.api import load_sample
+        from tt_meteor.host.inputs import SAMPLES_DIR, prepare_request
+
+        n = int(rigs[0].partition(":")[2] or 4)
+        kw = load_sample(SAMPLES_DIR / "synthetic_8cam.json")
+        base = prepare_request(kw["images"], kw["calibration"], kw["ego_speed"], None)
+        rigs = [f"shipped{k}" for k in range(n)]
+        for k, name in enumerate(rigs):
+            shift = np.eye(4, dtype=np.float32)
+            shift[2, 3] = -0.05 * k                 # ego -> ego lowered by 5k cm = every camera raised by 5k cm
+            T = (base.T_cam_ego.astype(np.float64) @ shift).astype(np.float32)
+            fr = MeteorFrame(base.imgs, base.K, T, base.v0, base.present)
+            data[name] = (fr, lift_geometry(fr.K, fr.T_cam_ego, points=params.ground_points()), None)
+    for name in ([] if data else rigs):
+        with np.load(GOLDENS / name / "taps.npz") as z:
+            g = {k: z[k] for k in INPUT_KEYS + CHECK_KEYS}
+        fr = MeteorFrame(g["input.imgs"], g["input.K"], g["input.T_cam_ego"], g["input.v0"], g["input.present"])
+        geom = lift_geometry(fr.K, fr.T_cam_ego, points=params.ground_points())
+        data[name] = (fr, geom, g)
+    log(fh, ev="host_ready", s=round(time.perf_counter() - t0, 1), keys={n: d[1].key for n, d in data.items()})
+
+    dev = open_device(dispatch=a.dispatch, allow_fallback=False, num_command_queues=a.open_cqs)
+    rc = 0
+    try:
+        log(fh, ev="device_open", info=describe_device(dev))
+        first = rigs[0]
+        fr0, geom0, _ = data[first]
+        if a.mode == "replay":
+            warm = {"imgs": TtMETEOR.image_rows(fr0.imgs), "v0": float(np.asarray(fr0.v0).reshape(-1)[0]),
+                    **lift_tables(geom0)}
+            tt = TtMETEOR(dev, params, warmup=warm, tables_key=geom0.key, num_command_queues=a.runner_cqs)
+        elif a.mode == "segrigs":
+            tt = TtMETEOR(dev, params, segmented=True, split_lift=a.split_lift, num_command_queues=a.runner_cqs)
+        else:
+            tt = TtMETEOR(dev, params, num_command_queues=a.runner_cqs)
+        t1 = time.perf_counter()
+        tt.capture()
+        ttnn.synchronize_device(dev)
+        log(fh, ev="captured", s=round(time.perf_counter() - t1, 1), timings=tt.runner.timings_ms,
+            program_cache_entries=tt.describe().get("program_cache_entries"))
+        faulthandler.dump_traceback_later(a.stall_s, repeat=True)
+        ref_digest = {}
+        mismatches = 0
+        for i in range(a.frames):
+            faulthandler.dump_traceback_later(a.stall_s, repeat=True)   # re-armed per frame: fires only on a stall
+            ts = time.perf_counter()
+            if a.mode == "replay":
+                name = first
+                tt.runner.replay("frame")
+                ttnn.synchronize_device(dev)
+                rec = {"ev": "frame", "i": i, "rig": name, "replay_ms": round((time.perf_counter() - ts) * 1e3, 1)}
+                if i % a.read_every == 0 or i == a.frames - 1:
+                    tr = time.perf_counter()
+                    out = outputs_from_device(tt.unpack(tt.runner.read("frame")))
+                    rec["read_ms"] = round((time.perf_counter() - tr) * 1e3, 1)
+                else:
+                    out = None
+            elif a.mode == "segrigs":
+                name = rigs[0] if a.no_tables else rigs[i % len(rigs)]
+                fr, geom, _ = data[name]
+                tw = time.perf_counter()
+                wrote = tt.set_tables(geom)
+                rec = {"ev": "frame", "i": i, "rig": name, "tables": wrote,
+                       "tables_ms": round((time.perf_counter() - tw) * 1e3, 1)}
+                r = tt.runner
+                for k, stage in enumerate(tt.segments):
+                    ta = time.perf_counter()
+                    print(f"[seg] i={i} {stage} start", flush=True)
+                    if k == 0:
+                        r.run("seg_image", inputs={"imgs": tt.image_rows(fr.imgs)}, params=tt.frame_params(fr))
+                    else:
+                        r.run(f"seg_{stage}")
+                    ttnn.synchronize_device(dev)
+                    if stage.startswith(("lift", "lr_")):
+                        for rep in range(1, a.lift_reps):
+                            print(f"[seg] i={i} {stage} rep {rep} start", flush=True)
+                            r.run(f"seg_{stage}")
+                            ttnn.synchronize_device(dev)
+                    rec[f"{stage}_ms"] = round((time.perf_counter() - ta) * 1e3, 1)
+                ego = np.array(r.read("seg_head"), copy=True)
+                out = {"ego": ego}
+            else:
+                name = rigs[i % len(rigs)]
+                fr, geom, _ = data[name]
+                tw = time.perf_counter()
+                wrote = tt.set_tables(geom)
+                tables_ms = (time.perf_counter() - tw) * 1e3
+                tr = time.perf_counter()
+                out = tt.run_frame(fr, geom)
+                rec = {"ev": "frame", "i": i, "rig": name, "tables": wrote, "tables_ms": round(tables_ms, 1),
+                       "run_ms": round((time.perf_counter() - tr) * 1e3, 1)}
+            if out is not None:
+                d = digest(out)
+                rec["digest"] = d
+                if name not in ref_digest:
+                    ref_digest[name] = d
+                    if "lane" in out and data[name][2] is not None:
+                        rec.update(checks(out, data[name][2]))
+                elif d != ref_digest[name]:
+                    mismatches += 1
+                    rec["MISMATCH_vs"] = ref_digest[name]
+                    if "lane" in out and data[name][2] is not None:
+                        rec.update(checks(out, data[name][2]))
+            log(fh, **rec)
+        faulthandler.cancel_dump_traceback_later()
+        log(fh, ev="done", frames=a.frames, mismatches=mismatches, digests=ref_digest,
+            s=round(time.perf_counter() - t0, 1))
+        rc = 1 if mismatches else 0
+        tt.release()
+    finally:
+        faulthandler.dump_traceback_later(300, repeat=True)
+        close_device(dev)
+        faulthandler.cancel_dump_traceback_later()
+        log(fh, ev="closed", rc=rc)
+    return rc
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/tt_meteor/__init__.py

@@ -0,0 +1,40 @@
+# SPDX-License-Identifier: Apache-2.0
+"""METEOR (TIER IV, AutowareFoundation/meteor) on a Tenstorrent Blackhole p150 (tt-nn), packaged as changh95/meteor-p150.
+
+Python API (see PYTHON.md)::
+
+    from tt_meteor import METEOR, load_sample
+
+    with METEOR.from_pretrained(device_id=0) as model:
+        out = model(**load_sample("code/tt_meteor/samples/synthetic_8cam.json"))
+
+Importing the package has no side effects (no device, no ttnn / torch import, no network); the names below load
+on first use. The HTTP server is ``tt_meteor.server.app:app`` (SERVING.md). ``tt_meteor.ttaw`` is the
+vendored shared package of the Autoware ports (``ttaw/VENDORED.json`` records its version and file hashes; never
+edit it here).
+"""
+
+__version__ = "0.1.0"
+__all__ = ["METEOR", "Output", "open_device", "load_points", "load_image", "load_sample", "__version__"]
+
+_LAZY = {
+    "METEOR": (".api", "METEOR"),
+    "Output": (".api", "Output"),
+    "open_device": (".device", "open_device"),
+    "load_points": (".io", "load_points"),
+    "load_image": (".io", "load_image"),
+    "load_sample": (".api", "load_sample"),
+}
+
+
+def __getattr__(name):
+    if name in _LAZY:
+        import importlib
+
+        module, attr = _LAZY[name]
+        return getattr(importlib.import_module(module, __name__), attr)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+    return sorted(list(globals()) + __all__)

code/tt_meteor/api.py

@@ -0,0 +1,205 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Python API: METEOR (TIER IV, AutowareFoundation/meteor) on one Tenstorrent Blackhole p150.
+
+    from tt_meteor import METEOR, load_sample
+
+    with METEOR.from_pretrained(device_id=0) as model:  # weights -> HF cache, device open, traces captured
+        out = model(**load_sample("code/tt_meteor/samples/synthetic_8cam.json"))  # 8 cameras + calibration + speed
+    print(out.to_dict())  # the same JSON as POST /predict
+
+The contract shared by every bundle of the Autoware collection is the vendored ``ttaw.api_base.ModelBase``
+(BUNDLE_CONVENTIONS.md section 8): ``from_pretrained`` resolves the pinned weights before it claims the chip, opens it
+(ETH dispatch, 12x10), builds the graph and captures every trace variant in ``warmup_variants``, so the first call is
+as fast as the later ones; calls are serialised by a lock (one chip, batch 1) and fill ``timing_ms``; ``close()`` is
+idempotent, also runs at interpreter exit, and closes the chip only if the model opened it. The HTTP server
+(``tt_meteor.server.app``) calls this class, so ``/predict`` and ``model(...)`` agree bit for bit.
+
+Inputs per call (``_prepare``; host code in ``tt_meteor.host``, METEOR's own runtime semantics: there is no Autoware
+package):
+
+- ``images``: the cameras ``CAM_FRONT_WIDE``, ``CAM_FRONT_LEFT``, ``CAM_FRONT_RIGHT``, ``CAM_BACK_WIDE``,
+  ``CAM_BACK_LEFT``, ``CAM_BACK_RIGHT``, ``CAM_FRONT_NARROW``, ``CAM_BACK_NARROW`` (any order; raw, unrectified frames
+  of at least 768x432, resized with OpenCV INTER_AREA semantics); the two narrow cameras may be absent (zero image +
+  donor pose, as trained);
+- ``calibration``: per camera ``intrinsics`` (of the image as sent) and ``T_ref_from_camera`` (camera optical frame ->
+  ego / base_link), or ``{"preset": name}``;
+- ``ego_speed``: m/s (the graph's ``v0``);
+- ``stream``: ``{"id", "reset", "timestamp_s", "T_world_from_ego"}`` (or ``"pose": [x, y, yaw]``): the host temporal
+  post-processing per stream (BEV seg fusion needs the pose; yaw smoothing; mode hysteresis).
+
+Runtime params (``RUNTIME_PARAMS``): every knob of METEOR's C++ renderer (``host.postprocess.PostConfig``). Load-time
+knobs: :data:`KNOBS` (``METEOR_*``): ``INPUT_NORM`` (``onnx`` = the released /255, D12; ``imagenet`` = the trained
+normalisation), ``DEPTH_MEAN_BINS`` (``log`` as exported; ``linear``), ``MAX_STREAMS``.
+
+The device graph is ``tt_meteor.tt.model.TtMETEOR``: the whole network in ONE trace (``frame``), replayed per call;
+the host half (``_build_host``, ``_prepare``, ``_postprocess``) is what the host tests exercise. Importing this module
+has no side effects (device code is imported in ``_build``).
+"""
+from __future__ import annotations
+
+from collections import OrderedDict
+from typing import Any, Dict, Mapping, Optional
+
+from . import io as tio
+from .device import DEVICE_DEFAULTS
+from .host.outputs import LABELS as _LABELS
+from .host.outputs import MeteorOutput
+from .reference.config import CAMERAS
+from .ttaw.api_base import ModelBase
+from .ttaw.knobs import Knob, Knobs
+
+__all__ = ["METEOR", "Output", "KNOBS", "load_sample"]
+
+# The result class of this model (the multi-task MeteorOutput of host.outputs; BUNDLE_CONVENTIONS 7.4 "per head").
+Output = MeteorOutput
+
+# Load-time switches (read once at build; env METEOR_<NAME>). Each optimization adds its knob here (A/B switch).
+KNOBS = Knobs("METEOR", [
+    Knob("INPUT_NORM", "onnx", "input normalisation: the released graph's /255 (D12 default, ORT parity) or the "
+                               "trained ImageNet mean/std (SPEC section 10 risk 1)", choices=("onnx", "imagenet")),
+    Knob("DEPTH_MEAN_BINS", "log", "depth_mean bin centres: log-spaced as exported (parity) or linear 1 + 1.25 b "
+                                   "(the trained bins, SPEC section 10 risk 2)", choices=("log", "linear")),
+    Knob("MAX_STREAMS", 16, "host temporal states kept (seg fusion, yaw tracks, mode); the least recent is dropped"),
+    Knob("IMAGE_PRECISION", "terms3", "image branch (ResNet-34, FPN, image heads): terms3 = fp32 activations, every "
+                                      "conv as three bf16 terms (needed by the depth / seg2d argmax gates, PORT_LOG.md "
+                                      "E20); bf16 = bf16 activations (A/B only: below the depth argmax gate, "
+                                      "0.9797 < 0.99)", choices=("terms3", "bf16")),
+])
+
+
+def load_sample(path: Any, *, with_stream: bool = True) -> Dict[str, Any]:
+    """``model(**load_sample("samples/<name>.json"))``: cameras + calibration preset + ego speed (+ stream)."""
+    from .host.inputs import load_sample as _load
+
+    return _load(path, with_stream=with_stream)
+
+
+class METEOR(ModelBase):
+    """METEOR (TIER IV, AutowareFoundation/meteor) on one Blackhole p150. Create it with :meth:`from_pretrained`."""
+
+    MODEL_NAME = "meteor-p150"
+    ENV_PREFIX = "METEOR"  # prefix of the environment knobs (SERVING.md section 3.4)
+    DEFAULT_REPO = "AutowareFoundation/meteor"
+    DEFAULT_TAG = "v1.0"  # the AutowareFoundation release tag (no Autoware ansible pin exists for METEOR)
+    # the commit DEFAULT_TAG points to (pinned: tags can move)
+    DEFAULT_REVISION = "01a5f6d71df5ecbbb5853ec600825481d57b9c6b"
+    ALLOW_PATTERNS = ["meteor_v157c3Z.onnx", "meteor_v157.param.yaml", "LICENSE", "SHA256SUMS"]
+    VARIANTS = ["default"]  # load-time: selects weights files and trace shapes
+    DEFAULT_VARIANT = "default"
+    INPUT_KIND = "multicam"  # lidar | camera | multicam | lidar+multicam | planner
+    CAMERA_ORDER = CAMERAS  # the network's fixed camera order (meteor_v157.param.yaml:10)
+    POINT_FIELDS = tio.DEFAULT_POINT_FIELDS  # unused (camera model); kept for the shared /info schema
+    LABELS = list(_LABELS)  # 3D box classes (label_id 0 vehicle, 1 VRU)
+    # Per-request knobs: name -> (type, min, max, default). Host-side post-processing only (SPEC section 9, RT-host;
+    # render.cpp:345-376 defaults).
+    RUNTIME_PARAMS = {
+        "det3d_threshold": (float, 0.0, 1.0, 0.15),
+        "det3d_topk": (int, 1, 1024, 64),
+        "vehicle_threshold": (float, 0.0, 1.0, 0.35),
+        "vru_threshold": (float, 0.0, 1.0, 0.15),
+        "bev_nms_iou": (float, 0.0, 1.0, 0.3),
+        "bev_nms_containment": (float, 0.0, 1.0, 0.6),
+        "stationary_logit_threshold": (float, None, None, 0.0),
+        "det2d_threshold": (float, 0.0, 1.0, 0.30),
+        "det2d_topk": (int, 1, 1024, 48),
+        "det2d_hide": (str, None, None, "7"),
+        "unk2d": (bool, None, None, True),
+        "unk2d_threshold": (float, 0.0, 1.0, None),
+        "ground_z": (float, -5.0, 5.0, 0.0),
+        "mode_hysteresis": (float, 0.0, 10.0, 0.35),
+        "straight_margin": (float, 0.0, 10.0, 1.0),
+        "seg_fuse": (bool, None, None, True),
+        "thin_road_edge": (bool, None, None, True),
+        "yaw_smoothing": (bool, None, None, True),
+        "heads": (bool, None, None, False),
+    }
+    EXTRA_INPUTS = ("ego_speed",)
+    DEVICE_DEFAULTS = DEVICE_DEFAULTS  # validated open parameters (device.py)
+
+    # ---- port-specific hooks (called by ModelBase; keep host work out of _forward) ---------------------------
+    def _build(self) -> None:
+        """Weights (``reference.weights.MeteorWeights``: the ONNX parameters by consuming node) -> the device graph of
+        ``tt_meteor.tt`` (``TtMETEOR``: the whole graph as ONE ``TraceRunner`` variant ``frame``, no capture yet)."""
+        self._build_host()
+        from .host.calib import CalibrationCache
+        from .reference.weights import MeteorWeights
+        from .tt.model import TtMETEOR
+        from .tt.params import MeteorParams
+
+        params = MeteorParams(MeteorWeights(self.weights_path), self.cfg)
+        self.calib_cache = CalibrationCache(points=params.ground_points())
+        self.tt = TtMETEOR(self.device, params, num_command_queues=self.device_info.get("num_command_queues"),
+                           image_precision=str(self.knobs.IMAGE_PRECISION))
+        self.runner = self.tt.runner
+
+    def _warm_one(self, variant: Any) -> None:
+        """Warm-up of the ``frame`` variant (compiles every program) and its capture."""
+        self.runner.capture()
+
+    def _build_host(self) -> None:
+        """The host half of the model: load-time knobs and the per-stream host temporal states (no device, no
+        weights; the host tests call it directly)."""
+        from .host.postprocess import PostConfig
+        from .reference.config import default_config
+
+        params = {k.upper(): v for k, v in getattr(self, "compile_params", {}).items() if k.upper() in KNOBS.knobs}
+        self.knobs = KNOBS.read(**params)
+        self.cfg = default_config(input_norm=str(self.knobs.INPUT_NORM),
+                                  depth_mean_bins=str(self.knobs.DEPTH_MEAN_BINS))
+        self.post = PostConfig()
+        self._streams: "OrderedDict[str, Any]" = OrderedDict()
+
+    def _stream_state(self, stream: Optional[Mapping[str, Any]]):
+        from .host.temporal import StreamState
+
+        sid = str((stream or {}).get("id", "default"))
+        st = self._streams.get(sid)
+        if st is None:
+            st = self._streams[sid] = StreamState()
+            while len(self._streams) > int(self.knobs.MAX_STREAMS):
+                self._streams.popitem(last=False)
+        elif (stream or {}).get("reset"):
+            st.reset()
+        self._streams.move_to_end(sid)
+        return sid, st
+
+    def _prepare(self, points: Any = None, *, images: Any = None, calibration: Any = None, stream: Any = None,
+                 ego_speed: Any = None, **_: Any) -> Dict[str, Any]:
+        """Cameras + calibration + ego speed -> the graph feed (``host.inputs.prepare_request``)."""
+        from .host.inputs import prepare_request
+
+        if points is not None:
+            raise tio.InputError("this model takes eight camera images (+ calibration, ego_speed), not a point cloud")
+        frame = prepare_request(images, calibration, ego_speed, stream)
+        sid, state = self._stream_state(stream)
+        return {"frame": frame, "stream_id": sid, "state": state}
+
+    def _forward(self, prepared: Dict[str, Any]) -> Any:
+        """Upload (cameras, v0; the lift tables when the rig changed) + replay of the ``frame`` trace + one packed
+        read -> the 19 outputs (``tt.unpack``: ONNX names / layouts / dtypes)."""
+        frame = prepared["frame"]
+        geom = self.calib_cache.get(frame.K, frame.T_cam_ego)
+        return self.tt.run_frame(frame, geom)
+
+    def _postprocess(self, raw: Any, prepared: Dict[str, Any], params: Dict[str, Any]) -> Output:
+        """METEOR's C++ host decode on the 19 outputs (``host.result.build_output``) with this stream's state."""
+        from .host.result import build_output
+
+        hide = tuple(int(v) for v in str(params.get("det2d_hide", "7")).replace(" ", "").split(",") if v != "")
+        knobs = {k: v for k, v in params.items() if k not in ("det2d_hide", "heads")}
+        cfg = self.post.with_params(det2d_hide=hide, **knobs)
+        return build_output(raw, prepared["frame"], cfg, prepared["state"], heads=bool(params.get("heads")),
+                            model=self.MODEL_NAME, meta={"stream_id": prepared["stream_id"]})
+
+    def _release(self) -> None:
+        """Release the traces and persistent device tensors; also called when ``from_pretrained`` fails half-way."""
+        runner = getattr(self, "runner", None)
+        if runner is not None:
+            runner.release()
+
+    def extra_info(self) -> Dict[str, Any]:
+        runner = getattr(self, "runner", None)
+        info: Dict[str, Any] = {"knobs": self.knobs.as_dict()} if hasattr(self, "knobs") else {}
+        if runner is not None:
+            info["trace"] = runner.describe()
+        return info

code/tt_meteor/calib/README.md

@@ -0,0 +1,13 @@
+# Calibration presets
+
+`<sensor_kit>.json` files in the calibration envelope of SERVING.md section 3.3 (`{"frame_id", "cameras": {name:
+{"intrinsics", "T_ref_from_camera", "image_size"}}, "T_ref_from_lidar"}`), e.g. the camera rig of the public sample
+data. The server lists them in `/info` (`calibration_presets`) and a request may say `"calibration": {"preset":
+"<name>"}` instead of sending the matrices. Cite the source file and commit of every number.
+
+| preset | rig | source |
+|---|---|---|
+| `synthetic_8cam.json` | the generated rig of the shipped sample `samples/synthetic_8cam.json`: METEOR's eight slots, K at 768x432 (98 deg wide front / back and corner cameras, 30.4 deg narrow cameras, principal point at the image centre, fy = 0.87 fx), `T_ref_from_camera` = camera optical frame -> base_link (x forward, y left, z up, origin on the road), mounted 1.9 m high, corner cameras at +-60 / +-120 deg yaw pitched 25 deg down; `_rig` lists the round numbers | `code/scripts/make_synthetic_sample.py` (no third-party data), Apache-2.0 |
+
+The PandaSet 019 rig (`pandaset_019.json`, CC BY 4.0 + PandaSet Dataset Terms) belongs to the staged PandaSet sample
+and is not shipped yet (`staging_samples_pandaset/calib/`).

code/tt_meteor/calib/synthetic_8cam.json

@@ -0,0 +1,464 @@
+{
+ "frame_id": "base_link",
+ "_source": "generated by code/scripts/make_synthetic_sample.py (a generic METEOR-like 8-camera rig of round numbers; no third-party data), Apache-2.0",
+ "_rig": {
+  "CAM_FRONT_WIDE": {
+   "x": 2.1,
+   "y": 0.0,
+   "z": 1.9,
+   "yaw_deg": 0.0,
+   "pitch_down_deg": 2.0,
+   "hfov_deg": 98.0
+  },
+  "CAM_FRONT_LEFT": {
+   "x": 1.9,
+   "y": 0.85,
+   "z": 1.9,
+   "yaw_deg": 60.0,
+   "pitch_down_deg": 25.0,
+   "hfov_deg": 98.0
+  },
+  "CAM_FRONT_RIGHT": {
+   "x": 1.9,
+   "y": -0.85,
+   "z": 1.9,
+   "yaw_deg": -60.0,
+   "pitch_down_deg": 25.0,
+   "hfov_deg": 98.0
+  },
+  "CAM_BACK_WIDE": {
+   "x": -0.9,
+   "y": 0.0,
+   "z": 1.9,
+   "yaw_deg": 180.0,
+   "pitch_down_deg": 2.0,
+   "hfov_deg": 98.0
+  },
+  "CAM_BACK_LEFT": {
+   "x": -0.7,
+   "y": 0.85,
+   "z": 1.9,
+   "yaw_deg": 120.0,
+   "pitch_down_deg": 25.0,
+   "hfov_deg": 98.0
+  },
+  "CAM_BACK_RIGHT": {
+   "x": -0.7,
+   "y": -0.85,
+   "z": 1.9,
+   "yaw_deg": -120.0,
+   "pitch_down_deg": 25.0,
+   "hfov_deg": 98.0
+  },
+  "CAM_FRONT_NARROW": {
+   "x": 2.1,
+   "y": 0.0,
+   "z": 1.88,
+   "yaw_deg": 0.0,
+   "pitch_down_deg": 1.0,
+   "hfov_deg": 30.4
+  },
+  "CAM_BACK_NARROW": {
+   "x": -0.9,
+   "y": 0.0,
+   "z": 1.88,
+   "yaw_deg": 180.0,
+   "pitch_down_deg": 2.0,
+   "hfov_deg": 30.4
+  }
+ },
+ "cameras": {
+  "CAM_FRONT_WIDE": {
+   "intrinsics": [
+    [
+     333.8061,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     290.4113,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     0.0,
+     -0.034899497,
+     0.999390827,
+     2.1
+    ],
+    [
+     -1.0,
+     -0.0,
+     0.0,
+     0.0
+    ],
+    [
+     0.0,
+     -0.999390827,
+     -0.034899497,
+     1.9
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_FRONT_LEFT": {
+   "intrinsics": [
+    [
+     333.8061,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     290.4113,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     0.866025404,
+     -0.211309131,
+     0.453153894,
+     1.9
+    ],
+    [
+     -0.5,
+     -0.365998151,
+     0.784885567,
+     0.85
+    ],
+    [
+     0.0,
+     -0.906307787,
+     -0.422618262,
+     1.9
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_FRONT_RIGHT": {
+   "intrinsics": [
+    [
+     333.8061,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     290.4113,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     -0.866025404,
+     -0.211309131,
+     0.453153894,
+     1.9
+    ],
+    [
+     -0.5,
+     0.365998151,
+     -0.784885567,
+     -0.85
+    ],
+    [
+     0.0,
+     -0.906307787,
+     -0.422618262,
+     1.9
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_BACK_WIDE": {
+   "intrinsics": [
+    [
+     333.8061,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     290.4113,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     0.0,
+     0.034899497,
+     -0.999390827,
+     -0.9
+    ],
+    [
+     1.0,
+     -0.0,
+     0.0,
+     0.0
+    ],
+    [
+     0.0,
+     -0.999390827,
+     -0.034899497,
+     1.9
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_BACK_LEFT": {
+   "intrinsics": [
+    [
+     333.8061,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     290.4113,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     0.866025404,
+     0.211309131,
+     -0.453153894,
+     -0.7
+    ],
+    [
+     0.5,
+     -0.365998151,
+     0.784885567,
+     0.85
+    ],
+    [
+     0.0,
+     -0.906307787,
+     -0.422618262,
+     1.9
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_BACK_RIGHT": {
+   "intrinsics": [
+    [
+     333.8061,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     290.4113,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     -0.866025404,
+     0.211309131,
+     -0.453153894,
+     -0.7
+    ],
+    [
+     0.5,
+     0.365998151,
+     -0.784885567,
+     -0.85
+    ],
+    [
+     0.0,
+     -0.906307787,
+     -0.422618262,
+     1.9
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_FRONT_NARROW": {
+   "intrinsics": [
+    [
+     1413.3548,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     1229.6187,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     0.0,
+     -0.017452406,
+     0.999847695,
+     2.1
+    ],
+    [
+     -1.0,
+     -0.0,
+     0.0,
+     0.0
+    ],
+    [
+     0.0,
+     -0.999847695,
+     -0.017452406,
+     1.88
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  },
+  "CAM_BACK_NARROW": {
+   "intrinsics": [
+    [
+     1413.3548,
+     0.0,
+     384.0
+    ],
+    [
+     0.0,
+     1229.6187,
+     216.0
+    ],
+    [
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "T_ref_from_camera": [
+    [
+     0.0,
+     0.034899497,
+     -0.999390827,
+     -0.9
+    ],
+    [
+     1.0,
+     -0.0,
+     0.0,
+     0.0
+    ],
+    [
+     0.0,
+     -0.999390827,
+     -0.034899497,
+     1.88
+    ],
+    [
+     0.0,
+     0.0,
+     0.0,
+     1.0
+    ]
+   ],
+   "image_size": [
+    768,
+    432
+   ]
+  }
+ }
+}

code/tt_meteor/device.py

@@ -0,0 +1,65 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Opening the Blackhole p150 the way every published number of meteor-p150 was measured.
+
+The implementation is the vendored ``ttaw.device`` (C01): dispatch on the idle ETH cores
+(``ttnn.DispatchCoreConfig(ttnn.DispatchCoreType.ETH)``), which gives a 12x10 = 120-core compute grid on a p150
+(11x10 with WORKER dispatch, the A/B switch). ETH dispatch needs ``patches/tt-metal-eth-dispatch.patch`` on tt-metal
+44d6650 (the container image is built from a patched tree); if the ETH open fails, a ``RuntimeWarning`` is issued and
+WORKER dispatch is used, unless ``allow_fallback=False``. Never hard-code the grid: use :func:`compute_grid` or
+``device.compute_with_storage_grid_size()``.
+
+This module binds it to the validated open parameters of this port, :data:`DEVICE_DEFAULTS` (the model class uses
+the same dict), which the ``METEOR_DISPATCH``, ``_NUM_CQS``, ``_L1_SMALL``, ``_TRACE_REGION`` and
+``_WORKER_L1_SIZE`` variables and ``TT_DEVICE_ID`` override per process (SERVING.md section 3.4). Importing it has
+no side effects (``ttnn`` is imported when a device is opened).
+"""
+from __future__ import annotations
+
+import contextlib
+import dataclasses
+from typing import Any, Dict, Iterator, Optional
+
+from .ttaw.device import DeviceConfig, close_device, compute_grid, core_grid, describe_device, full_core_range_set
+
+__all__ = ["ENV_PREFIX", "DEVICE_DEFAULTS", "DeviceConfig", "device_config", "open_device", "device_session",
+           "close_device", "describe_device", "compute_grid", "core_grid", "full_core_range_set"]
+
+ENV_PREFIX = "METEOR"
+
+# Validated device-open parameters of this port (fill per model; the card's numbers are measured with them).
+DEVICE_DEFAULTS: Dict[str, Any] = {
+    # ETH dispatch (12x10) needs BOTH local tt-metal patches (patches/): the ETH-dispatch patch, and the reshape patch
+    # (Blackhole SYS-1419: the dual-kernel ROW_MAJOR reshape into small DRAM pages hangs this model under ETH dispatch
+    # without it; PORT_LOG.md section 12). ttaw 0.23.1's Resize2d TILE tail protects METEOR on its own as well.
+    # METEOR_DISPATCH=worker (11x10) is the A/B opt-in.
+    "num_command_queues": 1,  # 1, or 2 when the input upload (CQ1) overlaps the trace (CQ0)
+    "l1_small_size": 32768,  # L1_SMALL bytes per core (conv / pool config tensors)
+    # DRAM bytes for the traces of all captured variants: the whole graph is one trace of ~2,000+ programs (many
+    # DRAM-sliced 800x500 convs); the device tests also capture four debug variants. Measured in PORT_LOG.md.
+    "trace_region_size": 256 << 20,
+}
+
+
+def device_config(**overrides: Any) -> DeviceConfig:
+    """:data:`DEVICE_DEFAULTS` < the ``METEOR_*`` / ``TT_DEVICE_ID`` environment < explicit non-None ``overrides``
+    (``device_id``, ``dispatch``, ``num_command_queues``, ``l1_small_size``, ``trace_region_size``,
+    ``worker_l1_size``, ``allow_fallback``): the same resolution as ``METEOR.from_pretrained``."""
+    config = DeviceConfig.from_env(ENV_PREFIX, **DEVICE_DEFAULTS)
+    return dataclasses.replace(config, **{k: v for k, v in overrides.items() if v is not None})
+
+
+def open_device(device_id: Optional[int] = None, *, dispatch: Optional[str] = None, **overrides: Any):
+    """Open one chip like the published numbers: ETH dispatch (``dispatch="worker"`` is the A/B opt-in) and this
+    port's sizes. Close it with :func:`close_device`, or use :func:`device_session`."""
+    return device_config(device_id=device_id, dispatch=dispatch, **overrides).open()
+
+
+@contextlib.contextmanager
+def device_session(device_id: Optional[int] = None, **overrides: Any) -> Iterator[Any]:
+    """``with device_session() as dev:`` opens with :func:`open_device` and always closes, also when the body
+    raises."""
+    device = open_device(device_id, **overrides)
+    try:
+        yield device
+    finally:
+        close_device(device)

code/tt_meteor/host/__init__.py

@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host pre- and post-processing of METEOR's own runtimes (no Autoware package exists; tier4/METEOR @ dc193a81e995
+``hf/onnx_smoke_test.py``, ``deploy/cpp/realtime_main.cpp``, ``decode.cpp``, ``render.cpp``; research/meteor/SPEC.md
+sections 2, 3 and 5), shared by the Python API, the HTTP server and the CPU reference. numpy (+ the vendored ``ttaw``;
+OpenCV only for the rotated-rectangle intersection and the seg-fusion warp, as the C++).
+
+- ``preprocess.py``  frames -> the graph feed (INTER_AREA via ``ttaw.image_area``, K per-axis scaling, T inversion,
+                     absent-camera donors); METEOR's demo-scene reader.
+- ``inputs.py``      API inputs (cameras + calibration / presets + ego speed + stream) -> ``MeteorFrame``.
+- ``calib.py``       the lift geometry of a calibration (grid, validity, depth bins, gather tables): RT-dev tables.
+- ``postprocess.py`` 3D boxes + rotated NMS, stationary flags, agent futures, 2D boxes, unknown obstacles, plan,
+                     road-edge thinning (stateless, ``PostConfig`` = the RT-host knobs).
+- ``temporal.py``    per-stream state: BEV seg fusion, yaw smoothing, mode hysteresis.
+- ``outputs.py``     ``MeteorOutput`` (the ``/predict`` JSON).
+- ``result.py``      outputs of the network + frame (+ stream state) -> ``MeteorOutput``.
+"""

code/tt_meteor/host/calib.py

@@ -0,0 +1,199 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Lift geometry of the deployed 0.4 m lift from the calibration (host side; RT-dev tables of the TT port).
+
+K and T_cam_ego are consumed by the graph ONLY through the lift (one Reshape each, ONNX nodes 265 / 280; SPEC
+section 9), so everything the lift needs besides the image features is a function of the calibration and can be
+computed once per rig and uploaded as persistent device tensors (PLAN.md 2.13 "at calibration: lift geometry";
+RT-dev). This module computes it exactly as the graph does (float32, the graph's operation order; ONNX nodes 265-390;
+SPEC section 4.3), for the 8 cameras x 100,000 ground points of the 400 x 250 grid:
+
+- ``grid`` [8, 100000, 2]: the normalised sampling position (u / 767 * 2 - 1, v / 431 * 2 - 1, clipped to +-2) on the
+  108 x 192 feature maps (align_corners=False, zero padding);
+- ``valid`` [8, 100000]: z > 0.5, 0 <= u < 768, 0 <= v < 432, range < 90 m;
+- ``b0`` / ``fr`` [8, 100000]: the LINEAR depth bin of the range, b = clip((d - 1) / 1.25, 0, 62.9999), b0 = floor(b),
+  fr = b - b0 (the lift interpolates prob[b0] and prob[b0 + 1]);
+- the pair statistics (pairs per camera, cameras per cell, k_max) that size the gather-form tables of a later lift
+  kernel: :func:`ell_tables` packs the valid pairs per cell into ``K_SLOTS`` slots (k_max is 3 on every rig seen so far;
+  SPEC section 9 recommends 4 with a host assert).
+
+The projection ``T @ [x, y, 0, 1]`` is evaluated as a float32 fused-multiply-add chain (``x T0 -> fma(y, T1) ->
+fma(0, T2) -> fma(1, T3)``), which reproduces ONNX Runtime's MatMul (and torch's) bit for bit on this host; the rest is
+elementwise float32 in graph order. ``tests/test_host_calib.py`` checks grid / valid against the ORT taps exactly.
+"""
+from __future__ import annotations
+
+import hashlib
+import threading
+from collections import OrderedDict
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+import numpy as np
+
+from ..reference import config as C
+
+__all__ = ["LiftGeometry", "lift_geometry", "lift_ground_points", "ell_tables", "EllTables", "K_SLOTS",
+           "CalibrationCache", "calibration_key"]
+
+F32 = np.float32
+K_SLOTS = 4          # cameras per BEV lift cell a table slot set can hold (k_max = 3 measured on all rigs; SPEC 9)
+
+
+def lift_ground_points() -> np.ndarray:
+    """The [4, 100000] homogeneous ground points of the 400 x 250 lift grid: x = linspace(79.8, -79.8, 400) per row,
+    y = linspace(49.8, -49.8, 250) per column, z = 0, w = 1 (V52.__init__, model.py:5269-5282). The graph's constant
+    ``onnx::Expand_1643`` holds torch's float32 ``linspace`` (its two half-ranges are computed from either end), which
+    numpy's ``linspace`` misses by up to 7.6e-6 at 132,900 entries, so the values come from torch here (checked equal
+    to the constant in ``tests/test_host_calib.py``); the TT port passes the constant read from the ONNX instead."""
+    import torch
+
+    xs = torch.linspace(79.8, -79.8, C.LIFT_H, dtype=torch.float32).numpy()
+    ys = torch.linspace(49.8, -49.8, C.LIFT_W, dtype=torch.float32).numpy()
+    gx, gy = np.meshgrid(xs, ys, indexing="ij")
+    n = C.LIFT_H * C.LIFT_W
+    return np.stack([gx.ravel(), gy.ravel(), np.zeros(n, F32), np.ones(n, F32)], 0)
+
+
+def _fma32(a: np.ndarray, b: np.ndarray, c: np.ndarray) -> np.ndarray:
+    """float32 fused multiply-add (the float32 product is exact in float64; one rounding of the sum)."""
+    return (a.astype(np.float64) * b.astype(np.float64) + c.astype(np.float64)).astype(F32)
+
+
+@dataclass(frozen=True)
+class LiftGeometry:
+    """Per (camera, ground point) lift geometry of one calibration (see the module docstring)."""
+
+    grid: np.ndarray          # [8, N, 2] float32
+    valid: np.ndarray         # [8, N] bool
+    b0: np.ndarray            # [8, N] int64 in [0, 62]
+    fr: np.ndarray            # [8, N] float32 in [0, 1)
+    dist: np.ndarray          # [8, N] float32 range from the camera (metres)
+    key: str = ""             # calibration hash
+
+    @property
+    def pairs_per_camera(self) -> np.ndarray:
+        return self.valid.sum(axis=1)
+
+    @property
+    def cameras_per_cell(self) -> np.ndarray:
+        return self.valid.sum(axis=0)
+
+    def stats(self) -> Dict[str, Any]:
+        k = self.cameras_per_cell
+        return {"pairs_total": int(self.valid.sum()),
+                "pairs_per_camera": dict(zip(C.CAMERAS, self.pairs_per_camera.tolist())),
+                "cells_per_k": np.bincount(k, minlength=C.N_CAMS + 1).tolist(), "k_max": int(k.max()),
+                "cells_covered": int((k > 0).sum()),
+                "b0_range": ([int(self.b0[self.valid].min()), int(self.b0[self.valid].max())]
+                             if self.valid.any() else [])}
+
+    def bin_weights(self) -> np.ndarray:
+        """The dense [8, N, 64] float32 bin-interpolation table of the functional lift (PLAN 2.13 "a bin-interpolation
+        weight table for the fallback"): (1 - fr) at b0, fr at b0 + 1, zero elsewhere and for invalid pairs, so that
+        w = valid * (0.05 + sum_b prob_s[b] * table[b]). (b0 + 1 <= 63 always: the bin index is clipped to 62.9999.)"""
+        n, p = self.b0.shape
+        t = np.zeros((n, p, C.DEPTH_BINS), F32)
+        ii, jj = np.nonzero(self.valid)
+        t[ii, jj, self.b0[ii, jj]] = F32(1.0) - self.fr[ii, jj]
+        t[ii, jj, self.b0[ii, jj] + 1] = self.fr[ii, jj]
+        return t
+
+
+def calibration_key(K: Any, T_cam_ego: Any) -> str:
+    h = hashlib.sha256()
+    h.update(np.ascontiguousarray(np.asarray(K, F32).reshape(C.N_CAMS, 3, 3)).tobytes())
+    h.update(np.ascontiguousarray(np.asarray(T_cam_ego, F32).reshape(C.N_CAMS, 4, 4)).tobytes())
+    return h.hexdigest()[:16]
+
+
+def lift_geometry(K: Any, T_cam_ego: Any, points: Optional[np.ndarray] = None) -> LiftGeometry:
+    """The graph's lift geometry for ``K`` [(1,) 8, 3, 3] (at 768x432) and ``T_cam_ego`` [(1,) 8, 4, 4] (ego ->
+    camera); ``points`` overrides the ground points (the TT port passes the ONNX constant)."""
+    Km = np.asarray(K, F32).reshape(C.N_CAMS, 3, 3)
+    T = np.asarray(T_cam_ego, F32).reshape(C.N_CAMS, 4, 4)
+    P = lift_ground_points() if points is None else np.asarray(points, F32).reshape(4, -1)
+    pc = []
+    for r in range(3):                                    # rows x, y, z of T @ [px, py, 0, 1] (the 4th is unused)
+        acc = T[:, r, 0, None] * P[0][None]
+        for c in range(1, 4):
+            acc = _fma32(T[:, r, c, None], P[c][None], acc)
+        pc.append(acc)
+    x, y, z = pc
+    zc = np.maximum(z, F32(C.LIFT_MIN_Z))
+    u = (Km[:, 0, 0, None] * x) / zc + Km[:, 0, 2, None]
+    v = (Km[:, 1, 1, None] * y) / zc + Km[:, 1, 2, None]
+    dist = np.sqrt((x * x + y * y) + z * z)
+    valid = ((z > F32(C.LIFT_MIN_Z)) & (u >= F32(0)) & (u < F32(C.IMG_W)) & (v >= F32(0)) & (v < F32(C.IMG_H))
+             & (dist < F32(C.LIFT_MAX_DIST)))
+    gu = np.clip((u / F32(C.IMG_W - 1)) * F32(2) - F32(1), F32(-2), F32(2))
+    gv = np.clip((v / F32(C.IMG_H - 1)) * F32(2) - F32(1), F32(-2), F32(2))
+    b = np.clip((dist - F32(C.DEPTH_MIN)) / F32(C.DEPTH_STEP), F32(0), F32(C.LIFT_BIN_CLIP))
+    b0 = np.floor(b).astype(np.int64)
+    fr = (b - b0.astype(F32)).astype(F32)
+    return LiftGeometry(np.stack([gu, gv], axis=-1), valid, b0, fr, dist.astype(F32), calibration_key(Km, T))
+
+
+@dataclass(frozen=True)
+class EllTables:
+    """Gather-form lift tables, ``K_SLOTS`` slots per lift cell (unused slots have ``valid`` 0): camera, normalised
+    grid position, depth bin and fraction of every valid (camera, cell) pair, in camera order."""
+
+    cam: np.ndarray           # [N, K_SLOTS] int32
+    grid: np.ndarray          # [N, K_SLOTS, 2] float32
+    b0: np.ndarray            # [N, K_SLOTS] int32
+    fr: np.ndarray            # [N, K_SLOTS] float32
+    valid: np.ndarray         # [N, K_SLOTS] bool
+    k_max: int
+
+
+def ell_tables(geom: LiftGeometry, k_slots: int = K_SLOTS) -> EllTables:
+    """Pack the valid pairs of every cell into ``k_slots`` slots; raises if a cell is seen by more cameras (the
+    static table shape would overflow: SPEC section 9 K_SLOTS)."""
+    valid = geom.valid                                    # [8, N]
+    k = valid.sum(axis=0)
+    if int(k.max()) > k_slots:
+        raise ValueError(f"{int((k > k_slots).sum())} lift cells are seen by more than K_SLOTS={k_slots} cameras "
+                         f"(k_max {int(k.max())}): this rig needs larger tables")
+    n = valid.shape[1]
+    slot = np.cumsum(valid, axis=0) - 1                   # slot of each valid pair in its cell, cameras in order
+    cam = np.zeros((n, k_slots), np.int32)
+    grid = np.zeros((n, k_slots, 2), F32)
+    b0 = np.zeros((n, k_slots), np.int32)
+    fr = np.zeros((n, k_slots), F32)
+    ok = np.zeros((n, k_slots), bool)
+    cc, pp = np.nonzero(valid)
+    ss = slot[cc, pp]
+    cam[pp, ss] = cc
+    grid[pp, ss] = geom.grid[cc, pp]
+    b0[pp, ss] = geom.b0[cc, pp]
+    fr[pp, ss] = geom.fr[cc, pp]
+    ok[pp, ss] = True
+    return EllTables(cam, grid, b0, fr, ok, int(k.max()))
+
+
+class CalibrationCache:
+    """A small thread-safe LRU of lift geometries keyed by the calibration hash (a rig keeps its calibration, so the
+    tables are built once per rig; the demo data has 2 rigs, the public samples 2 more). ``points``: the ground points
+    (the TT port passes the graph's constant, ``MeteorWeights.lift_ground_points``; default: torch's linspace)."""
+
+    def __init__(self, maxsize: int = 8, points: Optional[np.ndarray] = None):
+        self.points = None if points is None else np.asarray(points, F32).reshape(4, -1)
+        self.maxsize = int(maxsize)
+        self._items: "OrderedDict[str, LiftGeometry]" = OrderedDict()
+        self._lock = threading.Lock()
+        self.hits = self.misses = 0
+
+    def get(self, K: Any, T_cam_ego: Any) -> LiftGeometry:
+        key = calibration_key(K, T_cam_ego)
+        with self._lock:
+            if key in self._items:
+                self._items.move_to_end(key)
+                self.hits += 1
+                return self._items[key]
+        geom = lift_geometry(K, T_cam_ego, points=self.points)
+        with self._lock:
+            self.misses += 1
+            self._items[key] = geom
+            while len(self._items) > self.maxsize:
+                self._items.popitem(last=False)
+        return geom

code/tt_meteor/host/inputs.py

@@ -0,0 +1,160 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Input handling shared by the Python API, the HTTP server and the CPU reference: up to eight cameras + calibration +
+ego speed + stream -> one :class:`~.preprocess.MeteorFrame`.
+
+Accepted ``images`` (Python API): a list of ``ttaw.io.CameraImage`` (what the server decodes ``/predict`` into), a list
+of dicts ``{"camera": name, "image": path | bytes | array | PIL, "intrinsics": ..., "T_ref_from_camera": ...}``, or a
+mapping ``{camera name: image}``. Calibration not given inline comes from ``calibration["cameras"][name]``
+(``intrinsics`` of the image AS SENT, ``T_ref_from_camera`` = camera optical frame -> base_link (METEOR's ego frame:
+x forward, y left, z up, origin on the ground), any ``ttaw.io.parse_transform`` spelling) or from a preset
+(``{"preset": name}`` -> ``calib/<name>.json``). The six wide / corner cameras are required; a missing
+``CAM_FRONT_NARROW`` / ``CAM_BACK_NARROW`` is an absent camera (zero image + its donor's K and pose, as trained:
+``config.DONOR``). ``ego_speed`` (m/s) is the graph's ``v0``. ``stream`` = ``{"id", "reset", "timestamp_s",
+"T_world_from_ego"}``: the pose drives the host temporal post-processing (BEV seg fusion, yaw tracks).
+
+:func:`load_sample` reads the bundle's sample manifests (``samples/<name>.json``: image paths relative to the file, a
+calibration preset, the ego speed and an optional stream) into the keyword arguments of ``model(...)``. A preset is
+looked up in the ``calib/`` directory beside the manifest's ``samples/`` directory first (a staged sample carries its
+own preset), then in the package's ``calib/``; it is returned inline.
+"""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any, Dict, Mapping, Optional, Tuple
+
+import numpy as np
+
+from ..reference import config as C
+from ..ttaw import io as tio
+from .preprocess import MeteorFrame, assemble_frame, pose_from_transform
+
+__all__ = ["load_cameras", "prepare_request", "load_sample", "SAMPLES_DIR", "CALIB_DIR", "stream_pose"]
+
+PKG_DIR = Path(__file__).resolve().parents[1]
+SAMPLES_DIR = PKG_DIR / "samples"
+CALIB_DIR = PKG_DIR / "calib"
+REQUIRED = tuple(c for c in C.CAMERAS if c not in C.DONOR)
+
+
+def _camera_from(entry: Any, name: Optional[str], cams_cal: Mapping[str, Any]) -> "tio.CameraImage":
+    if isinstance(entry, tio.CameraImage):
+        cam = entry
+    elif isinstance(entry, Mapping):
+        if "camera" not in entry and name is None:
+            raise tio.InputError("every image needs a 'camera' name")
+        name = str(entry.get("camera", name))
+        if "data" in entry:                                           # the /predict spelling (base64)
+            return tio.decode_cameras([dict(entry, camera=name)], {"cameras": dict(cams_cal)}, order=None,
+                                      require_calibration=True)[0]
+        src = entry.get("image", entry.get("path"))
+        if src is None:
+            raise tio.InputError(f"camera {name!r}: no 'image' / 'path' / 'data'")
+        cam = tio.CameraImage(name=name, image=tio.load_image(src), intrinsics=entry.get("intrinsics"),
+                              T_ref_from_camera=entry.get("T_ref_from_camera", entry.get("extrinsics")),
+                              timestamp_s=entry.get("timestamp_s"))
+    else:
+        if name is None:
+            raise tio.InputError("an image without a camera name: pass dicts with 'camera' or a {name: image} map")
+        cam = tio.CameraImage(name=str(name), image=tio.load_image(entry))
+    cal = dict(cams_cal.get(cam.name) or {})
+    k = cam.intrinsics if cam.intrinsics is not None else cal.get("intrinsics")
+    t = cam.T_ref_from_camera if cam.T_ref_from_camera is not None else cal.get(
+        "T_ref_from_camera", cal.get("extrinsics"))
+    if k is None or t is None:
+        raise tio.InputError(f"camera {cam.name!r} needs 'intrinsics' and 'T_ref_from_camera' (inline or in "
+                             "calibration.cameras)")
+    return tio.CameraImage(name=cam.name, image=cam.image,
+                           intrinsics=tio.parse_intrinsics(k, field=f"{cam.name}.intrinsics"),
+                           T_ref_from_camera=tio.parse_transform(t, field=f"{cam.name}.T_ref_from_camera"),
+                           distortion=cam.distortion, timestamp_s=cam.timestamp_s)
+
+
+def resolve_calibration(calibration: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
+    calibration = dict(calibration or {})
+    if "preset" in calibration:
+        calibration = tio.resolve_calibration(calibration, CALIB_DIR)
+    return calibration
+
+
+def load_cameras(images: Any, calibration: Optional[Mapping[str, Any]]) -> Dict[str, "tio.CameraImage"]:
+    """``images`` (+ ``calibration``) -> {camera name: CameraImage with K and T_ref_from_camera} (the present ones)."""
+    if images is None:
+        raise tio.InputError(f"this model needs 'images': the cameras {list(C.CAMERAS)}")
+    cams_cal = dict(resolve_calibration(calibration).get("cameras") or {})
+    if isinstance(images, Mapping):
+        cams = [_camera_from(v, k, cams_cal) for k, v in images.items()]
+    else:
+        cams = [_camera_from(e, None, cams_cal) for e in images]
+    by_name: Dict[str, Any] = {}
+    for c in cams:
+        if c.name in by_name:
+            raise tio.InputError(f"camera {c.name!r} given twice")
+        by_name[c.name] = c
+    missing = [n for n in REQUIRED if n not in by_name]
+    extra = [n for n in by_name if n not in C.CAMERAS]
+    if missing or extra:
+        raise tio.InputError(f"cameras must be {list(C.CAMERAS)} (the narrow ones may be absent); missing {missing}, "
+                             f"unexpected {extra}")
+    for c in by_name.values():
+        a = np.asarray(c.image)
+        if a.ndim != 3 or a.shape[2] != 3 or a.dtype != np.uint8:
+            raise tio.InputError(f"camera {c.name!r}: expected an (H, W, 3) uint8 image, got {a.shape} {a.dtype}")
+        if a.shape[0] < C.IMG_H or a.shape[1] < C.IMG_W:
+            raise tio.InputError(f"camera {c.name!r}: {a.shape[1]}x{a.shape[0]} is smaller than 768x432")
+    return by_name
+
+
+def stream_pose(stream: Optional[Mapping[str, Any]]) -> Optional[Tuple[float, float, float]]:
+    """(x, y, yaw) from ``stream["T_world_from_ego"]`` (or ``stream["pose"]`` = [x, y, yaw]); None without one."""
+    if not stream:
+        return None
+    if stream.get("pose") is not None:
+        p = [float(v) for v in stream["pose"]]
+        if len(p) != 3:
+            raise tio.InputError("stream.pose must be [x, y, yaw]")
+        return p[0], p[1], p[2]
+    t = stream.get("T_world_from_ego")
+    if t is None:
+        return None
+    return pose_from_transform(tio.parse_transform(t, field="stream.T_world_from_ego"))
+
+
+def prepare_request(images: Any, calibration: Optional[Mapping[str, Any]], ego_speed: Any,
+                    stream: Optional[Mapping[str, Any]] = None, *, inverse: str = "float32") -> MeteorFrame:
+    """The API inputs -> the network input of one frame (``MeteorFrame``), with the ego pose of ``stream``."""
+    if ego_speed is None:
+        raise tio.InputError("this model needs 'ego_speed' (m/s, the graph's v0)")
+    try:
+        v0 = float(ego_speed)
+    except (TypeError, ValueError):
+        raise tio.InputError(f"ego_speed must be a number (m/s), got {ego_speed!r}") from None
+    if not np.isfinite(v0):
+        raise tio.InputError("ego_speed must be finite")
+    cams = load_cameras(images, calibration)
+    try:
+        frame = assemble_frame({n: c.image for n, c in cams.items()}, {n: c.intrinsics for n, c in cams.items()},
+                               {n: c.T_ref_from_camera for n, c in cams.items()}, v0, inverse=inverse,
+                               pose=stream_pose(stream))
+    except ValueError as e:
+        raise tio.InputError(str(e)) from None
+    frame.meta["cameras"] = sorted(cams)
+    return frame
+
+
+def load_sample(path: Any, *, with_stream: bool = True) -> Dict[str, Any]:
+    """A sample manifest ``{"images": {camera: relative path}, "calibration": {...} | {"preset": name},
+    "ego_speed": m/s, "stream": {...}}`` -> the keyword arguments of ``model(...)``."""
+    p = Path(path)
+    spec = json.loads(p.read_text())
+    calib = dict(spec.get("calibration") or {})
+    beside = p.parent.parent / "calib"
+    if "preset" in calib and beside.resolve() != CALIB_DIR.resolve() and (beside / f"{calib['preset']}.json").is_file():
+        calib = tio.resolve_calibration(calib, beside)
+    calib = resolve_calibration(calib)
+    images = [{"camera": name, "image": str((p.parent / rel).resolve())} for name, rel in spec["images"].items()]
+    cams = [_camera_from(e, None, dict(calib.get("cameras") or {})) for e in images]
+    out: Dict[str, Any] = {"images": cams, "calibration": calib, "ego_speed": float(spec["ego_speed"])}
+    if with_stream and spec.get("stream"):
+        out["stream"] = dict(spec["stream"])
+    return out

code/tt_meteor/host/outputs.py

@@ -0,0 +1,111 @@
+# SPDX-License-Identifier: Apache-2.0
+"""``MeteorOutput``: the multi-task result of one frame, what ``model(...)`` returns and ``/predict`` serialises.
+
+BUNDLE_CONVENTIONS.md section 7.4 leaves METEOR's payload "per head"; this port's JSON (``to_dict()``):
+
+==========================  ==================================================================================
+key                         content
+==========================  ==================================================================================
+``detections``              3D BEV boxes sorted by score: ``label`` (VEHICLE / VRU), ``label_id``, ``score``,
+                            ``center`` [x, y] and ``size`` [length, width] in metres (ego frame, x forward,
+                            y left; METEOR predicts no z, height or velocity), ``yaw`` (rad, CCW from +x),
+                            ``stationary``, ``future`` (6 x [x, y] at 0.5 .. 3 s, the best agent mode) and
+                            ``future_mode``
+``trajectory``, ``columns``  the selected ego path, 6 x [x, y] at 0.5 s steps (``ttaw.server.smoke`` compares it)
+``plan``                    ``mode``, ``mode_probs`` (softmax of the hysteresis-adjusted logits), ``mode_logits``,
+                            ``paths`` (3 x 6 x [x, y]), ``steer`` (rad), ``accel`` (m/s^2), ``brake_prob``
+``detections_2d``           per camera: ``label``, ``label_id``, ``score``, ``box_xyxy`` (pixels at 768x432)
+``unknown_obstacles``       [x, y] of 2D "obs" boxes placed on the ground plane (``render.cpp`` unk2d)
+``traffic_light``           ``state`` (none / green / yellow / red), ``probs``
+``lane``                    the BEV lane map, uint8 [800, 500] (PNG; 0.2 m cells, row 0 = +80 m, col 0 = +50 m),
+                            after the optional seg fusion and road-edge thinning; ``lane_classes`` names its values
+``heads`` (npz only)        with ``output_format="npz"``: the dense outputs ``seg2d``, ``depth`` (bins),
+                            ``depth_mean``, ``occupancy`` (class per voxel), ``risk`` (sigmoid), ``stationary``
+==========================  ==================================================================================
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, List
+
+import numpy as np
+
+from ..reference import config as C
+from ..ttaw.io import encode_array, encode_png, to_jsonable
+from .postprocess import Box2D, Box3D
+
+__all__ = ["MeteorOutput", "LABELS"]
+
+LABELS = ("VEHICLE", "VRU")
+
+
+@dataclass
+class MeteorOutput:
+    boxes3d: List[Box3D]
+    boxes2d: List[List[Box2D]]
+    unknown: List[Dict[str, Any]]
+    plan: Dict[str, Any]
+    traffic_light: Dict[str, Any]
+    lane: np.ndarray
+    stationary_healthy: bool = True
+    heads: Dict[str, np.ndarray] = field(default_factory=dict)
+    model: str = "meteor-p150"
+    frame_id: str = "base_link"
+    timing_ms: Dict[str, float] = field(default_factory=dict)
+    meta: Dict[str, Any] = field(default_factory=dict)
+
+    def __post_init__(self) -> None:
+        self.boxes3d = sorted(self.boxes3d, key=lambda b: -b.score)
+
+    # ---- views ----------------------------------------------------------------------------------------------------
+    def to_dicts(self) -> List[Dict[str, Any]]:
+        """The 3D detection list (``detections`` of :meth:`to_dict`)."""
+        out = []
+        for b in self.boxes3d:
+            d = {"label": LABELS[b.cls], "label_id": int(b.cls), "score": float(b.score),
+                 "center": [float(b.x), float(b.y)], "size": [float(b.l), float(b.w)], "yaw": float(b.yaw),
+                 "stationary": bool(b.stationary), "future_mode": int(b.future_mode)}
+            if b.future is not None:
+                d["future"] = np.asarray(b.future, np.float64).tolist()
+            out.append(d)
+        return out
+
+    def detections_2d(self) -> Dict[str, List[Dict[str, Any]]]:
+        return {cam: [{"label": b.label, "label_id": int(b.cls), "score": float(b.score), "box_xyxy": b.xyxy()}
+                      for b in boxes] for cam, boxes in zip(C.CAMERAS, self.boxes2d)}
+
+    @property
+    def path(self) -> np.ndarray:
+        return np.asarray(self.plan["path"], np.float64)
+
+    def to_dict(self, output_format: str = "json") -> Dict[str, Any]:
+        if output_format not in ("json", "npz"):
+            raise ValueError(f"output_format must be 'json' or 'npz', not {output_format!r}")
+        plan = {"mode": int(self.plan["mode"]), "mode_probs": np.asarray(self.plan["mode_probs"]).tolist(),
+                "mode_logits": np.asarray(self.plan["mode_logits"]).tolist(),
+                "paths": np.asarray(self.plan["paths"]).tolist(), "steer": float(self.plan["steer"]),
+                "accel": float(self.plan["accel"]), "brake_prob": float(self.plan["brake_prob"]),
+                "dt": float(self.plan.get("dt", C.EGO_DT))}
+        body: Dict[str, Any] = {
+            "model": self.model, "frame_id": self.frame_id, "num_detections": len(self.boxes3d),
+            "detections": self.to_dicts(),
+            "trajectory": self.path.tolist(), "columns": ["x", "y"],
+            "plan": plan,
+            "detections_2d": self.detections_2d(),
+            "unknown_obstacles": [{k: (float(v) if isinstance(v, (float, np.floating)) else v) for k, v in u.items()}
+                                  for u in self.unknown],
+            "traffic_light": {"state": self.traffic_light["state"], "state_id": int(self.traffic_light["state_id"]),
+                              "probs": np.asarray(self.traffic_light["probs"]).tolist()},
+            "lane": encode_png(np.asarray(self.lane, np.uint8), key="lane"),
+            "lane_classes": list(C.LANE_CLASSES),
+            "stationary_head_healthy": bool(self.stationary_healthy),
+            "meta": to_jsonable(self.meta),
+            "timing_ms": {k: float(v) for k, v in self.timing_ms.items()},
+        }
+        if output_format == "npz":
+            body["heads"] = {k: encode_array(np.asarray(v), fmt="npz", key=k) for k, v in self.heads.items()}
+        return body
+
+    def __repr__(self) -> str:
+        return (f"MeteorOutput(boxes3d={len(self.boxes3d)}, boxes2d={sum(len(b) for b in self.boxes2d)}, "
+                f"mode={self.plan.get('mode')}, tl={self.traffic_light.get('state')!r})")

code/tt_meteor/host/postprocess.py

@@ -0,0 +1,462 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host post-processing of METEOR's C++ runtime (tier4/METEOR @ dc193a81e995 ``deploy/cpp/decode.cpp`` + ``render.cpp``;
+research/meteor/SPEC.md section 5). numpy (OpenCV for the rotated-rectangle intersection when installed).
+
+Per frame, stateless (the temporal parts are in ``host.temporal``):
+
+- 3D boxes (``decode.cpp:27-57`` via ``render.cpp:383-395``): sigmoid(hm) > 0.15 at 3x3 peaks (no strictly greater
+  neighbour), best 64 by score; x = 80 - (r + reg0) 0.4, y = 50 - (c + reg1) 0.4, l = e^reg2, w = e^reg3,
+  yaw = atan2(reg4, reg5) (float32 as the C++); vehicle > 0.35, VRU > 0.15, x in [-(80 - 2), 80 - 2]; greedy
+  same-class BEV NMS, suppressed if rotated IoU > 0.3 or intersection / smaller area > 0.6 (``render.cpp:283-319``).
+- Stationary flag (``render.cpp:265-276, 590-610``): stationary logit at the box cell > 0.0 when the head is healthy
+  (std >= 0.05 and range >= 0.2 over the map), else the best traj mode's last waypoint |wp6| < 0.5 m.
+- Agent futures (``render.cpp:631-655``): the traj mode argmax at the box cell, waypoints = centre + (dx, dy)_t at
+  t = 0.5 .. 3 s (the renderer draws vehicles only; every box carries its future here, ``kind`` tells them apart).
+- 2D boxes per camera and stride (``decode.cpp:59-108``): logits clipped to +-50, sigmoid > 0.30 at 3x3 peaks (on the
+  clipped plane), best 48 per camera and scale; cx = (c + reg1) s, cy = (r + reg0) s, w = e^reg2 s, h = e^reg3 s;
+  class 7 (road paint) hidden (``METEOR_2D_HIDE``).
+- Unknown obstacles (``render.cpp:657-729``): 2D class 0 boxes >= TH_UNK placed on the ground plane through the
+  bottom-centre ray (range 1.5-60 m), else back-projected with the median ``depth_mean`` of a 5x5 window (1.5-50 m);
+  |x| <= 60, |y| <= 25; greedy de-duplication within 1.5 m by score.
+- Plan (``render.cpp:432-452``): mode logits + 0.35 on the previous mode, argmax, straight preference (mode 0 unless
+  the margin is >= 1.0); the selected path, softmax confidences of the (hysteresis-adjusted) logits, controls.
+- BEV seg road-edge thinning (``render.cpp:252-262``), traffic light argmax, risk = sigmoid, occupancy argmax, metric
+  depth of a bin index (linear 1 + 1.25 b as trained, SPEC section 10 risk 2).
+
+Ties: ``std::sort`` of the C++ is not stable; this module sorts candidates stably in (class, row, col) order, so equal
+scores (saturated sigmoids) keep the scan order (documented policy, a set-based gate is insensitive to it).
+"""
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field, replace
+from typing import Any, Dict, List, Optional, Sequence, Tuple
+
+import numpy as np
+
+from ..reference import config as C
+
+__all__ = ["PostConfig", "Box3D", "Box2D", "sigmoid32", "peak_mask", "decode_boxes_3d", "filter_boxes_3d",
+           "rotated_intersection", "bev_nms", "stationary_healthy", "annotate_boxes", "decode_boxes_2d",
+           "ground_point", "unknown_obstacles", "select_mode", "plan_from_ego", "thin_road_edge", "decode_frame",
+           "metric_depth", "tl_state", "occupancy_classes"]
+
+F32 = np.float32
+
+
+@dataclass(frozen=True)
+class PostConfig:
+    """Every host knob of METEOR's runtime with its default (RT-host; ``render.cpp:345-376``, ``param.yaml:36-40``)."""
+
+    det3d_threshold: float = 0.15            # decode pre-threshold (render.cpp:388)
+    det3d_topk: int = 64
+    vehicle_threshold: float = 0.35          # render.cpp:391
+    vru_threshold: float = 0.15
+    range_margin: float = 2.0                # x in [-(XR - 2), XF - 2]
+    bev_nms_iou: float = 0.3                 # METEOR_BEV_NMS_IOU
+    bev_nms_containment: float = 0.6         # render.cpp:316
+    stationary_logit_threshold: float = 0.0  # METEOR_STAT_LOGIT_THRESH
+    stationary_health_std: float = 0.05
+    stationary_health_range: float = 0.2
+    stationary_fallback_m: float = 0.5
+    det2d_threshold: float = 0.30            # METEOR_TH2D (C++ and param.yaml; Python orin_render 0.50)
+    det2d_topk: int = 48
+    det2d_clip: float = 50.0
+    det2d_hide: Tuple[int, ...] = (7,)       # METEOR_2D_HIDE (road_paint)
+    unk2d: bool = True                       # METEOR_UNK2D
+    unk2d_threshold: Optional[float] = None  # METEOR_UNK2D_TH (default: det2d_threshold)
+    ground_z: float = 0.0                    # METEOR_GROUND_Z
+    unk2d_ground_range: Tuple[float, float] = (1.5, 60.0)
+    unk2d_depth_range: Tuple[float, float] = (1.5, 50.0)
+    unk2d_max_x: float = 60.0
+    unk2d_max_y: float = 25.0
+    unk2d_dedupe_m: float = 1.5
+    mode_hysteresis: float = 0.35            # render.cpp:436
+    straight_margin: float = 1.0             # render.cpp:441
+    thin_road_edge: bool = True              # METEOR_NO_THIN = 0
+    seg_fuse: bool = True                    # METEOR_SEG_FUSE (host.temporal)
+    yaw_smoothing: bool = True               # render.cpp:396-421 (host.temporal)
+    risk_gain: float = 1.0                   # METEOR_RISK_GAIN (display only)
+
+    def with_params(self, **params: Any) -> "PostConfig":
+        return replace(self, **{k: v for k, v in params.items() if v is not None})
+
+    @property
+    def th_unk(self) -> float:
+        return self.det2d_threshold if self.unk2d_threshold is None else self.unk2d_threshold
+
+
+@dataclass
+class Box3D:
+    """One BEV box (ego frame, metres, radians CCW from +x); ``cell`` is the 400x250 detection-grid cell."""
+
+    cls: int
+    score: float
+    x: float
+    y: float
+    l: float
+    w: float
+    yaw: float
+    cell: Tuple[int, int]
+    stationary: bool = False
+    future_mode: int = -1
+    future: Optional[np.ndarray] = None   # [6, 2] absolute waypoints (ego frame)
+
+    @property
+    def label(self) -> str:
+        return C.DET3D_CLASSES[self.cls]
+
+    def to_dict(self) -> Dict[str, Any]:
+        d = {"cls": self.cls, "label": self.label, "score": float(self.score), "x": float(self.x), "y": float(self.y),
+             "l": float(self.l), "w": float(self.w), "yaw": float(self.yaw), "cell": [int(v) for v in self.cell],
+             "stationary": bool(self.stationary), "future_mode": int(self.future_mode)}
+        if self.future is not None:
+            d["future"] = np.asarray(self.future, np.float64).round(4).tolist()
+        return d
+
+
+@dataclass
+class Box2D:
+    cls: int
+    score: float
+    cx: float
+    cy: float
+    w: float
+    h: float
+    stride: int
+
+    @property
+    def label(self) -> str:
+        return C.DET2D_CLASSES[self.cls]
+
+    def xyxy(self) -> List[float]:
+        return [self.cx - self.w / 2, self.cy - self.h / 2, self.cx + self.w / 2, self.cy + self.h / 2]
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {"cls": self.cls, "label": self.label, "score": float(self.score), "cx": float(self.cx),
+                "cy": float(self.cy), "w": float(self.w), "h": float(self.h), "stride": int(self.stride)}
+
+
+def sigmoid32(x: Any) -> np.ndarray:
+    """``1.0f / (1.0f + std::exp(-x))`` in float32 (decode.cpp ``sigmoidf``)."""
+    x = np.asarray(x, F32)
+    with np.errstate(over="ignore"):
+        return (F32(1.0) / (F32(1.0) + np.exp(-x))).astype(F32)
+
+
+def peak_mask(plane: np.ndarray) -> np.ndarray:
+    """True where no in-bounds 3x3 neighbour is strictly greater (``decode.cpp`` isPeak)."""
+    p = np.pad(np.asarray(plane, F32), 1, constant_values=-np.inf)
+    h, w = plane.shape
+    m = np.full(plane.shape, -np.inf, F32)
+    for dr in range(3):
+        for dc in range(3):
+            m = np.maximum(m, p[dr:dr + h, dc:dc + w])
+    return plane >= m
+
+
+def _candidates(planes: np.ndarray, thresh: float, topk: int, clip: Optional[float] = None):
+    """(score, cls, row, col) of the thresholded peaks of [C, H, W] logits, best ``topk`` (stable by scan order)."""
+    out = []
+    for c in range(planes.shape[0]):
+        v = np.asarray(planes[c], F32)
+        if clip is not None:
+            v = np.clip(v, F32(-clip), F32(clip))
+        p = sigmoid32(v)
+        rr, cc = np.nonzero((p > F32(thresh)) & peak_mask(v))
+        for r, q in zip(rr.tolist(), cc.tolist()):
+            out.append((float(p[r, q]), c, r, q))
+    out.sort(key=lambda z: -z[0])
+    return out[:topk]
+
+
+def decode_boxes_3d(hm: np.ndarray, reg: np.ndarray, *, thresh: float = 0.15, topk: int = 64) -> List[Box3D]:
+    """``decode_boxes`` (decode.cpp:27-57) on hm [2, 400, 250] logits and reg [6, 400, 250]; float32 box math."""
+    hm = np.asarray(hm, F32).reshape(-1, C.DET_H, C.DET_W)
+    reg = np.asarray(reg, F32).reshape(6, C.DET_H, C.DET_W)
+    res = F32(C.DET_RES)
+    boxes = []
+    for sc, c, r, q in _candidates(hm, thresh, topk):
+        R = reg[:, r, q]
+        x = F32(80.0) - (F32(r) + R[0]) * res
+        y = F32(50.0) - (F32(q) + R[1]) * res
+        boxes.append(Box3D(c, sc, float(x), float(y), float(np.exp(R[2])), float(np.exp(R[3])),
+                           float(np.arctan2(R[4], R[5])), (r, q)))
+    return boxes
+
+
+def filter_boxes_3d(boxes: Sequence[Box3D], cfg: PostConfig = PostConfig()) -> List[Box3D]:
+    """Class thresholds (vehicle > 0.35, VRU > 0.15) and the grid-range clip (``render.cpp:388-394``)."""
+    xr = C.BEV_H * C.BEV_RES - C.BEV_XF
+    out = []
+    for b in boxes:
+        th = cfg.vehicle_threshold if b.cls == 0 else cfg.vru_threshold
+        if F32(b.score) > F32(th) and -(xr - cfg.range_margin) <= b.x <= C.BEV_XF - cfg.range_margin:
+            out.append(b)
+    return out
+
+
+def _rect(b: Box3D):
+    return ((float(b.x), float(b.y)), (float(b.l), float(b.w)), float(F32(b.yaw * 180.0 / math.pi)))
+
+
+def rotated_intersection(a: Box3D, b: Box3D) -> float:
+    """Area of the intersection of two rotated BEV rectangles: ``cv::rotatedRectangleIntersection`` + ``convexHull`` +
+    ``contourArea`` as ``render.cpp:283-307`` (OpenCV when installed, else exact convex clipping)."""
+    try:
+        import cv2
+    except ImportError:  # pragma: no cover - the image and both dev envs have OpenCV
+        cv2 = None
+    if cv2 is not None:
+        r, pts = cv2.rotatedRectangleIntersection(_rect(a), _rect(b))
+        if r == cv2.INTERSECT_NONE or pts is None or len(pts) < 3:
+            return 0.0
+        return float(cv2.contourArea(cv2.convexHull(pts)))
+    from ..ttaw.nms import bev_box_corners, clip_convex, polygon_area
+    pa = bev_box_corners(np.array([a.x]), np.array([a.y]), np.array([a.l]), np.array([a.w]), np.array([a.yaw]))[0]
+    pb = bev_box_corners(np.array([b.x]), np.array([b.y]), np.array([b.l]), np.array([b.w]), np.array([b.yaw]))[0]
+    poly = clip_convex(pa, pb)
+    return float(polygon_area(poly)) if len(poly) >= 3 else 0.0
+
+
+def bev_nms(boxes: Sequence[Box3D], iou_threshold: float = 0.3, containment: float = 0.6) -> List[Box3D]:
+    """``bevBoxNms`` (render.cpp:309-319): score order, greedy, same class only; a box is dropped if its rotated IoU
+    with a kept box exceeds ``iou_threshold`` or intersection / smaller area exceeds ``containment``."""
+    order = sorted(boxes, key=lambda b: -b.score)
+    kept: List[Box3D] = []
+    for d in order:
+        ok = True
+        for k in kept:
+            if k.cls != d.cls:
+                continue
+            inter = rotated_intersection(d, k)
+            if inter <= 0.0:
+                continue
+            ad, ak = float(d.l) * float(d.w), float(k.l) * float(k.w)
+            if inter / max(ad + ak - inter, 1e-6) > iou_threshold or inter / max(min(ad, ak), 1e-6) > containment:
+                ok = False
+                break
+        if ok:
+            kept.append(d)
+    return kept
+
+
+def stationary_healthy(stat: np.ndarray, cfg: PostConfig = PostConfig()) -> bool:
+    """``stationaryHealthy`` (render.cpp:265-276): std >= 0.05 and max - min >= 0.2 over the finite values (double)."""
+    s = np.asarray(stat, np.float64).ravel()
+    s = s[np.isfinite(s)]
+    if s.size == 0:
+        return False
+    mean = s.sum() / s.size
+    var = (s * s).sum() / s.size - mean * mean
+    return bool(math.sqrt(max(var, 0.0)) >= cfg.stationary_health_std
+                and (s.max() - s.min()) >= cfg.stationary_health_range)
+
+
+def _cell(b: Box3D) -> Tuple[int, int]:
+    return int((C.BEV_XF - b.x) / C.DET_RES), int((C.BEV_YH - b.y) / C.DET_RES)
+
+
+def annotate_boxes(boxes: Sequence[Box3D], stationary: Optional[np.ndarray], traj: Optional[np.ndarray],
+                   cfg: PostConfig = PostConfig()) -> bool:
+    """Stationary flag and agent future of every box, in place; returns whether the stationary head is healthy."""
+    stat = None if stationary is None else np.asarray(stationary, F32).reshape(C.DET_H, C.DET_W)
+    tr = None if traj is None else np.asarray(traj, F32).reshape(C.TRAJ_C, C.DET_H, C.DET_W)
+    ok = stat is not None and stationary_healthy(stat, cfg)
+    for b in boxes:
+        r0, c0 = _cell(b)
+        inside = 0 <= r0 < C.DET_H and 0 <= c0 < C.DET_W
+        fut = None
+        if tr is not None and inside:
+            v = tr[:, r0, c0]
+            kb = int(np.argmax(v[36:39]))                       # first max (strict > scan)
+            fut = v[kb * 12:kb * 12 + 12].reshape(6, 2)
+            b.future_mode = kb
+            b.future = fut.astype(np.float64) + np.array([b.x, b.y])
+        if ok and inside:
+            b.stationary = bool(stat[r0, c0] > F32(cfg.stationary_logit_threshold))
+        elif fut is not None:
+            b.stationary = bool(math.hypot(float(fut[5, 0]), float(fut[5, 1])) < cfg.stationary_fallback_m)
+        else:
+            b.stationary = False
+    return ok
+
+
+def decode_boxes_2d(outputs: Dict[str, np.ndarray], cfg: PostConfig = PostConfig(), *,
+                    keep_hidden: bool = False) -> List[List[Box2D]]:
+    """``decode_boxes2d_ms`` (decode.cpp:59-108) over the three strides -> 8 lists of boxes (pixels at 768x432);
+    hidden classes (``det2d_hide``) are removed unless ``keep_hidden``."""
+    res: List[List[Box2D]] = [[] for _ in range(C.N_CAMS)]
+    for si, s in enumerate(C.DET2D_STRIDES):
+        hm = np.asarray(outputs[f"hm2d_s{si}"], F32).reshape(C.N_CAMS, len(C.DET2D_CLASSES), -1)
+        h, w = C.FEAT_H * 4 // s, C.FEAT_W * 4 // s
+        hm = hm.reshape(C.N_CAMS, -1, h, w)
+        reg = np.asarray(outputs[f"reg2d_s{si}"], F32).reshape(C.N_CAMS, 4, h, w)
+        sf = F32(s)
+        for n in range(C.N_CAMS):
+            for sc, c, r, q in _candidates(hm[n], cfg.det2d_threshold, cfg.det2d_topk, clip=cfg.det2d_clip):
+                if c in cfg.det2d_hide and not keep_hidden:
+                    continue
+                R = reg[n, :, r, q]
+                res[n].append(Box2D(c, sc, float((F32(q) + R[1]) * sf), float((F32(r) + R[0]) * sf),
+                                    float(np.exp(R[2]) * sf), float(np.exp(R[3]) * sf), s))
+    return res
+
+
+def ground_point(u: float, v: float, Kc: np.ndarray, T: np.ndarray,
+                 ground_z: float = 0.0) -> Optional[Tuple[float, float]]:
+    """``groundPoint`` (render.cpp:321-336): where the ray through pixel (u, v) meets the plane z = ground_z (ego
+    frame); ``T`` = T_cam_ego (ego -> camera). None if the ray points up or behind."""
+    Kc = np.asarray(Kc, np.float64).reshape(3, 3)
+    T = np.asarray(T, np.float64).reshape(4, 4)
+    dc = np.array([(u - Kc[0, 2]) / Kc[0, 0], (v - Kc[1, 2]) / Kc[1, 1], 1.0])
+    R, t = T[:3, :3], T[:3, 3]
+    c = -(R.T @ t)
+    d = R.T @ dc
+    if d[2] >= -1e-6:
+        return None
+    lam = (ground_z - c[2]) / d[2]
+    if lam <= 0:
+        return None
+    return float(c[0] + lam * d[0]), float(c[1] + lam * d[1])
+
+
+def unknown_obstacles(boxes2d: Sequence[Sequence[Box2D]], depth_mean: Optional[np.ndarray], K: np.ndarray,
+                      T_cam_ego: np.ndarray, cfg: PostConfig = PostConfig()) -> List[Dict[str, float]]:
+    """2D 'obs' boxes lifted to the BEV (``render.cpp:657-729``) -> [{x, y, score, range}] after de-duplication."""
+    K = np.asarray(K, np.float64).reshape(C.N_CAMS, 3, 3)
+    T = np.asarray(T_cam_ego, np.float64).reshape(C.N_CAMS, 4, 4)
+    dm = None if depth_mean is None else np.asarray(depth_mean, np.float64).reshape(C.N_CAMS, -1, C.FEAT_W // 2)
+    cand = []
+    for i in range(min(C.N_CAMS, len(boxes2d))):
+        for b in boxes2d[i]:
+            if b.cls != 0 or b.score < cfg.th_unk:
+                continue
+            gp = ground_point(b.cx, b.cy + 0.5 * b.h, K[i], T[i], cfg.ground_z)
+            lo, hi = cfg.unk2d_ground_range
+            if gp is not None and lo < math.hypot(*gp) < hi:
+                x, y, d = gp[0], gp[1], math.hypot(*gp)
+            else:
+                if dm is None:
+                    continue
+                dmh, dmw = dm.shape[1], dm.shape[2]
+                u = min(max(int(b.cx / C.IMG_W * dmw), 0), dmw - 1)
+                v = min(max(int((b.cy + 0.35 * b.h) / C.IMG_H * dmh), 0), dmh - 1)
+                win = np.sort(dm[i, max(0, v - 2):min(dmh, v + 3), max(0, u - 2):min(dmw, u + 3)].ravel())
+                d = float(win[win.size // 2])
+                lo, hi = cfg.unk2d_depth_range
+                if not (lo < d < hi):
+                    continue
+                pc = np.array([(b.cx - K[i, 0, 2]) / K[i, 0, 0] * d, (b.cy - K[i, 1, 2]) / K[i, 1, 1] * d, d])
+                pe = T[i, :3, :3].T @ (pc - T[i, :3, 3])
+                x, y = float(pe[0]), float(pe[1])
+            if abs(x) > cfg.unk2d_max_x or abs(y) > cfg.unk2d_max_y:
+                continue
+            cand.append({"x": x, "y": y, "score": float(b.score), "range": float(d), "camera": C.CAMERAS[i]})
+    cand.sort(key=lambda p: -p["score"])
+    kept: List[Dict[str, float]] = []
+    for p in cand:
+        if all((p["x"] - k["x"]) ** 2 + (p["y"] - k["y"]) ** 2 > cfg.unk2d_dedupe_m ** 2 for k in kept):
+            kept.append(p)
+    return kept
+
+
+def select_mode(ego: np.ndarray, prev_mode: Optional[int] = None,
+                cfg: PostConfig = PostConfig()) -> Tuple[int, np.ndarray]:
+    """``render.cpp:432-445``: float32 mode logits + hysteresis on the previous mode, first argmax, straight preference.
+    Returns (mode, the adjusted logits)."""
+    lg = np.asarray(ego, F32).reshape(-1)[36:39].copy()
+    if prev_mode is not None and 0 <= prev_mode < C.EGO_MODES:
+        lg[prev_mode] = lg[prev_mode] + F32(cfg.mode_hysteresis)
+    k = 0
+    for i in range(1, C.EGO_MODES):
+        if lg[i] > lg[k]:
+            k = i
+    if k != 0 and (lg[k] - lg[0]) < F32(cfg.straight_margin):
+        k = 0
+    return k, lg
+
+
+def plan_from_ego(ego: np.ndarray, prev_mode: Optional[int] = None, cfg: PostConfig = PostConfig()) -> Dict[str, Any]:
+    """The plan of ``render.cpp:432-452``: selected mode and path, the three paths, confidences (softmax of the
+    adjusted logits, double), controls (steer rad, accel m/s^2, brake logit and probability)."""
+    e = np.asarray(ego, F32).reshape(-1)
+    k, lg = select_mode(e, prev_mode, cfg)
+    mx = float(lg.max())
+    pr = np.exp(lg.astype(np.float64) - mx)
+    pr /= pr.sum()
+    paths = e[:36].reshape(C.EGO_MODES, C.EGO_STEPS, 2)
+    brake = float(e[41])
+    return {"mode": int(k), "path": paths[k].astype(np.float64), "paths": paths.astype(np.float64),
+            "mode_logits": e[36:39].astype(np.float64), "mode_probs": pr, "steer": float(e[39]),
+            "accel": float(e[40]), "brake_logit": brake, "brake_prob": 1.0 / (1.0 + math.exp(-brake)),
+            "dt": C.EGO_DT}
+
+
+def thin_road_edge(lane: np.ndarray) -> np.ndarray:
+    """``thin_road_edge`` (render.cpp:252-262): keep only road-edge (6) pixels next to drivable (1, 3, 4, 5) pixels
+    that are not drivable themselves; every other edge pixel becomes background. Returns a new map."""
+    lane = np.asarray(lane, np.uint8)
+    edge = lane == 6
+    if not edge.any():
+        return lane.copy()
+    drv = np.isin(lane, (1, 3, 4, 5))
+    p = np.pad(drv, 1)
+    h, w = lane.shape
+    dil = np.zeros_like(drv)
+    for dr in range(3):
+        for dc in range(3):
+            dil |= p[dr:dr + h, dc:dc + w]
+    out = lane.copy()
+    out[edge] = 0
+    out[edge & dil & ~drv] = 6
+    return out
+
+
+def metric_depth(bins: np.ndarray, kind: str = "linear") -> np.ndarray:
+    """Depth in metres of a bin index map: ``linear`` 1 + 1.25 b (the trained bins, SPEC 10 risk 2) or ``log`` (the
+    exported depth_mean centres)."""
+    return C.depth_bin_centres(kind)[np.asarray(bins, np.int64)]
+
+
+def tl_state(tl: np.ndarray) -> Dict[str, Any]:
+    t = np.asarray(tl, np.float64).reshape(-1)
+    p = np.exp(t - t.max())
+    p /= p.sum()
+    k = int(np.argmax(t))
+    return {"state": C.TL_CLASSES[k], "state_id": k, "probs": p}
+
+
+def occupancy_classes(occ: np.ndarray) -> np.ndarray:
+    """[1, 10, 16, 200, 200] logits -> uint8 [16, 200, 200] class per voxel (first max)."""
+    return np.argmax(np.asarray(occ).reshape(len(C.OCC_CLASSES), C.OCC_Z_BINS, 200, 200), axis=0).astype(np.uint8)
+
+
+@dataclass
+class FrameDecode:
+    """The stateless decode of one frame's outputs."""
+
+    boxes3d: List[Box3D]
+    stationary_healthy: bool
+    boxes2d: List[List[Box2D]]
+    unknown: List[Dict[str, float]]
+    plan: Dict[str, Any]
+    traffic_light: Dict[str, Any]
+    lane: np.ndarray
+    extras: Dict[str, Any] = field(default_factory=dict)
+
+
+def decode_frame(outputs: Dict[str, np.ndarray], K: Optional[np.ndarray] = None, T_cam_ego: Optional[np.ndarray] = None,
+                 cfg: PostConfig = PostConfig(), *, prev_mode: Optional[int] = None) -> FrameDecode:
+    """Every stateless decode of ``render.cpp`` on the 19 outputs (numpy, ONNX shapes). Seg fusion, yaw smoothing and
+    the mode history are ``host.temporal``'s."""
+    boxes = decode_boxes_3d(outputs["hm"], outputs["reg"], thresh=cfg.det3d_threshold, topk=cfg.det3d_topk)
+    boxes = bev_nms(filter_boxes_3d(boxes, cfg), cfg.bev_nms_iou, cfg.bev_nms_containment)
+    healthy = annotate_boxes(boxes, outputs.get("stationary"), outputs.get("traj"), cfg)
+    b2d = decode_boxes_2d(outputs, cfg)
+    unk: List[Dict[str, float]] = []
+    if cfg.unk2d and K is not None and T_cam_ego is not None:
+        unk = unknown_obstacles(b2d, outputs.get("depth_mean"), K, T_cam_ego, cfg)
+    lane = np.asarray(outputs["lane"], np.uint8).reshape(C.BEV_H, C.BEV_W)
+    return FrameDecode(boxes, healthy, b2d, unk, plan_from_ego(outputs["ego"], prev_mode, cfg),
+                       tl_state(outputs["tl"]), lane)

code/tt_meteor/host/preprocess.py

@@ -0,0 +1,189 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host pre-processing of METEOR's runtimes: eight camera frames + calibration + ego speed -> the graph's feed.
+
+The contract (research/meteor/SPEC.md sections 2-3; ``meteor_v157.param.yaml``; tier4/METEOR ``hf/onnx_smoke_test.py:
+34-56``, the stated reference for feeding the graph, and ``deploy/cpp/realtime_main.cpp:123-140``):
+
+- ``imgs`` uint8 [1, 8, 3, 432, 768], RGB, CHW, cameras in the training order (``config.CAMERAS``). A frame of another
+  size is resized with OpenCV ``INTER_AREA`` (``ttaw.image_area.meteor_resize``, bit-exact), anisotropically if the
+  aspect differs; frames are raw and unrectified (training never undistorted).
+- ``K`` float32 [1, 8, 3, 3] at 768x432: the intrinsics of the image as captured, row 0 x 768 / w0 and row 1 x 432 / h0
+  (``extract_gt.py:128-130``); only fx, fy, cx, cy reach the graph.
+- ``T_cam_ego`` float32 [1, 8, 4, 4] = inv(T_ego_cam): ``np.linalg.inv`` on float32 like ``onnx_smoke_test.py:46`` (the
+  research goldens' feed, byte-identical; ``inverse="float64"`` gives the C++ runtime's double inverse cast to float).
+- ``v0`` float32 [1]: the ego speed in m/s.
+- An absent camera (the 7-camera rig, or a dropped slot) is an all-zero image plus the DONOR's K and pose
+  (``bevlane/dataset.py:17-21``: CAM_BACK_NARROW <- CAM_BACK_WIDE, CAM_FRONT_NARROW <- CAM_FRONT_WIDE). With the
+  released ``/255`` input a uint8 zero image is exactly the trained "absent" input; with ``input_norm="imagenet"`` the
+  camera must be zeroed after normalising (``present`` mask, :func:`normalize_images`).
+
+:func:`load_scene_frame` reads METEOR's demo-scene layout (``manifest.json``, ``img/NNNN_<CAM>.jpg``,
+``ego_motion.npz``), which the public-dataset samples reuse; it reproduces ``research/meteor/scripts/meteor_io.py``'s
+feed byte for byte (``tests/test_host_preprocess.py`` checks the sha256 of the shipped sample against its golden).
+:func:`assemble_frame` is the API path (images of any size + calibration + speed).
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import math
+import os
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, Dict, Mapping, Optional, Sequence, Tuple
+
+import numpy as np
+
+from ..reference import config as C
+from ..ttaw.image_area import meteor_resize, scale_intrinsics
+from ..ttaw.io import load_image
+
+__all__ = ["MeteorFrame", "assemble_frame", "load_scene_frame", "load_manifest", "scene_dir_of", "invert_extrinsics",
+           "prepare_camera", "normalize_images", "pose_from_transform"]
+
+F32 = np.float32
+
+
+@dataclass
+class MeteorFrame:
+    """One network input (the ONNX feed) plus what the host post-processing needs."""
+
+    imgs: np.ndarray                      # uint8 [1, 8, 3, 432, 768] RGB
+    K: np.ndarray                         # float32 [1, 8, 3, 3] at 768x432
+    T_cam_ego: np.ndarray                 # float32 [1, 8, 4, 4] ego -> camera
+    v0: np.ndarray                        # float32 [1] m/s
+    present: np.ndarray                   # bool [8]
+    pose: Optional[Tuple[float, float, float]] = None   # ego (x, y, yaw) in a world frame (seg fusion, yaw tracks)
+    meta: Dict[str, Any] = field(default_factory=dict)
+
+    def feed(self) -> Dict[str, np.ndarray]:
+        """The ONNX input dict (``imgs``, ``K``, ``T_cam_ego``, ``v0``)."""
+        return {"imgs": self.imgs, "K": self.K, "T_cam_ego": self.T_cam_ego, "v0": self.v0}
+
+    def sha256(self) -> Dict[str, str]:
+        return {k: hashlib.sha256(np.ascontiguousarray(v).tobytes()).hexdigest() for k, v in self.feed().items()}
+
+
+def invert_extrinsics(T_ego_cam: Any, inverse: str = "float32") -> np.ndarray:
+    """camera -> ego (4x4) to the graph's ego -> camera, float32. ``float32``: ``np.linalg.inv`` of the float32 matrix
+    (``onnx_smoke_test.py:46``, the research feed); ``float64``: double inverse cast to float (``realtime_main.cpp:
+    123-128``)."""
+    if inverse == "float32":
+        return np.linalg.inv(np.asarray(T_ego_cam, F32).reshape(4, 4)).astype(F32)
+    if inverse == "float64":
+        return np.linalg.inv(np.asarray(T_ego_cam, np.float64).reshape(4, 4)).astype(F32)
+    raise ValueError(f"inverse must be 'float32' or 'float64', not {inverse!r}")
+
+
+def prepare_camera(image: Any, K: Optional[Any] = None) -> Tuple[np.ndarray, Optional[np.ndarray], Tuple[int, int]]:
+    """One frame (anything ``ttaw.io.load_image`` takes; RGB) -> (uint8 [3, 432, 768] RGB CHW, K at 768x432 or None,
+    the source (h, w)). INTER_AREA when the size differs (``ttaw.image_area``); K is scaled per axis."""
+    rgb = load_image(image)
+    src_hw = (int(rgb.shape[0]), int(rgb.shape[1]))
+    if src_hw != (C.IMG_H, C.IMG_W):
+        if src_hw[0] < C.IMG_H or src_hw[1] < C.IMG_W:
+            raise ValueError(f"camera frame {src_hw[1]}x{src_hw[0]} is smaller than METEOR's 768x432 input "
+                             "(INTER_AREA up-scaling is not supported)")
+        rgb = meteor_resize(rgb)
+    k = None if K is None else scale_intrinsics(K, src_hw)
+    return np.ascontiguousarray(rgb.transpose(2, 0, 1)), k, src_hw
+
+
+def pose_from_transform(T_world_from_ego: Any) -> Tuple[float, float, float]:
+    """(x, y, yaw) of a 4x4 world <- ego transform (METEOR's ``ego_motion.npz`` ``pose`` convention)."""
+    T = np.asarray(T_world_from_ego, np.float64).reshape(4, 4)
+    return float(T[0, 3]), float(T[1, 3]), float(math.atan2(T[1, 0], T[0, 0]))
+
+
+def assemble_frame(images: Mapping[str, Any], intrinsics: Mapping[str, Any], T_ego_cam: Mapping[str, Any],
+                   v0: float, *, present: Optional[Mapping[str, bool]] = None, inverse: str = "float32",
+                   pose: Optional[Tuple[float, float, float]] = None) -> MeteorFrame:
+    """The API path: per camera name an image (any size; None = absent), the intrinsics of that image and its
+    camera -> ego transform; ``v0`` in m/s. Absent cameras get a zero image and their donor's K and pose; a camera
+    without a donor (only the two narrow cameras have one) must be present. A narrow camera sent as an all-zero image
+    is absent too (METEOR's data convention, ``absent_zero.jpg``): its calibration is replaced by the donor's, as in
+    training (``bevlane/dataset.py:131-145``)."""
+    imgs = np.zeros((1, C.N_CAMS, 3, C.IMG_H, C.IMG_W), np.uint8)
+    K = np.zeros((1, C.N_CAMS, 3, 3), F32)
+    T = np.zeros((1, C.N_CAMS, 4, 4), F32)
+    flags = np.zeros(C.N_CAMS, bool)
+    src_sizes: Dict[str, Any] = {}
+    cams: Dict[str, Tuple[np.ndarray, np.ndarray]] = {}
+    for name in C.CAMERAS:
+        img = images.get(name)
+        ok = img is not None and (present is None or bool(present.get(name, True)))
+        if not ok:
+            continue
+        chw, k, src_hw = prepare_camera(img, intrinsics.get(name))
+        if name in C.DONOR and not chw.any():
+            continue                  # an all-zero narrow camera is METEOR's "absent" convention: zero + donor pose
+        if k is None or T_ego_cam.get(name) is None:
+            raise ValueError(f"{name}: intrinsics and T_ego_cam are required for a present camera")
+        i = C.CAMERAS.index(name)
+        imgs[0, i] = chw
+        flags[i] = True
+        cams[name] = (k, np.asarray(T_ego_cam[name], np.float64).reshape(4, 4))
+        src_sizes[name] = list(src_hw)
+    donors: Dict[str, str] = {}
+    for i, name in enumerate(C.CAMERAS):
+        src = name if flags[i] else C.DONOR.get(name)
+        if src is None or src not in cams:
+            raise ValueError(f"{name} is absent and has no present donor camera (donors: {C.DONOR})")
+        if src != name:
+            donors[name] = src
+        k, t = cams[src]
+        K[0, i] = np.asarray(k, F32)
+        T[0, i] = invert_extrinsics(t, inverse)
+    return MeteorFrame(imgs, K, T, np.array([v0], F32), flags, pose,
+                       {"source_hw": src_sizes, "donors": donors, "inverse": inverse})
+
+
+def scene_dir_of(root: os.PathLike) -> Path:
+    """``<root>/scenes.txt`` -> ``<root>/<first scene>`` (METEOR demo-root layout)."""
+    root = Path(root)
+    return root / (root / "scenes.txt").read_text().split()[0]
+
+
+def load_manifest(scene_dir: os.PathLike) -> Dict[str, Any]:
+    return json.loads((Path(scene_dir) / "manifest.json").read_text())
+
+
+def load_scene_frame(scene_dir: os.PathLike, frame: int, manifest: Optional[Mapping[str, Any]] = None, *,
+                     inverse: str = "float32") -> MeteorFrame:
+    """Frame ``frame`` of a METEOR demo-scene directory (``manifest.json`` cams[c].K at 768x432 and T_ego_cam,
+    frames[i].imgs[c]; ``ego_motion.npz`` v0 / pose / wp / valid), as ``meteor_io.load_frame`` feeds the graph."""
+    scene_dir = Path(scene_dir)
+    m = manifest or load_manifest(scene_dir)
+    fr = m["frames"][frame]
+    imgs = np.zeros((1, C.N_CAMS, 3, C.IMG_H, C.IMG_W), np.uint8)
+    K = np.zeros((1, C.N_CAMS, 3, 3), F32)
+    T = np.zeros((1, C.N_CAMS, 4, 4), F32)
+    for i, cam in enumerate(C.CAMERAS):
+        chw, _, _ = prepare_camera(scene_dir / fr["imgs"][cam])
+        imgs[0, i] = chw
+        K[0, i] = np.asarray(m["cams"][cam]["K"], F32)
+        T[0, i] = invert_extrinsics(m["cams"][cam]["T_ego_cam"], inverse)
+    present = np.asarray(m.get("present_mask", [1] * C.N_CAMS), bool).reshape(C.N_CAMS)
+    ego = np.load(scene_dir / "ego_motion.npz")
+    v0 = float(ego["v0"][frame])
+    pose = tuple(float(v) for v in ego["pose"][frame]) if "pose" in ego.files else None
+    meta: Dict[str, Any] = {"scene": m.get("scene", scene_dir.name), "frame": int(frame)}
+    for key in ("wp", "valid"):
+        if key in ego.files:
+            meta[key] = np.asarray(ego[key][frame]).tolist()
+    return MeteorFrame(imgs, K, T, np.array([v0], F32), present, pose, meta)
+
+
+def normalize_images(imgs_u8: np.ndarray, mode: str = "onnx", present: Optional[Sequence[bool]] = None) -> np.ndarray:
+    """Host oracle of the device input normalisation: uint8 [1, 8, 3, H, W] -> float32 [8, 3, H, W]; ``onnx`` = x / 255
+    (as released), ``imagenet`` = (x / 255 - mean) / std with absent cameras zeroed afterwards (D12)."""
+    x = np.asarray(imgs_u8).astype(F32) / F32(255.0)
+    if mode == "imagenet":
+        mean = np.asarray(C.IMAGENET_MEAN, F32).reshape(1, 1, 3, 1, 1)
+        std = np.asarray(C.IMAGENET_STD, F32).reshape(1, 1, 3, 1, 1)
+        x = (x - mean) / std
+        if present is not None:
+            x = x * np.asarray(present, F32).reshape(1, -1, 1, 1, 1)
+    elif mode != "onnx":
+        raise ValueError(f"input_norm must be one of {C.INPUT_NORMS}, not {mode!r}")
+    return x.reshape(C.N_CAMS, 3, C.IMG_H, C.IMG_W)

code/tt_meteor/host/result.py

@@ -0,0 +1,51 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The 19 network outputs of one frame (+ the frame, + the stream's host state) -> :class:`~.outputs.MeteorOutput`,
+in the order of ``render.cpp``'s ``compose_frame``: 3D decode -> class filter -> BEV NMS -> yaw smoothing ->
+stationary / futures; 2D decode -> unknown obstacles; mode selection with hysteresis; lane -> seg fusion -> road-edge
+thinning. Shared by the API (``_postprocess``) and the CPU reference, so both apply exactly the same host code."""
+from __future__ import annotations
+
+from typing import Any, Dict, Mapping, Optional
+
+import numpy as np
+
+from ..reference import config as C
+from .outputs import MeteorOutput
+from .postprocess import PostConfig, decode_frame, occupancy_classes, plan_from_ego, sigmoid32, thin_road_edge
+from .preprocess import MeteorFrame
+from .temporal import StreamState
+
+__all__ = ["build_output", "dense_heads"]
+
+
+def dense_heads(outputs: Mapping[str, np.ndarray]) -> Dict[str, np.ndarray]:
+    """The dense per-head arrays of ``output_format="npz"``."""
+    return {"seg2d": np.asarray(outputs["seg2d"], np.uint8).reshape(C.N_CAMS, C.FEAT_H, C.FEAT_W),
+            "depth": np.asarray(outputs["depth"], np.uint8).reshape(C.N_CAMS, C.FEAT_H, C.FEAT_W),
+            "depth_mean": np.asarray(outputs["depth_mean"], np.float16).reshape(C.N_CAMS, C.FEAT_H // 2, C.FEAT_W // 2),
+            "occupancy": occupancy_classes(outputs["occ"]),
+            "risk": sigmoid32(np.asarray(outputs["risk"]).reshape(C.DET_H, C.DET_W)).astype(np.float16),
+            "stationary": np.asarray(outputs["stationary"], np.float32).reshape(C.DET_H, C.DET_W)}
+
+
+def build_output(outputs: Mapping[str, np.ndarray], frame: MeteorFrame, cfg: PostConfig = PostConfig(),
+                 state: Optional[StreamState] = None, *, heads: bool = False, model: str = "meteor-p150",
+                 timing_ms: Optional[Dict[str, float]] = None, meta: Optional[Dict[str, Any]] = None) -> MeteorOutput:
+    dec = decode_frame(outputs, frame.K, frame.T_cam_ego, cfg)
+    lane = dec.lane
+    plan = dec.plan
+    if state is not None:
+        if cfg.yaw_smoothing:
+            state.yaw.apply(dec.boxes3d)
+        plan = plan_from_ego(outputs["ego"], state.mode.prev, cfg)
+        state.mode.prev = int(plan["mode"])
+        if cfg.seg_fuse:
+            lane = state.seg.apply(lane, outputs["lane_logit"], frame.pose)
+        state.frames += 1
+    if cfg.thin_road_edge:
+        lane = thin_road_edge(lane)
+    out_meta = {"present": [bool(v) for v in frame.present], "v0": float(frame.v0[0])}
+    out_meta.update(meta or {})
+    return MeteorOutput(dec.boxes3d, dec.boxes2d, dec.unknown, plan, dec.traffic_light, lane, dec.stationary_healthy,
+                        dense_heads(outputs) if heads else {}, model=model, timing_ms=dict(timing_ms or {}),
+                        meta=out_meta)

code/tt_meteor/host/temporal.py

@@ -0,0 +1,180 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host-side temporal state of METEOR's runtime (the deployed graph itself is single-frame; SPEC section 3 end).
+
+Three pieces of state live in ``render.cpp``, all host post-processing (RT-host), kept here per ``stream.id``:
+
+- **BEV seg fusion** (``seg_fuse``, ``render.cpp:131-198``): the 9-class log-softmax of ``lane_logit`` is accumulated
+  with decay 0.7, the previous accumulator warped into the current ego frame by the pose change (nearest,
+  ``cv::warpAffine`` with ``WARP_INVERSE_MAP``, 5 px/m about row 400 / col 250); the fused argmax replaces the lane
+  class on rows >= 175 (x <= 45 m) except where the current class is thin (3 crosswalk, 4 laneline, 5 stopline,
+  6 road edge). The state resets itself on a pose jump (|tx| + |ty| >= 10 m in the previous frame, or |dyaw| >= 0.5).
+- **Yaw smoothing** (``render.cpp:396-421``): every box matched to the nearest previous box within 2.5 m has its yaw
+  flipped by pi if it turned by more than pi / 2 (no 180 deg flips) and is blended 0.4 towards the new value.
+- **Mode hysteresis** (``render.cpp:432-442``): the previously selected ego mode gets +0.35 on its logit.
+
+Upstream resets only the seg accumulator (on a pose jump); the yaw tracks and the previous mode have no reset. This
+port resets all three on ``stream.reset`` / a new stream id (SPEC section 3, "our design"). Seg fusion needs the ego
+pose (``stream.T_world_from_ego`` -> (x, y, yaw)); without a pose it is skipped and its state cleared, as the C++ does.
+The warp uses OpenCV (as the C++); the log-softmax is the C++'s float32 loop order (max, sum of exp, log).
+"""
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Tuple
+
+import numpy as np
+
+from ..reference import config as C
+from .postprocess import Box3D, PostConfig, select_mode
+
+__all__ = ["SegFusion", "YawTracks", "ModeHistory", "StreamState", "warp_matrix", "log_softmax9"]
+
+F32 = np.float32
+THIN = (3, 4, 5, 6)
+SEG_DECAY = 0.7
+SEG_ROW0 = 175                 # rows >= 175 (x <= 45 m) are replaced by the fused class
+YAW_GATE2 = 6.25               # 2.5 m squared
+YAW_EMA = 0.4
+
+
+def warp_matrix(pose: Tuple[float, float, float], prev: Tuple[float, float, float]) -> Optional[np.ndarray]:
+    """``warpM`` (render.cpp:149-160): the inverse-map affine of the 800x500 grid from the previous pose to the
+    current one, float32 [2, 3]; None on a pose jump (|tx| + |ty| >= 10 m or |dyaw| >= 0.5 rad)."""
+    cp, sp = math.cos(prev[2]), math.sin(prev[2])
+    dx, dy = pose[0] - prev[0], pose[1] - prev[1]
+    tx, ty = cp * dx + sp * dy, -sp * dx + cp * dy
+    dyaw = pose[2] - prev[2]
+    if abs(tx) + abs(ty) >= 10.0 or abs(dyaw) >= 0.5:
+        return None
+    c, s2 = math.cos(dyaw), math.sin(dyaw)
+    return np.array([[c, s2, -250 * c - 400 * s2 - 5 * ty + 250],
+                     [-s2, c, 250 * s2 - 400 * c - 5 * tx + 400]], F32)
+
+
+def log_softmax9(logit: np.ndarray) -> np.ndarray:
+    """[9, H, W] logits -> [H, W, 9] float32 log-probabilities, the C++ loop: mx = max_k, se = sum_k exp(v - mx) in
+    class order (float), lse = mx + log(se), lp = v - lse."""
+    v = np.asarray(logit, F32)
+    mx = v.max(axis=0)
+    se = np.zeros(mx.shape, F32)
+    for k in range(v.shape[0]):
+        se = se + np.exp(v[k] - mx)
+    lse = mx + np.log(se)
+    return np.ascontiguousarray((v - lse[None]).transpose(1, 2, 0))
+
+
+class SegFusion:
+    """The BEV seg temporal accumulator of one stream (``seg_fuse`` with logits)."""
+
+    def __init__(self) -> None:
+        self.acc: Optional[np.ndarray] = None            # [800, 500, 9] float32 log-probabilities
+        self.pose: Optional[Tuple[float, float, float]] = None
+
+    def reset(self) -> None:
+        self.acc, self.pose = None, None
+
+    def apply(self, lane: np.ndarray, lane_logit: np.ndarray, pose: Optional[Tuple[float, float, float]]) -> np.ndarray:
+        """Fused lane map (uint8 [800, 500]; a new array) from this frame's argmax and logits [9, 800, 500]."""
+        lane = np.array(np.asarray(lane, np.uint8).reshape(C.BEV_H, C.BEV_W))
+        if pose is None:
+            self.reset()
+            return lane
+        import cv2
+
+        lp = log_softmax9(np.asarray(lane_logit).reshape(len(C.LANE_CLASSES), C.BEV_H, C.BEV_W))
+        if self.acc is not None:
+            M = warp_matrix(pose, self.pose)
+            if M is not None:
+                warped = np.concatenate(
+                    [cv2.warpAffine(np.ascontiguousarray(self.acc[:, :, k:k + 3]), M, (C.BEV_W, C.BEV_H),
+                                    flags=cv2.INTER_NEAREST | cv2.WARP_INVERSE_MAP, borderMode=cv2.BORDER_CONSTANT,
+                                    borderValue=(0, 0, 0)) for k in range(0, 9, 3)], axis=2)
+                lp = lp + F32(SEG_DECAY) * warped
+        self.acc = lp
+        self.pose = (float(pose[0]), float(pose[1]), float(pose[2]))
+        sub = lp[SEG_ROW0:]
+        best = np.zeros(sub.shape[:2], np.int64)          # first max with a strict > scan (class order)
+        bv = sub[:, :, 0].copy()
+        for k in range(1, sub.shape[2]):
+            better = sub[:, :, k] > bv
+            best[better] = k
+            bv[better] = sub[:, :, k][better]
+        cur = lane[SEG_ROW0:]
+        keep = np.isin(cur, THIN)
+        cur[~keep] = best[~keep].astype(np.uint8)
+        return lane
+
+
+class YawTracks:
+    """The yaw-smoothing tracks of one stream (``g_yawTracks``): the previous frame's boxes (x, y, yaw)."""
+
+    def __init__(self) -> None:
+        self.tracks: List[Tuple[float, float, float]] = []
+
+    def reset(self) -> None:
+        self.tracks = []
+
+    def apply(self, boxes: List[Box3D]) -> List[Box3D]:
+        """Smooth ``boxes`` (in place, NMS order) against the previous frame and make them the new tracks."""
+        nxt = []
+        for b in boxes:
+            best, bd = None, 1e30
+            bx, by = F32(b.x), F32(b.y)
+            for t in self.tracks:
+                ddx, ddy = bx - F32(t[0]), by - F32(t[1])
+                dd = float(ddx * ddx + ddy * ddy)                       # float arithmetic, compared in double
+                if dd < YAW_GATE2 and dd < bd:
+                    best, bd = t, dd
+            if best is not None:
+                py = F32(best[2])
+                yaw = F32(b.yaw)
+                dy = F32(math.fmod(float(yaw) - float(py) + math.pi, 2 * math.pi))
+                if dy < 0:
+                    dy = F32(float(dy) + 2 * math.pi)
+                dy = F32(float(dy) - math.pi)
+                if abs(float(dy)) > math.pi / 2:
+                    yaw = F32(float(yaw) + (math.pi if dy < 0 else -math.pi))
+                    dy = F32(math.fmod(float(yaw) - float(py) + math.pi, 2 * math.pi))
+                    if dy < 0:
+                        dy = F32(float(dy) + 2 * math.pi)
+                    dy = F32(float(dy) - math.pi)
+                b.yaw = float(py + F32(YAW_EMA) * dy)
+            nxt.append((float(F32(b.x)), float(F32(b.y)), float(F32(b.yaw))))
+        self.tracks = nxt
+        return boxes
+
+
+class ModeHistory:
+    """The previously selected ego mode of one stream (``g_prevMode``)."""
+
+    def __init__(self) -> None:
+        self.prev: Optional[int] = None
+
+    def reset(self) -> None:
+        self.prev = None
+
+    def select(self, ego: np.ndarray, cfg: PostConfig = PostConfig()) -> int:
+        k, _ = select_mode(ego, self.prev, cfg)
+        self.prev = k
+        return k
+
+
+@dataclass
+class StreamState:
+    """All host temporal state of one ``stream.id``."""
+
+    seg: SegFusion = field(default_factory=SegFusion)
+    yaw: YawTracks = field(default_factory=YawTracks)
+    mode: ModeHistory = field(default_factory=ModeHistory)
+    frames: int = 0
+
+    def reset(self) -> None:
+        self.seg.reset()
+        self.yaw.reset()
+        self.mode.reset()
+        self.frames = 0
+
+    def describe(self) -> Dict[str, Any]:
+        return {"frames": self.frames, "seg_pose": self.seg.pose, "yaw_tracks": len(self.yaw.tracks),
+                "prev_mode": self.mode.prev}

code/tt_meteor/io.py

@@ -0,0 +1,34 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Input decoding and output encoding shared by the Python API and the HTTP server of meteor-p150.
+
+The implementation is the vendored ``ttaw.io`` (C08): point clouds (``bin`` / ``npy`` / ``npz`` / ``pcd`` /
+``list``), images, cameras + calibration, transforms, named arrays and the output encoders; numpy only, data
+parsing only, and every client mistake raises :class:`InputError`, which the server maps to HTTP 400. This module
+re-exports it with the Autoware input layout of this model bound as the default point fields, so ``model(points)``
+and ``POST /predict`` decode the same bytes into the same array.
+"""
+from __future__ import annotations
+
+from typing import Any, Mapping, Optional, Sequence
+
+from .ttaw import io as _io
+from .ttaw.io import *  # noqa: F401,F403  (the decoders and encoders: ttaw/API.md section 9)
+from .ttaw.io import MAX_POINTS_DEFAULT, PointCloud
+
+__all__ = list(_io.__all__)  # the same public names; the three below are rebound to this model's layout
+
+DEFAULT_POINT_FIELDS = tuple(["x", "y", "z", "intensity"])  # the Autoware input layout of this model
+
+
+def decode_points(spec: Mapping[str, Any], *, max_points: int = MAX_POINTS_DEFAULT, max_bytes: Optional[int] = None,
+                  default_fields: Sequence[str] = DEFAULT_POINT_FIELDS) -> PointCloud:
+    """The ``points`` envelope of ``/predict`` -> :class:`PointCloud` (``ttaw.io.decode_points`` with this model's
+    default fields)."""
+    return _io.decode_points(spec, max_points=max_points, max_bytes=max_bytes, default_fields=default_fields)
+
+
+def load_points(source: Any, *, fields: Optional[Sequence[str]] = None, fmt: Optional[str] = None,
+                frame_id: str = "base_link", default_fields: Sequence[str] = DEFAULT_POINT_FIELDS) -> PointCloud:
+    """Python-API point input (path, bytes + ``fmt``, (N, C) array or tensor, structured array, envelope,
+    :class:`PointCloud`) -> :class:`PointCloud` (``ttaw.io.load_points`` with this model's default fields)."""
+    return _io.load_points(source, fields=fields, fmt=fmt, frame_id=frame_id, default_fields=default_fields)

code/tt_meteor/reference/__init__.py

@@ -0,0 +1,25 @@
+# SPDX-License-Identifier: Apache-2.0
+"""CPU fp32 reference of METEOR (TIER IV, AutowareFoundation/meteor v1.0, ``meteor_v157c3Z.onnx``): the ground truth
+every PCC / output-agreement gate compares against (PLAN.md 0.3 item 1; research/meteor/SPEC.md section 6).
+
+Layout (importable without ttnn; torch / onnx are imported inside the functions that need them):
+
+- ``config.py``     the compile-time constants of the deployed graph (cameras, sizes, bins, crops, class lists) and
+                    the load-time options (``input_norm``, ``depth_mean_bins``), each with its source.
+- ``weights.py``    ``MeteorWeights``: the ONNX parsed as data with the vendored ``ttaw.weights`` (parameters
+                    addressed by their consuming node; conv attributes from the graph; BN already folded by the
+                    export), plus a graph check that the file is the deployed network.
+- ``model.py``      ``MeteorNet``: the whole graph in torch fp32 (a node-by-node transliteration) with per-module taps.
+- ``port_form.py``  the TT port's exact rewrites (input scale fold, merged heads, paint fold, tgate + tfuse merge,
+                    traj_stem without the zero history channels, the lift from host tables) with CPU proofs.
+- ``pipeline.py``   ``MeteorReference``: the end-to-end frame (network + the shared host pre / post-processing),
+                    stateless ``run_frame`` and the API-equivalent ``__call__``.
+- ``ort.py``        the ONNX Runtime oracle with any internal tensor as a named tap (research and tests only;
+                    onnxruntime is NOT a runtime dependency of the image).
+
+There is no Autoware package for METEOR: the pre- and post-processing reproduced in ``tt_meteor.host`` are METEOR's own
+runtimes (tier4/METEOR @ dc193a81e9959de57b751e269127ae9b67e3f300, ``deploy/cpp``).
+
+The ``to_dict()`` of this reference's output on the shipped sample is stored as
+``samples/<sample stem>.reference.json``: ``server/smoke_test.py`` compares the served output with it.
+"""

code/tt_meteor/reference/config.py

@@ -0,0 +1,147 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Compile-time constants of the METEOR port (``meteor_v157c3Z.onnx``, AutowareFoundation/meteor@v1.0).
+
+Every value here is either a static shape of the deployed graph or a constant baked into it; the source of each is
+the ONNX node it comes from (``research/meteor/SPEC.md``, cited as ``S:<line>``) or METEOR's runtime
+(``tier4/METEOR`` @ ``dc193a81e995``, ``deploy/cpp``). Changing one of them means a different graph, a new trace and
+a new image (PLAN.md section 0.4, class COMPILE). Host post-processing defaults (class RT-host) live in
+``host/postprocess.py`` (``PostConfig``); load-time options (``input_norm``, ``depth_mean_bins``) in
+:class:`MeteorConfig`.
+"""
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass, field
+from typing import Tuple
+
+__all__ = [
+    "ONNX_FILE", "ONNX_SHA256", "PARAM_YAML", "HF_REPO", "HF_TAG", "HF_REVISION", "METEOR_UPSTREAM_COMMIT",
+    "CAMERAS", "N_CAMS", "IMG_H", "IMG_W", "FEAT_H", "FEAT_W", "FPN_C", "CTX_C", "BEV_C", "BEV_H", "BEV_W",
+    "BEV_RES", "BEV_XF", "BEV_YH", "LIFT_H", "LIFT_W", "LIFT_RES", "DET_H", "DET_W", "DET_RES", "DEPTH_BINS",
+    "DEPTH_MIN", "DEPTH_STEP", "DEPTH_LOG_MAX", "LANE_CLASSES", "SEG2D_CLASSES", "DET2D_CLASSES", "DET3D_CLASSES",
+    "OCC_CLASSES", "TL_CLASSES", "DET2D_STRIDES", "PAINT_CLASSES", "OCC_CROP", "RISK_CROP", "EGO_MODES", "EGO_STEPS",
+    "EGO_DT", "EGO_DIM", "TRAJ_C", "DONOR", "IMAGENET_MEAN", "IMAGENET_STD", "INPUT_NORMS", "DEPTH_MEAN_BINS",
+    "MeteorConfig", "default_config", "depth_bin_centres",
+]
+
+# ---- artifact pins (S:99-118) ---------------------------------------------------------------------------------------
+ONNX_FILE = "meteor_v157c3Z.onnx"
+ONNX_SHA256 = "50397b4df76d33df2a71e8063b38d164399255008e1f0758c2a83675a200b1a7"
+PARAM_YAML = "meteor_v157.param.yaml"
+HF_REPO = "AutowareFoundation/meteor"
+HF_TAG = "v1.0"
+HF_REVISION = "01a5f6d71df5ecbbb5853ec600825481d57b9c6b"
+METEOR_UPSTREAM_COMMIT = "dc193a81e9959de57b751e269127ae9b67e3f300"   # tier4/METEOR, defines pre/post (no Autoware pkg)
+
+# ---- cameras and image branch (param.yaml:10, S:136-140) ------------------------------------------------------------
+CAMERAS: Tuple[str, ...] = ("CAM_FRONT_WIDE", "CAM_FRONT_LEFT", "CAM_FRONT_RIGHT", "CAM_BACK_WIDE", "CAM_BACK_LEFT",
+                            "CAM_BACK_RIGHT", "CAM_FRONT_NARROW", "CAM_BACK_NARROW")
+N_CAMS = 8
+IMG_H, IMG_W = 432, 768                 # network input per camera (the graph normalises u by 767, v by 431)
+FEAT_H, FEAT_W = 108, 192               # stride-4 FPN map
+FPN_C = 160                             # FPN width
+CTX_C = 96                              # painted context channels lifted to the BEV
+# the absent-camera donor rule of training (bevlane/dataset.py:17-21): zero image + the donor's K and pose
+DONOR = {"CAM_BACK_NARROW": "CAM_BACK_WIDE", "CAM_FRONT_NARROW": "CAM_FRONT_WIDE"}
+
+# ---- BEV (model.py:34-47, 82-89, 877-879; S:155-160) ----------------------------------------------------------------
+BEV_C = 96
+BEV_H, BEV_W = 800, 500                 # rows: x = +80 .. -80 m, cols: y = +50 .. -50 m
+BEV_RES = 0.2
+BEV_XF, BEV_YH = 80.0, 50.0             # forward extent, half width
+LIFT_H, LIFT_W = 400, 250               # 0.4 m lift grid (V52, LIFT_DIV 2): x 79.8 .. -79.8, y 49.8 .. -49.8
+LIFT_RES = 0.4
+DET_H, DET_W = 400, 250                 # 3D detection / traj / stationary / risk grids
+DET_RES = 0.4
+
+# ---- depth bins (S:325-336; SPEC section 10 risk 2) -----------------------------------------------------------------
+DEPTH_BINS = 64
+DEPTH_MIN, DEPTH_STEP = 1.0, 1.25       # LINEAR bins of the lift and the trained head: 1.0 + 1.25 * b metres
+DEPTH_LOG_MAX = 4.378896236419678       # ln(79.75) as the float32 constant of /Div_1: depth_mean's log-spaced centres
+LIFT_MIN_Z = 0.5                        # camera z clamp and validity threshold
+LIFT_MAX_DIST = 90.0                    # validity: Euclidean range from the camera < 90 m
+LIFT_BIN_CLIP = 62.9999                 # clip of the continuous bin index (/net/Clip_4, float32 62.99990081787109)
+LIFT_W_EPS = 0.05                       # + 0.05 on the interpolated bin probability (/net/Add_10)
+LIFT_DEN_EPS = 1e-4                     # max(sum_n w, 1e-4) (/net/Clip_6)
+
+# ---- heads -----------------------------------------------------------------------------------------------------
+LANE_CLASSES = ("bg", "road", "sidewalk", "crosswalk", "laneline", "stopline", "road_edge", "marking", "parking")
+SEG2D_CLASSES = ("bg", "obstacle", "car", "truck", "bus", "motorcycle", "bicycle", "pedestrian", "marking",
+                 "traffic_light", "sign", "road", "sidewalk", "lane_stop_line", "crosswalk", "unused", "wall",
+                 "building", "vegetation", "sky", "pole")          # comlops-21cls-autolabel-2504.csv (0, 15 unused)
+DET2D_CLASSES = ("obs", "car", "truck", "bus", "bicycle", "motorcycle", "pedestrian", "road_paint", "traffic_light",
+                 "traffic_sign")                                    # comlops-instance-2510.csv
+DET3D_CLASSES = ("vehicle", "vru")
+OCC_CLASSES = ("free", "obstacle", "vehicle", "two_wheeler", "pedestrian", "road", "sidewalk", "vegetation",
+               "building_wall", "pole_sign_light")                 # extract_occ.py:4-8; 16 z-bins of 0.4 m from -1 m
+OCC_Z_BINS = 16
+TL_CLASSES = ("none", "green", "yellow", "red")
+DET2D_STRIDES = (4, 8, 16)
+PAINT_CLASSES = (2, 3, 4, 5, 6, 7, 8, 13)  # seg2d classes painted into the context (/net/Gather indices; checked)
+SEG2D_CLAMP = 30.0                      # +-30 clamp, ONLY before the seg2d argmax (the paint softmax is unclamped)
+OCC_CROP = ((200, 600), (50, 450))      # rows, cols of the RAW BEV (/net/Slice_8, Slice_9)
+RISK_CROP = ((200, 600), (125, 375))    # rows, cols of the FUSED BEV (/net/Slice_11, Slice_12)
+TGATE_SCALE = 4.0                       # fused = bev + tfuse(bev * 4 * softmax(tgate)[0]) (/net/Mul_12)
+
+# ---- planner (S:315, 4.4) --------------------------------------------------------------------------------------
+EGO_MODES, EGO_STEPS, EGO_DT = 3, 6, 0.5
+EGO_DIM = 42                            # [0:36] 3 paths x 6 x (x, y); [36:39] mode logits; 39 steer, 40 accel, 41 brake
+TRAJ_C = 39                             # per-cell agent futures: 3 modes x 6 x (dx, dy) + 3 mode logits
+V0_SCALE = 15.0                         # kin_delta input [v0 / 15, 0, ..., 0] (7 wide)
+KIN_DIM = 7
+DEC_SPEED_MAX = 25.0                    # decoupled decoder: v = min(softplus(.), 25)
+RISK_X_HALF, RISK_Y_HALF = 40.0, 25.0   # risk grid +-40 m x +-25 m at 0.2 m (400 x 250)
+E2E_LN_EPS = 1e-5                       # LayerNormalization(139) epsilon (float32 9.99999974738e-06)
+E2E_SCALE = 3.0                         # ego[:36] += 3 * tanh(mlp(LN(...)))
+SEG_REFINE_CLAMP, SEG_REFINE_SCALE = 20.0, 8.0     # lane_logit = clamp(lane_pre, +-20) + 8 tanh(res / 8)
+BOX_HM_CLAMP, BOX_REG_CLAMP, BOX_REFINE_SCALE = 15.0, 20.0, 6.0
+
+# ---- input normalisation (D12; S:209-229, section 10 risk 1) ---------------------------------------------------------
+IMAGENET_MEAN = (0.485, 0.456, 0.406)
+IMAGENET_STD = (0.229, 0.224, 0.225)
+INPUT_NORMS = ("onnx", "imagenet")      # onnx: x / 255 as released (default); imagenet: (x / 255 - mean) / std
+DEPTH_MEAN_BINS = ("log", "linear")     # depth_mean bin centres: log as exported (default, parity) or linear (trained)
+
+
+def depth_bin_centres(kind: str = "log"):
+    """The 64 bin centres of ``depth_mean`` in metres: ``log`` = exp(r * ln(79.75) / 63) in float32 exactly as the
+    graph (/Mul_1 -> /Exp), ``linear`` = 1 + 1.25 r (the bins the head and the lift were trained with)."""
+    import numpy as np
+
+    r = np.arange(DEPTH_BINS, dtype=np.float32)
+    if kind == "log":
+        step = np.float32(DEPTH_LOG_MAX) / np.float32(DEPTH_BINS - 1)
+        return np.exp(r * step + np.float32(0.0)).astype(np.float32)
+    if kind == "linear":
+        return (np.float32(DEPTH_MIN) + np.float32(DEPTH_STEP) * r).astype(np.float32)
+    raise ValueError(f"depth_mean bins must be one of {DEPTH_MEAN_BINS}, not {kind!r}")
+
+
+@dataclass(frozen=True)
+class MeteorConfig:
+    """Load-time options of the reference (and of the TT port, ``METEOR_INPUT_NORM`` / ``METEOR_DEPTH_MEAN_BINS``).
+
+    ``input_norm="onnx"`` reproduces the released graph (only /255; D12 default, validated against ORT);
+    ``"imagenet"`` adds the training normalisation after /255 (SPEC section 10 risk 1; an in-memory patched ONNX is
+    its oracle). ``depth_mean_bins="log"`` keeps ONNX parity, ``"linear"`` gives metric depth as trained."""
+
+    input_norm: str = "onnx"
+    depth_mean_bins: str = "log"
+    cameras: Tuple[str, ...] = field(default=CAMERAS)
+
+    def check(self) -> "MeteorConfig":
+        if self.input_norm not in INPUT_NORMS:
+            raise ValueError(f"input_norm must be one of {INPUT_NORMS}, not {self.input_norm!r}")
+        if self.depth_mean_bins not in DEPTH_MEAN_BINS:
+            raise ValueError(f"depth_mean_bins must be one of {DEPTH_MEAN_BINS}, not {self.depth_mean_bins!r}")
+        if tuple(self.cameras) != CAMERAS:
+            raise ValueError("the camera order is fixed by training (param.yaml:10)")
+        return self
+
+
+def default_config(**overrides) -> MeteorConfig:
+    return MeteorConfig(**overrides).check()
+
+
+# sanity: the log centres span 1 .. 79.75 m (S:183)
+assert abs(math.exp(DEPTH_LOG_MAX) - 79.75) < 1e-4

code/tt_meteor/reference/model.py

@@ -0,0 +1,539 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Pure-PyTorch fp32 re-implementation of ``meteor_v157c3Z.onnx`` with per-module taps.
+
+A node-by-node transliteration of the deployed graph (2,021 ONNX nodes; ``research/meteor/SPEC.md`` sections 4.2-4.4,
+cited ``S:<line>``), written from the graph itself (no upstream code is imported or executed): every convolution runs
+with the weights and the attributes of its ONNX node (``reference.weights``), every non-conv constant (attention
+pools, LayerNorm, gates, pooling matrices, lift ground points) is read from its consuming node, and the arithmetic
+keeps the graph's operation order where it decides a rounding or a discrete choice (lift projection, validity and
+depth-bin selection, risk sampling, decoupled decoder, depth_mean). Shape glue (Shape / Gather / Concat / Reshape of
+shapes, ~470 nodes) disappears. ``MeteorNet.forward`` returns the 19 graph outputs with the ONNX names, dtypes and
+shapes; ``taps=`` records the intermediates below (names shared with ``reference.ort.ORT_TAPS``, the device gates
+and the goldens).
+
+Module boundaries (what a TT stage computes) and taps:
+
+==========================  ===========================  ====================================================
+tap                         shape                        graph tensor / meaning
+==========================  ===========================  ====================================================
+``img.input``               [8, 3, 432, 768]             uint8 / 255 (+ ImageNet mean/std with input_norm=imagenet)
+``img.stem``                [8, 64, 108, 192]            7x7 s2 + ReLU + maxpool 3x3 s2
+``img.layer1..4``           [8, 64..512, ...]            ResNet-34 stages (BasicBlocks, BN folded)
+``img.fpn``                 [8, 160, 108, 192]           lat1..4 + bilinear (x2, x4, 14x24 -> 108x192) + fuse 3x3 ReLU
+``seg2d.logits``            [8, 21, 108, 192]            SegHeadED raw logits (the PointPainting input, unclamped)
+``seg2d.clamped``           [8, 21, 108, 192]            nan_to_num + clamp +-30 (the seg2d argmax input only)
+``depth.logits`` / ``.prob``  [8, 64, 108, 192]          depth head 1x1 / softmax over the 64 linear bins
+``ctx.painted``             [8, 96, 108, 192]            ctx 1x1 + paint_proj(softmax(seg)[2,3,4,5,6,7,8,13])
+``lift.grid``               [8, 100000, 1, 2]            normalised (u, v) of the 400x250 ground points per camera
+``lift.valid``              [8, 100000] bool             z > 0.5, inside the image, range < 90 m
+``lift.weight``             [8, 1, 100000]               (interpolated bin probability + 0.05) * valid
+``lift.bev``                [1, 96, 400, 250]            sum_n ctx_s w / max(sum_n w, 1e-4)
+``bev.raw``                 [1, 96, 800, 500]            bilinear x2 of lift.bev
+``bev.gate``                [1, 1, 800, 500]             4 softmax(tgate(raw))[0]
+``bev.fused``               [1, 96, 800, 500]            raw + tfuse(raw * gate)
+``lane.pre``                [1, 9, 800, 500]             lane decoder + lane_branch on channels 4..6
+``det.feat``                [1, 128, 400, 250]           det stem (ED) output
+``det.hm_pre/.reg_pre``     [1, 2 | 6, 400, 250]         heads before the box refiner
+``occ.feat``                [1, 192, 200, 200]           occ stem on raw rows 200:600, cols 50:450
+``traj.stem / .feat``       [1, 128 | 256, 400, 250]     traj stem; cat(stem, det.feat) + agent_delta
+``traj.agent_delta``        [1, 256, 1, 1]               agent attention pool -> 1x1 conv
+``stat.feat``               [1, 48, 400, 250]            delta_stat convs on avgpool2(|raw|)
+``risk.feat``               [1, 64, 400, 250]            risk ConvBlocks on fused rows 200:600, cols 125:375
+``ego.*``                   [1, 42] (and others)         planner chain: stem, mlp, attn, plus_attn, sem_in, sem,
+                                                         plus_sem, kin, plus_kin, risk_sample, after_risk,
+                                                         dec_head, dec_wp, after_dec
+``bev.fused_mean``          [1, 96]                      global mean of the fused BEV (computed once; S:368-369)
+``refiner.seg_res``         [1, 9, 800, 500]             seg refiner residual (before 8 tanh(x / 8))
+``refiner.box_res``         [1, 8, 400, 250]             box refiner residual (before 6 tanh(x / 6))
+``refiner.e2e_ln / _res``   [1, 139] / [1, 36]           e2e LayerNorm output / 3 tanh(mlp)
+==========================  ===========================  ====================================================
+
+The 19 outputs are tapped under their ONNX names (``lane``, ``depth``, ``seg2d``, ``hm`` ... ``depth_mean``).
+"""
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+import numpy as np
+
+from ..ttaw.golden import NULL_TAPS, TapRegistry
+from . import config as C
+from .config import MeteorConfig, default_config
+from .weights import MeteorWeights
+
+__all__ = ["MeteorNet", "OUTPUT_NAMES", "child"]
+
+OUTPUT_NAMES = ("lane", "depth", "seg2d", "hm", "reg", "hm2d_s0", "hm2d_s1", "hm2d_s2", "reg2d_s0", "reg2d_s1",
+                "reg2d_s2", "ego", "occ", "traj", "stationary", "tl", "risk", "lane_logit", "depth_mean")
+
+
+def _torch():
+    import torch
+
+    return torch
+
+
+def child(module: str, index: int) -> str:
+    """ONNX scope of child ``index`` of a Sequential: ``child("seg_head/d1/d1.3", 0)`` = ``seg_head/d1/d1.3/d1.3.0``."""
+    return f"{module}/{module.rsplit('/', 1)[-1]}.{index}"
+
+
+class MeteorNet:
+    """The network as fp32 torch functions over the ONNX weights (no autograd, no nn.Module state)."""
+
+    def __init__(self, weights: MeteorWeights, cfg: Optional[MeteorConfig] = None, *, check_graph: bool = True):
+        torch = _torch()
+        self.cfg = (cfg or default_config()).check()
+        self.weights = weights
+        if check_graph:
+            weights.check_graph()
+        f32 = lambda a: torch.from_numpy(np.array(a, dtype=np.float32))  # own writable copy
+        self._conv: Dict[str, Any] = {}
+        for module, p in weights.iter_convs():
+            pads = tuple(int(v) for v in p.pads)
+            if len(pads) != 4 or pads[:2] != pads[2:] or p.group != 1 or tuple(p.dilations) != (1, 1):
+                raise ValueError(f"{module}: unsupported conv attributes {p}")
+            self._conv[module] = (f32(p.weight), None if p.bias is None else f32(p.bias),
+                                  tuple(int(s) for s in p.strides), pads[:2])
+        self._gemm: Dict[str, Any] = {}
+        for module in weights.gemm_modules():
+            g = weights.gemm(module)
+            if g.trans_a or not g.trans_b or g.alpha != 1.0 or g.beta != 1.0:
+                raise ValueError(f"{module}: unsupported Gemm attributes")
+            self._gemm[module] = (f32(g.weight), f32(g.bias))
+        self._pool = {name: f32(weights.pool_matrix(name)) for name in
+                      ("ego_stem_w", "ego_stem_h", "tl_w", "tl_h", "fused_pool_w", "fused_pool_h", "sem_w", "sem_h",
+                       "det_pool_w", "det_pool_h", "fused_mean_w", "fused_mean_h", "e2e_mean_w", "e2e_mean_h")}
+        self._mha = {name: weights.mha(name) for name in ("ego_attn", "agent_attn")}
+        self._mha_t = {name: (f32(m.in_weight), f32(m.in_bias), f32(m.out_weight), f32(m.out_bias))
+                       for name, m in self._mha.items()}
+        self._queries = {"ego": f32(weights.queries("ego")), "agent": f32(weights.queries("agent"))}
+        g, b, eps = weights.layer_norm()
+        self._ln = (f32(g), f32(b), float(eps))
+        self._risk_gate = f32(np.asarray(weights.risk_gate(), np.float32).reshape(1))
+        self._dec_gate = f32(np.asarray(weights.dec_gate(), np.float32).reshape(1))
+        self._ground = f32(weights.lift_ground_points())                     # [1, 4, 100000]
+        zc = weights.zero_constants()
+        self._hist = f32(zc["hist"])                                         # [1, 96, 800, 500] zeros (baked history)
+        self._mask = f32(zc["mask"])                                         # [1, 1, 800, 500] zeros
+        self._paint = torch.tensor(weights.paint_classes(), dtype=torch.long)
+        self._depth_centres = f32(C.depth_bin_centres(self.cfg.depth_mean_bins)).reshape(1, 1, C.DEPTH_BINS, 1, 1)
+
+    # ---- primitives ------------------------------------------------------------------------------------------------
+    def conv(self, x, module: str, relu: bool = False):
+        F = _torch().nn.functional
+        w, b, stride, pad = self._conv[module]
+        y = F.conv2d(x, w, b, stride=stride, padding=pad)
+        return F.relu(y) if relu else y
+
+    def block(self, x, module: str):
+        """ConvBlock = (3x3 conv, BN (folded), ReLU) x 2: children ``.0`` and ``.3`` (S:318)."""
+        return self.conv(self.conv(x, child(module, 0), True), child(module, 3), True)
+
+    def down(self, x, module: str):
+        """stride-2 3x3 conv + ReLU (child ``.0``) then a ConvBlock (child ``.3``): the encoder steps of the EDs."""
+        return self.block(self.conv(x, child(module, 0), True), child(module, 3))
+
+    def gemm(self, x, module: str):
+        w, b = self._gemm[module]
+        return _torch().addmm(b, x, w.t())
+
+    @staticmethod
+    def resize(x, hw):
+        """ONNX Resize linear / half_pixel to a target size = F.interpolate(bilinear, align_corners=False)."""
+        F = _torch().nn.functional
+        return F.interpolate(x, size=tuple(int(v) for v in hw), mode="bilinear", align_corners=False)
+
+    def pool2(self, x, w_name: str, h_name: str):
+        """The exported adaptive pool / mean: (H_mat @ (x @ W_mat)) on the last two dims (S:356-366)."""
+        torch = _torch()
+        return torch.matmul(self._pool[h_name], torch.matmul(x, self._pool[w_name]))
+
+    @staticmethod
+    def nan_to_num(x):
+        """The graph's IsNaN / IsInf / Where chain (ONNX nodes 190-213 and 233-255): NaN -> 0, +inf -> 30,
+        -inf -> -30; the identity for finite values."""
+        torch = _torch()
+        x = torch.where(torch.isnan(x), torch.zeros_like(x), x)
+        x = torch.where(torch.isinf(x) & (x > 0), torch.full_like(x, 30.0), x)
+        return torch.where(torch.isinf(x) & (x < 0), torch.full_like(x, -30.0), x)
+
+    def mha_pool(self, name: str, keys, queries):
+        """``nn.MultiheadAttention(q=learned queries, k=v=keys)`` as exported (torch 2.1, need_weights=False):
+        q and k|v projections, q / sqrt(head_dim), softmax over the 400 keys, out-projection.
+        ``keys`` [L, 1, E] (sequence first), ``queries`` [1, Q, E] -> [1, Q, E]."""
+        torch = _torch()
+        m = self._mha[name]
+        w_in, b_in, w_out, b_out = self._mha_t[name]
+        e, h = m.embed_dim, m.num_heads
+        hd = e // h
+        q_in = queries.transpose(0, 1)                                      # [Q, 1, E]
+        q = b_in[:e] + torch.matmul(q_in, w_in[:e].t())                     # /Add_4 = bias + MatMul
+        kv = b_in[e:] + torch.matmul(keys, w_in[e:].t())                    # [L, 1, 2E]
+        k, v = kv[..., :e], kv[..., e:]
+        nq, nk = q.shape[0], k.shape[0]
+        qh = q.reshape(nq, h, hd).transpose(0, 1)                           # [H, Q, hd]
+        kh = k.reshape(nk, h, hd).transpose(0, 1)                           # [H, L, hd]
+        vh = v.reshape(nk, h, hd).transpose(0, 1)
+        att = torch.softmax(torch.matmul(qh / m.scale_divisor, kh.transpose(1, 2)), dim=-1)
+        o = torch.matmul(att, vh).transpose(0, 1).reshape(nq, e)            # [Q, E]
+        o = torch.addmm(b_out, o, w_out.t())                                # out-projection (Gemm)
+        return o.reshape(nq, 1, e).transpose(0, 1)                          # [1, Q, E]
+
+    # ---- stages ----------------------------------------------------------------------------------------------------
+    def normalize(self, imgs_u8, present=None):
+        """uint8 [1, 8, 3, 432, 768] -> [8, 3, 432, 768] float32: /255 (``onnx``) or (x/255 - mean)/std with absent
+        cameras zeroed AFTER normalisation (``imagenet``, SPEC section 2 imgs row)."""
+        torch = _torch()
+        x = imgs_u8.to(torch.float32) / 255.0
+        if self.cfg.input_norm == "imagenet":
+            mean = torch.tensor(C.IMAGENET_MEAN, dtype=torch.float32).reshape(1, 1, 3, 1, 1)
+            std = torch.tensor(C.IMAGENET_STD, dtype=torch.float32).reshape(1, 1, 3, 1, 1)
+            x = (x - mean) / std
+            if present is not None:
+                keep = torch.as_tensor(np.asarray(present, np.float32)).reshape(1, -1, 1, 1, 1)
+                x = x * keep
+        return x.reshape(C.N_CAMS, 3, C.IMG_H, C.IMG_W)
+
+    def image_encoder(self, x, taps: TapRegistry = NULL_TAPS):
+        """ResNet-34 (torchvision layout, BN folded) + FPN -> f [8, 160, 108, 192] (S:301-302)."""
+        F = _torch().nn.functional
+        y = F.max_pool2d(self.conv(x, "stem/stem.0", True), kernel_size=3, stride=2, padding=1)
+        taps.tap("img.stem", y)
+        feats = []
+        for li, nb in enumerate((3, 4, 6, 3), start=1):
+            for bi in range(nb):
+                p = f"layer{li}/layer{li}.{bi}"
+                ds = f"{p}/downsample/downsample.0"
+                idt = self.conv(y, ds) if ds in self._conv else y
+                y = F.relu(self.conv(self.conv(y, f"{p}/conv1", True), f"{p}/conv2") + idt)
+            feats.append(taps.tap(f"img.layer{li}", y))
+        c1, c2, c3, c4 = feats
+        hw = (C.FEAT_H, C.FEAT_W)
+        s = self.conv(c1, "lat1") + self.resize(self.conv(c2, "lat2"), hw)
+        s = s + self.resize(self.conv(c3, "lat3"), hw)
+        s = s + self.resize(self.conv(c4, "lat4"), hw)                  # 14x24 -> 108x192: non-integer (S:302)
+        return taps.tap("img.fpn", self.conv(s, "fuse/fuse.0", True))
+
+    def seg2d_head(self, f, taps: TapRegistry = NULL_TAPS):
+        """SegHeadED -> raw logits [8, 21, 108, 192] (S:303)."""
+        d1 = self.down(f, "seg_head/d1")
+        d2 = self.down(d1, "seg_head/d2")
+        m1 = self.block(d1 + self.resize(self.conv(d2, "seg_head/u1"), d1.shape[-2:]), "seg_head/m1")
+        u2 = self.resize(self.conv(m1, "seg_head/u2"), f.shape[-2:])
+        y = self.conv(self.conv(self.conv(f, "seg_head/skip") + u2, "seg_head/out/out.0", True), "seg_head/out/out.3")
+        return taps.tap("seg2d.logits", y)
+
+    def depth_head(self, f, taps: TapRegistry = NULL_TAPS):
+        """4 ConvBlocks 160->128->128->96->64, 1x1 -> 64 bins, softmax over the bins (S:304)."""
+        torch = _torch()
+        y = f
+        for i in range(4):
+            y = self.block(y, f"depth_head/depth_head.{i}")
+        logits = taps.tap("depth.logits", self.conv(y, "depth_head/depth_head.4"))
+        return logits, taps.tap("depth.prob", torch.softmax(logits, dim=1))
+
+    def det2d_heads(self, f):
+        """det2d stem + d8 / d16 pyramid, 1x1 heads at strides 4 / 8 / 16 (S:305) -> the six [1, 8, k, h, w] outputs."""
+        s = self.block(f, "det2d_stem")
+        d8 = self.down(s, "det2d_d8")
+        d16 = self.down(d8, "det2d_d16")
+        out = {}
+        for i, (x, sfx) in enumerate(((s, ""), (d8, "8"), (d16, "16"))):
+            for kind, ch in (("hm2d", 10), ("reg2d", 4)):
+                y = self.conv(x, f"{kind}_head{sfx}")
+                out[f"{kind}_s{i}"] = y.reshape(1, C.N_CAMS, ch, y.shape[-2], y.shape[-1])
+        return out
+
+    def tl_head(self, f):
+        """cat(f[cam 0], f[cam 6]) -> s2 conv + CB -> s2 conv + CB -> global mean (MatMul consts) -> Linear (S:306)."""
+        torch = _torch()
+        x = torch.cat([f[0:1], f[6:7]], dim=1)
+        x = self.block(self.conv(x, "tl_head/tl_head.0", True), "tl_head/tl_head.3")
+        x = self.block(self.conv(x, "tl_head/tl_head.4", True), "tl_head/tl_head.7")
+        x = self.pool2(x, "tl_w", "tl_h").flatten(1)
+        return self.gemm(x, "tl_fc")
+
+    def context(self, f, seg_logits, taps: TapRegistry = NULL_TAPS):
+        """ctx 1x1 + paint_proj(softmax(nan_to_num(raw seg logits))[paint classes]) (S:307; UNCLAMPED logits)."""
+        torch = _torch()
+        p = torch.softmax(self.nan_to_num(seg_logits), dim=1).index_select(1, self._paint)
+        return taps.tap("ctx.painted", self.conv(f, "ctx") + self.conv(p, "paint_proj"))
+
+    def lift(self, ctx, prob, K, T_cam_ego, taps: TapRegistry = NULL_TAPS):
+        """Depth-gated IPM lift to the 0.4 m grid, then bilinear x2 to the raw BEV (S:333-351, ONNX nodes 261-421)."""
+        torch = _torch()
+        F = torch.nn.functional
+        n = C.N_CAMS
+        T = T_cam_ego.reshape(n, 4, 4).to(torch.float32)
+        Km = K.reshape(n, 3, 3).to(torch.float32)
+        pc = torch.matmul(T, self._ground)                                  # [8, 4, 100000]
+        x, y, z = pc[:, 0], pc[:, 1], pc[:, 2]
+        zc = torch.clamp(z, min=C.LIFT_MIN_Z)
+        u = Km[:, 0, 0].unsqueeze(-1) * x / zc + Km[:, 0, 2].unsqueeze(-1)
+        v = Km[:, 1, 1].unsqueeze(-1) * y / zc + Km[:, 1, 2].unsqueeze(-1)
+        dist = torch.sqrt(x * x + y * y + z * z)
+        valid = ((z > C.LIFT_MIN_Z) & (u >= 0.0) & (u < float(C.IMG_W)) & (v >= 0.0) & (v < float(C.IMG_H))
+                 & (dist < C.LIFT_MAX_DIST))
+        taps.tap("lift.valid", valid)
+        gu = torch.clamp(u / float(C.IMG_W - 1) * 2.0 - 1.0, -2.0, 2.0)
+        gv = torch.clamp(v / float(C.IMG_H - 1) * 2.0 - 1.0, -2.0, 2.0)
+        grid = taps.tap("lift.grid", torch.stack([gu, gv], dim=-1).unsqueeze(2))      # [8, 100000, 1, 2]
+        ctx_s = F.grid_sample(ctx, grid, mode="bilinear", padding_mode="zeros", align_corners=False).squeeze(3)
+        prob_s = F.grid_sample(prob, grid, mode="bilinear", padding_mode="zeros", align_corners=False).squeeze(3)
+        b = torch.clamp((dist - C.DEPTH_MIN) / C.DEPTH_STEP, 0.0, float(np.float32(C.LIFT_BIN_CLIP)))
+        b0 = torch.floor(b).to(torch.int64)
+        fr = (b - b0.to(torch.float32)).unsqueeze(1)                        # [8, 1, 100000]
+        g0 = torch.gather(prob_s, 1, b0.unsqueeze(1))
+        g1 = torch.gather(prob_s, 1, torch.clamp(b0 + 1, max=C.DEPTH_BINS - 1).unsqueeze(1))
+        w = (g0 * (1.0 - fr) + g1 * fr + C.LIFT_W_EPS) * valid.unsqueeze(1).to(torch.float32)
+        taps.tap("lift.weight", w)
+        wn = w.reshape(1, n, 1, -1)
+        num = (ctx_s.reshape(1, n, C.CTX_C, -1) * wn).sum(dim=1)            # [1, 96, 100000]
+        den = torch.clamp(wn.sum(dim=1), min=C.LIFT_DEN_EPS)                # [1, 1, 100000]
+        bev04 = taps.tap("lift.bev", (num / den).reshape(1, C.CTX_C, C.LIFT_H, C.LIFT_W))
+        return taps.tap("bev.raw", self.resize(bev04, (C.BEV_H, C.BEV_W)))
+
+    def temporal_fuse(self, raw, taps: TapRegistry = NULL_TAPS):
+        """No-history temporal fuse: fused = raw + tfuse(raw * 4 softmax(tgate(raw))[0]) (S:308)."""
+        torch = _torch()
+        gate = taps.tap("bev.gate", torch.softmax(self.conv(raw, "tgate_slim"), dim=1)[:, 0:1] * C.TGATE_SCALE)
+        t = self.block(self.conv(raw * gate, "tfuse3_slim/tfuse3_slim.0", True), "tfuse3_slim/tfuse3_slim.3")
+        return taps.tap("bev.fused", raw + t)
+
+    def lane_decoder(self, raw, taps: TapRegistry = NULL_TAPS):
+        """LaneDecED + lane_branch added to channels 4..6 -> lane_pre [1, 9, 800, 500] (S:309)."""
+        torch = _torch()
+        skip = self.block(raw, "dec/skip")
+        d1 = self.down(raw, "dec/d1")
+        d2 = self.down(d1, "dec/d2")
+        m1 = self.block(d1 + self.resize(self.conv(d2, "dec/u1"), d1.shape[-2:]), "dec/m1")
+        u2 = self.resize(self.conv(m1, "dec/u2"), skip.shape[-2:])
+        out = self.conv(self.conv(skip + u2, "dec/out/out.0", True), "dec/out/out.3")
+        lb = self.conv(self.conv(self.conv(raw, "lane_branch/lane_branch.0", True), "lane_branch/lane_branch.3", True),
+                       "lane_branch/lane_branch.6")
+        return taps.tap("lane.pre", torch.cat([out[:, :4], out[:, 4:7] + lb, out[:, 7:]], dim=1))
+
+    def det_stem(self, raw, taps: TapRegistry = NULL_TAPS):
+        """_DetStemED on the raw BEV -> det_feat [1, 128, 400, 250], hm_pre, reg_pre (S:310)."""
+        s = self.down(raw, "det_stem/stem")
+        d8 = self.down(s, "det_stem/d8")
+        feat = self.conv(s + self.resize(self.conv(d8, "det_stem/u"), s.shape[-2:]), "det_stem/m/m.0", True)
+        taps.tap("det.feat", feat)
+        hm = taps.tap("det.hm_pre", self.conv(feat, "hm_head"))
+        return feat, hm, taps.tap("det.reg_pre", self.conv(feat, "reg_head"))
+
+    def occupancy(self, raw, taps: TapRegistry = NULL_TAPS):
+        """occ stem on the raw BEV crop rows 200:600, cols 50:450 -> 1x1 160 -> view [1, 10, 16, 200, 200] (S:311)."""
+        (r0, r1), (c0, c1) = C.OCC_CROP
+        x = self.down(raw[:, :, r0:r1, c0:c1], "occ_stem")
+        taps.tap("occ.feat", x)
+        y = self.conv(x, "occ_head")
+        return y.reshape(1, len(C.OCC_CLASSES), C.OCC_Z_BINS, y.shape[-2], y.shape[-1])
+
+    def agent_forecast(self, raw, fused, det_feat, reg_pre, taps: TapRegistry = NULL_TAPS):
+        """traj_stem on cat(fused, mot = (raw - hist) * mask = 0), agent attention pool, traj head (S:312)."""
+        torch = _torch()
+        mot = (raw - self._hist) * self._mask                              # the baked-out history: all zeros
+        stem = self.block(self.conv(torch.cat([fused, mot], dim=1), "traj_stem/traj_stem.0", True),
+                          "traj_stem/traj_stem.3")
+        del mot
+        taps.tap("traj.stem", stem)
+        reg46 = reg_pre[:, 4:6]
+        keys = self.pool2(det_feat, "det_pool_w", "det_pool_h").reshape(1, det_feat.shape[1], -1).permute(2, 0, 1)
+        att = self.mha_pool("agent_attn", keys, self._queries["agent"])     # [1, 4, 128]
+        delta = self.conv(att.mean(dim=1).reshape(1, -1, 1, 1), "agent_delta")
+        taps.tap("traj.agent_delta", delta)
+        feat = taps.tap("traj.feat", torch.cat([stem, det_feat], dim=1) + delta)
+        return self.conv(torch.cat([feat, reg46], dim=1), "traj_head")
+
+    def stationary(self, raw, taps: TapRegistry = NULL_TAPS):
+        """delta_stat on avgpool2(|raw - 0|) (S:313)."""
+        F = _torch().nn.functional
+        x = F.avg_pool2d((raw - self._hist).abs(), kernel_size=2, stride=2)
+        x = self.conv(self.conv(x, "delta_stat/delta_stat.1", True), "delta_stat/delta_stat.4", True)
+        taps.tap("stat.feat", x)
+        return self.conv(x, "delta_stat/delta_stat.7")
+
+    def risk(self, fused, taps: TapRegistry = NULL_TAPS):
+        """risk head on the fused BEV crop rows 200:600, cols 125:375 (S:314)."""
+        (r0, r1), (c0, c1) = C.RISK_CROP
+        x = self.block(self.block(fused[:, :, r0:r1, c0:c1], "risk_head/risk_head.0"), "risk_head/risk_head.1")
+        taps.tap("risk.feat", x)
+        return self.conv(x, "risk_head/risk_head.2")
+
+    def planner(self, fused, lane_pre, hm_pre, risk, v0, taps: TapRegistry = NULL_TAPS):
+        """Ego planner up to the e2e refiner (S:315): ego_stem + MLP, + ego_attn, + sem_ego, + kin, risk-aware
+        mode logits, decoupled decoder. Returns ``(ego_after_dec [1, 42], fused_mean [1, 96])``."""
+        torch = _torch()
+        F = torch.nn.functional
+        with taps.scope("ego"):
+            e = self.block(self.conv(fused, "ego_stem/ego_stem.0", True), "ego_stem/ego_stem.3")     # s4
+            e = self.block(self.conv(e, "ego_stem/ego_stem.4", True), "ego_stem/ego_stem.7")
+            e = self.block(self.conv(e, "ego_stem/ego_stem.8", True), "ego_stem/ego_stem.11")
+            e = self.conv(e, "ego_stem/ego_stem.12", True)                                           # [1, 256, 25, 16]
+            e = taps.tap("stem", self.pool2(e, "ego_stem_w", "ego_stem_h").flatten(1))
+            v0 = v0.reshape(1, 1).to(torch.float32)
+            h = torch.cat([e, v0], dim=1)
+            for i in (0, 2, 4):
+                h = F.relu(self.gemm(h, f"ego_mlp/ego_mlp.{i}"))
+            ego = taps.tap("mlp", self.gemm(h, "ego_mlp/ego_mlp.6"))
+            keys = self.pool2(fused, "fused_pool_w", "fused_pool_h").reshape(1, fused.shape[1], -1).permute(2, 0, 1)
+            att = self.mha_pool("ego_attn", keys, self._queries["ego"])                              # [1, 3, 96]
+            ego = taps.tap("plus_attn", ego + taps.tap("attn", self.gemm(att.flatten(1), "ego_delta")))
+            sem_in = torch.cat([F.avg_pool2d(torch.softmax(lane_pre, dim=1), kernel_size=2, stride=2),
+                                torch.sigmoid(hm_pre)], dim=1)
+            taps.tap("sem_in", sem_in)
+            s = self.conv(self.conv(self.conv(sem_in, "sem_ego/sem_ego.0", True), "sem_ego/sem_ego.3", True),
+                          "sem_ego/sem_ego.6", True)
+            s = self.pool2(s, "sem_w", "sem_h").flatten(1)
+            s = self.gemm(F.relu(self.gemm(s, "sem_ego/sem_ego.11")), "sem_ego/sem_ego.13")
+            ego = taps.tap("plus_sem", ego + taps.tap("sem", s))
+            kin_in = torch.zeros(1, C.KIN_DIM, dtype=torch.float32)
+            kin_in[0, 0] = v0.reshape(()) / C.V0_SCALE
+            ego = taps.tap("plus_kin", ego + taps.tap("kin", self.gemm(kin_in, "kin_delta")))
+            # risk-aware mode logits: sample sigmoid(risk) (border padding) at each path's 6 waypoints (S:315)
+            paths = ego[:, :36].reshape(1, C.EGO_MODES, C.EGO_STEPS, 2)
+            gx = -paths[..., 1] / C.RISK_Y_HALF
+            gy = (C.RISK_X_HALF - paths[..., 0]) / C.RISK_X_HALF - 1.0
+            samp = F.grid_sample(torch.sigmoid(risk), torch.stack([gx, gy], dim=-1), mode="bilinear",
+                                 padding_mode="border", align_corners=False)                        # [1, 1, 3, 6]
+            samp = taps.tap("risk_sample", samp.mean(dim=3).squeeze(1))                              # [1, 3]
+            ego = torch.cat([ego[:, :36], ego[:, 36:39] - self._risk_gate * samp, ego[:, 39:]], dim=1)
+            taps.tap("after_risk", ego)
+            # decoupled decoder on mean(fused): heading phi and speed v = min(softplus, 25) -> cumulative waypoints
+            fmean = self.pool2(fused, "fused_mean_w", "fused_mean_h").flatten(1)
+            dh = taps.tap("dec_head", self.gemm(fmean, "dec_head")).reshape(1, C.EGO_MODES, 12)
+            phi, vr = dh[..., :6], dh[..., 6:]
+            sp = torch.clamp(vr, min=0.0) + torch.log(1.0 + torch.exp(-vr.abs()))
+            step = torch.clamp(sp, max=C.DEC_SPEED_MAX) * 0.5
+            wp = taps.tap("dec_wp", torch.stack([torch.cumsum(step * torch.cos(phi), dim=2),
+                                                 torch.cumsum(step * torch.sin(phi), dim=2)], dim=-1))
+            e36 = ego[:, :36].reshape(1, C.EGO_MODES, C.EGO_STEPS, 2)
+            e36 = e36 + torch.tanh(self._dec_gate) * (wp - e36)
+            ego = taps.tap("after_dec", torch.cat([e36.reshape(1, 36), ego[:, 36:]], dim=1))
+        return ego, fmean
+
+    @staticmethod
+    def _range_channel(h: int, w: int):
+        """The refiners' constant range channel: row i -> (80 - (i + 0.5) * (160 / h)) / 80, float32 in graph order
+        (Range + 0.5, Reciprocal(h) * 160, Mul, 80 - ., / 80)."""
+        torch = _torch()
+        step = (1.0 / torch.tensor(float(h), dtype=torch.float32)) * 160.0
+        rows = (80.0 - (torch.arange(h, dtype=torch.float32) + 0.5) * step) / 80.0
+        return rows.reshape(1, 1, h, 1).expand(1, 1, h, w)
+
+    def seg_refiner(self, lane_pre, taps: TapRegistry = NULL_TAPS):
+        """BEVSegRefiner: lane_logit = clamp(lane_pre, +-20) + 8 tanh(res / 8) (S:316)."""
+        torch = _torch()
+        lc = torch.clamp(lane_pre, -C.SEG_REFINE_CLAMP, C.SEG_REFINE_CLAMP)
+        x = torch.cat([lc, self._range_channel(lc.shape[-2], lc.shape[-1])], dim=1)
+        p = "refiner/seg"
+        stem = self.conv(x, f"{p}/stem/stem.0", True)
+        d1 = self.down(stem, f"{p}/d1")
+        d2 = self.down(d1, f"{p}/d2")
+        d3 = self.down(d2, f"{p}/d3")
+        d4 = self.down(d3, f"{p}/d4")
+        m3 = self.block(d3 + self.resize(self.conv(d4, f"{p}/u4"), d3.shape[-2:]), f"{p}/m3")
+        m2 = self.block(d2 + self.resize(self.conv(m3, f"{p}/u3"), d2.shape[-2:]), f"{p}/m2")
+        m1 = self.block(d1 + self.resize(self.conv(m2, f"{p}/u2"), d1.shape[-2:]), f"{p}/m1")
+        o = stem + self.resize(self.conv(m1, f"{p}/u1"), stem.shape[-2:])
+        res = taps.tap("refiner.seg_res", self.conv(self.conv(o, f"{p}/out/out.0", True), f"{p}/out/out.3"))
+        return lc + torch.tanh(res / C.SEG_REFINE_SCALE) * C.SEG_REFINE_SCALE
+
+    def box_refiner(self, hm_pre, reg_pre, taps: TapRegistry = NULL_TAPS):
+        """BEVBoxRefiner: hm, reg = clamp(.) + 6 tanh(res / 6) (S:317)."""
+        torch = _torch()
+        hc = torch.clamp(hm_pre, -C.BOX_HM_CLAMP, C.BOX_HM_CLAMP)
+        rc = torch.clamp(reg_pre, -C.BOX_REG_CLAMP, C.BOX_REG_CLAMP)
+        x = torch.cat([hc, rc, self._range_channel(hc.shape[-2], hc.shape[-1])], dim=1)
+        p = "refiner/box"
+        stem = self.conv(x, f"{p}/stem/stem.0", True)
+        d1 = self.down(stem, f"{p}/d1")
+        d2 = self.down(d1, f"{p}/d2")
+        d3 = self.down(d2, f"{p}/d3")
+        m2 = self.block(d2 + self.resize(self.conv(d3, f"{p}/u3"), d2.shape[-2:]), f"{p}/m2")
+        m1 = self.block(d1 + self.resize(self.conv(m2, f"{p}/u2"), d1.shape[-2:]), f"{p}/m1")
+        o = stem + self.resize(self.conv(m1, f"{p}/u1"), stem.shape[-2:])
+        res = taps.tap("refiner.box_res", self.conv(self.conv(o, f"{p}/out/out.0", True), f"{p}/out/out.3"))
+        r = torch.tanh(res / C.BOX_REFINE_SCALE) * C.BOX_REFINE_SCALE
+        return hc + r[:, :2], rc + r[:, 2:]
+
+    def e2e_refiner(self, ego, v0, fused_mean, taps: TapRegistry = NULL_TAPS):
+        """E2ERefiner: ego[:36] += 3 tanh(mlp(LN([ego, v0, mean(fused)]))) (S:318; LN at the logical width 139)."""
+        torch = _torch()
+        F = torch.nn.functional
+        g, b, eps = self._ln
+        x = torch.cat([ego, v0.reshape(1, 1).to(torch.float32), fused_mean], dim=1)
+        h = taps.tap("refiner.e2e_ln", F.layer_norm(x, (x.shape[-1],), g, b, eps))
+        h = F.relu(self.gemm(h, "refiner/e2e/mlp/mlp.0"))
+        h = F.relu(self.gemm(h, "refiner/e2e/mlp/mlp.2"))
+        r = taps.tap("refiner.e2e_res", torch.tanh(self.gemm(h, "refiner/e2e/mlp/mlp.4")) * C.E2E_SCALE)
+        return torch.cat([ego[:, :36] + r, ego[:, 36:]], dim=1)
+
+    def depth_mean(self, depth_logits):
+        """avgpool2 of the 64 logits -> softmax -> argmax -> +-2-bin window -> sum p c / max(sum p, 1e-6) -> fp16
+        (ONNX nodes 1946-2014; bin centres per ``cfg.depth_mean_bins``)."""
+        torch = _torch()
+        F = torch.nn.functional
+        lp = F.avg_pool2d(depth_logits, kernel_size=2, stride=2)
+        p = torch.softmax(lp.reshape(1, C.N_CAMS, C.DEPTH_BINS, lp.shape[-2], lp.shape[-1]), dim=2)
+        am = torch.argmax(p, dim=2, keepdim=True)
+        r = torch.arange(C.DEPTH_BINS, dtype=torch.int64).reshape(1, 1, C.DEPTH_BINS, 1, 1)
+        pm = p * ((r - am).abs() <= 2).to(torch.float32)
+        num = (pm * self._depth_centres).sum(dim=2)
+        den = torch.clamp(pm.sum(dim=2), min=1e-6)
+        return (num / den).to(torch.float16)
+
+    # ---- whole graph ------------------------------------------------------------------------------------------------
+    def forward(self, imgs, K, T_cam_ego, v0, *, present=None, taps: TapRegistry = NULL_TAPS) -> Dict[str, Any]:
+        """The 19 outputs of the graph (torch tensors, ONNX dtypes) for one frame.
+
+        ``imgs`` uint8 [1, 8, 3, 432, 768] RGB, ``K`` [1, 8, 3, 3] at 768x432, ``T_cam_ego`` [1, 8, 4, 4], ``v0`` [1];
+        numpy or torch. ``present`` (8 flags) only matters for ``input_norm="imagenet"``."""
+        torch = _torch()
+        t = lambda a, dt: a.to(dt) if isinstance(a, torch.Tensor) else torch.from_numpy(np.ascontiguousarray(a)).to(dt)
+        imgs = t(imgs, torch.uint8)
+        K = t(K, torch.float32)
+        T = t(T_cam_ego, torch.float32)
+        v0 = t(v0, torch.float32).reshape(1)
+        out: Dict[str, Any] = {}
+        with torch.inference_mode():
+            x = taps.tap("img.input", self.normalize(imgs, present))
+            f = self.image_encoder(x, taps)
+            del x
+            seg = self.seg2d_head(f, taps)
+            seg_c = taps.tap("seg2d.clamped", torch.clamp(self.nan_to_num(seg), -C.SEG2D_CLAMP, C.SEG2D_CLAMP))
+            dlog, dprob = self.depth_head(f, taps)
+            ctx = self.context(f, seg, taps)
+            out.update(self.det2d_heads(f))
+            out["tl"] = self.tl_head(f)
+            del f
+            raw = self.lift(ctx, dprob, K, T, taps)
+            del ctx, dprob
+            fused = self.temporal_fuse(raw, taps)
+            lane_pre = self.lane_decoder(raw, taps)
+            det_feat, hm_pre, reg_pre = self.det_stem(raw, taps)
+            out["occ"] = self.occupancy(raw, taps)
+            out["traj"] = self.agent_forecast(raw, fused, det_feat, reg_pre, taps)
+            out["stationary"] = self.stationary(raw, taps)
+            del raw, det_feat
+            out["risk"] = self.risk(fused, taps)
+            ego, fmean = self.planner(fused, lane_pre, hm_pre, out["risk"], v0, taps)
+            taps.tap("bev.fused_mean", fmean)
+            out["lane_logit"] = self.seg_refiner(lane_pre, taps)
+            out["hm"], out["reg"] = self.box_refiner(hm_pre, reg_pre, taps)
+            e2e_mean = self.pool2(fused, "e2e_mean_w", "e2e_mean_h").flatten(1)
+            out["ego"] = self.e2e_refiner(ego, v0, e2e_mean, taps)
+            del fused
+            out["lane"] = torch.argmax(out["lane_logit"], dim=1).to(torch.uint8)
+            dl5 = dlog.reshape(1, C.N_CAMS, C.DEPTH_BINS, C.FEAT_H, C.FEAT_W)
+            out["depth"] = torch.argmax(dl5, dim=2).to(torch.uint8)
+            out["seg2d"] = torch.argmax(seg_c.reshape(1, C.N_CAMS, len(C.SEG2D_CLASSES), C.FEAT_H, C.FEAT_W),
+                                        dim=2).to(torch.uint8)
+            out["depth_mean"] = self.depth_mean(dlog)
+            for name in OUTPUT_NAMES:
+                taps.tap(name, out[name])
+        return {name: out[name] for name in OUTPUT_NAMES}
+
+    __call__ = forward

code/tt_meteor/reference/ort.py

@@ -0,0 +1,139 @@
+# SPDX-License-Identifier: Apache-2.0
+"""ONNX Runtime oracle of the deployed graph: the 19 outputs plus any internal tensor as a named tap.
+
+Only for tests and golden generation in the research venv (onnxruntime is not a runtime dependency of the bundle,
+BUNDLE_CONVENTIONS.md section 13). The graph is modified in memory only (extra graph outputs; for
+``input_norm="imagenet"`` the training normalisation inserted after ``/Div``, the oracle of SPEC section 10 risk 1,
+re-implemented from ``research/meteor/scripts/norm_ab_test.py``); the downloaded file is never written.
+ORT on this graph is bit-reproducible at a fixed thread count (SPEC section 6).
+"""
+from __future__ import annotations
+
+import os
+from typing import Any, Dict, Iterable, Mapping, Optional, Sequence
+
+import numpy as np
+
+from . import config as C
+
+__all__ = ["ORT_TAPS", "OrtOracle", "available"]
+
+# reference tap name -> ONNX tensor name (shapes as in the graph; see reference/model.py)
+ORT_TAPS: Dict[str, str] = {
+    "img.input": "/net/Reshape_output_0",
+    "img.stem": "/net/stem/stem.3/MaxPool_output_0",
+    "img.layer1": "/net/layer1/layer1.2/relu_1/Relu_output_0",
+    "img.layer2": "/net/layer2/layer2.3/relu_1/Relu_output_0",
+    "img.layer3": "/net/layer3/layer3.5/relu_1/Relu_output_0",
+    "img.layer4": "/net/layer4/layer4.2/relu_1/Relu_output_0",
+    "img.fpn": "/net/fuse/fuse.2/Relu_output_0",
+    "seg2d.logits": "/net/seg_head/out/out.3/Conv_output_0",
+    "seg2d.clamped": "/net/Clip_output_0",
+    "depth.logits": "/net/depth_head/depth_head.4/Conv_output_0",
+    "depth.prob": "/net/Softmax_output_0",
+    "ctx.painted": "/net/Add_3_output_0",
+    "lift.grid": "/net/Unsqueeze_6_output_0",
+    "lift.valid": "/net/And_8_output_0",
+    "lift.weight": "/net/Mul_10_output_0",
+    "lift.bev": "/net/Reshape_5_output_0",
+    "bev.raw": "/net/Resize_3_output_0",
+    "bev.gate": "/net/Mul_12_output_0",
+    "bev.fused": "/net/Add_11_output_0",
+    "lane.pre": "/net/Concat_7_output_0",
+    "det.feat": "/net/det_stem/m/m.2/Relu_output_0",
+    "det.hm_pre": "/net/hm_head/Conv_output_0",
+    "det.reg_pre": "/net/reg_head/Conv_output_0",
+    "occ.feat": "/net/occ_stem/occ_stem.3/occ_stem.3.5/Relu_output_0",
+    "traj.stem": "/net/traj_stem/traj_stem.3/traj_stem.3.5/Relu_output_0",
+    "traj.agent_delta": "/net/agent_delta/Conv_output_0",
+    "traj.feat": "/net/Add_15_output_0",
+    "stat.feat": "/net/delta_stat/delta_stat.6/Relu_output_0",
+    "risk.feat": "/net/risk_head/risk_head.1/risk_head.1.5/Relu_output_0",
+    "ego.stem": "/net/Flatten_output_0",
+    "ego.mlp": "/net/ego_mlp/ego_mlp.6/Gemm_output_0",
+    "ego.attn": "/net/ego_delta/Gemm_output_0",
+    "ego.plus_attn": "/net/Add_13_output_0",
+    "ego.sem_in": "/net/Concat_23_output_0",
+    "ego.sem": "/net/sem_ego/sem_ego.13/Gemm_output_0",
+    "ego.plus_sem": "/net/Add_14_output_0",
+    "ego.kin": "/net/kin_delta/Gemm_output_0",
+    "ego.plus_kin": "/net/Add_17_output_0",
+    "ego.risk_sample": "/net/Squeeze_2_output_0",
+    "ego.after_risk": "/net/ScatterND_1_output_0",
+    "ego.dec_head": "/net/dec_head/Gemm_output_0",
+    "ego.dec_wp": "/net/Concat_36_output_0",
+    "ego.after_dec": "/net/ScatterND_2_output_0",
+    "bev.fused_mean": "/net/Flatten_3_output_0",
+    "refiner.seg_res": "/net/refiner/seg/out/out.3/Conv_output_0",
+    "refiner.box_res": "/net/refiner/box/out/out.3/Conv_output_0",
+    "refiner.e2e_ln": "/net/refiner/e2e/norm/LayerNormalization_output_0",
+    "refiner.e2e_res": "/net/refiner/e2e/Mul_output_0",
+}
+
+
+def available() -> bool:
+    try:
+        import onnxruntime  # noqa: F401  (probe only)
+    except ImportError:
+        return False
+    return True
+
+
+def _insert_imagenet(model) -> None:
+    """(x / 255 - mean) / std right after ``/Div`` (in memory; SPEC section 10 risk 1)."""
+    from onnx import helper, numpy_helper
+
+    g = model.graph
+    div = next(n for n in g.node if n.name == "/Div")
+    out = div.output[0]
+    mean = np.array(C.IMAGENET_MEAN, np.float32).reshape(1, 1, 3, 1, 1)
+    std = np.array(C.IMAGENET_STD, np.float32).reshape(1, 1, 3, 1, 1)
+    g.initializer.extend([numpy_helper.from_array(mean, "tt_meteor_mean"),
+                          numpy_helper.from_array(std, "tt_meteor_std")])
+    for n in g.node:
+        if n is not div:
+            for i, x in enumerate(n.input):
+                if x == out:
+                    n.input[i] = "tt_meteor_norm"
+    idx = list(g.node).index(div)
+    g.node.insert(idx + 1, helper.make_node("Sub", [out, "tt_meteor_mean"], ["tt_meteor_sub"], name="/tt_meteor/Sub"))
+    g.node.insert(idx + 2, helper.make_node("Div", ["tt_meteor_sub", "tt_meteor_std"], ["tt_meteor_norm"],
+                                            name="/tt_meteor/Div"))
+
+
+class OrtOracle:
+    """``OrtOracle(onnx_path, taps=[...], threads=4)``; ``oracle(feed) -> {name: array}`` with the 19 outputs under
+    their ONNX names and the requested taps under their reference names (``ORT_TAPS``; raw ONNX tensor names are
+    accepted too)."""
+
+    def __init__(self, onnx_path: os.PathLike, taps: Iterable[str] = (), *, threads: int = 4,
+                 input_norm: str = "onnx"):
+        import onnx
+        import onnxruntime as ort
+
+        self.taps = {t: ORT_TAPS.get(t, t) for t in taps}
+        model = onnx.load(str(onnx_path))
+        if input_norm == "imagenet":
+            _insert_imagenet(model)
+        elif input_norm != "onnx":
+            raise ValueError(f"input_norm {input_norm!r}")
+        have = {o.name for o in model.graph.output}
+        for name in self.taps.values():
+            if name not in have:
+                model.graph.output.append(onnx.helper.make_empty_tensor_value_info(name))
+        so = ort.SessionOptions()
+        so.intra_op_num_threads = int(threads)
+        so.inter_op_num_threads = 1
+        self.session = ort.InferenceSession(model.SerializeToString(), sess_options=so,
+                                            providers=["CPUExecutionProvider"])
+        del model
+        self.output_names = [o.name for o in self.session.get_outputs()]
+        self.version = ort.__version__
+        self.threads = int(threads)
+
+    def __call__(self, feed: Mapping[str, np.ndarray], *, outputs: Optional[Sequence[str]] = None) -> Dict[str, Any]:
+        inv = {v: k for k, v in self.taps.items()}
+        feed = {k: np.asarray(feed[k]) for k in ("imgs", "K", "T_cam_ego", "v0")}
+        res = self.session.run(outputs, feed)
+        names = list(outputs) if outputs is not None else self.output_names
+        return {inv.get(n, n): v for n, v in zip(names, res)}

code/tt_meteor/reference/pipeline.py

@@ -0,0 +1,86 @@
+# SPDX-License-Identifier: Apache-2.0
+"""End-to-end fp32 CPU reference of METEOR's runtime on the released graph: the oracle of every TT gate.
+
+Data flow per frame (SPEC sections 2-5):
+
+    eight cameras + calibration + ego speed -> host.inputs / host.preprocess -> MeteorFrame (the ONNX feed)
+      -> reference.model.MeteorNet (torch fp32, per-module taps) -> the 19 outputs
+      -> host.result.build_output (3D / 2D decode, NMS, stationary, futures, unk2d, plan; per-stream seg fusion,
+         yaw smoothing, mode hysteresis) -> host.outputs.MeteorOutput (the /predict JSON)
+
+:meth:`MeteorReference.run_frame` is the network alone (a ``MeteorFrame`` -> numpy outputs, optional taps: what the
+goldens hold). :meth:`MeteorReference.__call__` takes the same inputs as ``api.METEOR.__call__`` (``images``,
+``calibration``, ``ego_speed``, ``stream``, runtime params) and keeps the host state per stream id, so the served
+model and this reference can be compared request by request.
+
+torch is imported when the reference is built. ``threads`` caps torch's intra-op threads (the workspace rule: <= 4).
+"""
+from __future__ import annotations
+
+import time
+from typing import Any, Dict, Mapping, Optional
+
+import numpy as np
+
+from ..host.inputs import prepare_request
+from ..host.outputs import MeteorOutput
+from ..host.postprocess import PostConfig
+from ..host.preprocess import MeteorFrame
+from ..host.result import build_output
+from ..host.temporal import StreamState
+from ..ttaw.golden import NULL_TAPS, TapRegistry
+from .config import MeteorConfig, default_config
+from .model import OUTPUT_NAMES, MeteorNet
+from .weights import MeteorWeights
+
+__all__ = ["MeteorReference", "to_numpy_outputs"]
+
+
+def to_numpy_outputs(out: Mapping[str, Any]) -> Dict[str, np.ndarray]:
+    return {k: (v.numpy() if hasattr(v, "numpy") else np.asarray(v)) for k, v in out.items()}
+
+
+class MeteorReference:
+    """``MeteorReference(weights_dir=None, cfg=None, threads=4)``: weights (``reference.weights``), the torch network
+    and per-stream host state."""
+
+    def __init__(self, weights_dir: Optional[Any] = None, cfg: Optional[MeteorConfig] = None, *,
+                 threads: Optional[int] = 4, post: PostConfig = PostConfig(), weights: Optional[MeteorWeights] = None):
+        import torch
+
+        if threads:
+            torch.set_num_threads(int(threads))
+        self.cfg = (cfg or default_config()).check()
+        self.weights = weights or MeteorWeights(weights_dir)
+        self.net = MeteorNet(self.weights, self.cfg)
+        self.post = post
+        self.streams: Dict[str, StreamState] = {}
+
+    def run_frame(self, frame: MeteorFrame, taps: TapRegistry = NULL_TAPS) -> Dict[str, np.ndarray]:
+        """The 19 outputs (numpy, ONNX names / dtypes / shapes) of one frame."""
+        out = self.net(frame.imgs, frame.K, frame.T_cam_ego, frame.v0, present=frame.present, taps=taps)
+        return to_numpy_outputs({k: out[k] for k in OUTPUT_NAMES})
+
+    def stream(self, stream: Optional[Mapping[str, Any]]) -> StreamState:
+        sid = str((stream or {}).get("id", "default"))
+        st = self.streams.get(sid)
+        if st is None:
+            st = self.streams[sid] = StreamState()
+        elif (stream or {}).get("reset"):
+            st.reset()
+        return st
+
+    def __call__(self, images: Any = None, *, calibration: Optional[Mapping[str, Any]] = None, ego_speed: Any = None,
+                 stream: Optional[Mapping[str, Any]] = None, output_heads: bool = False, **params: Any) -> MeteorOutput:
+        t0 = time.perf_counter()
+        frame = prepare_request(images, calibration, ego_speed, stream)
+        t1 = time.perf_counter()
+        outputs = self.run_frame(frame)
+        t2 = time.perf_counter()
+        cfg = self.post.with_params(**params)
+        res = build_output(outputs, frame, cfg, self.stream(stream), heads=output_heads)
+        t3 = time.perf_counter()
+        res.timing_ms.update({"preprocess": 1e3 * (t1 - t0), "device": 1e3 * (t2 - t1), "postprocess": 1e3 * (t3 - t2),
+                              "total": 1e3 * (t3 - t0)})
+        res.meta["backend"] = "cpu-reference"
+        return res

code/tt_meteor/reference/port_form.py

@@ -0,0 +1,280 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The exact rewrites the TT port applies to the deployed graph (PLAN.md 2.13; SPEC sections 3 and 8.4), with torch
+emulations proven against the literal graph (``tests/test_reference_cpu.py::test_port_form_rewrites``, teacher-forced
+from the ONNX Runtime taps). Each is exact in real arithmetic; in float32 they differ from the graph only by
+rounding (measured max |difference| is recorded in PORT_LOG.md section 3).
+
+========================  ===========================================================================================
+rewrite                   what changes
+========================  ===========================================================================================
+``stem_input_fold``       the input normalisation folds into the stem conv: ``onnx``: W / 255 on raw uint8 values
+                          (zero padding is preserved under scaling); ``imagenet``: W / (255 std_c) on
+                          ``x_u8 - 255 mean_c`` (the mean stays explicit: zero padding of the normalised image is
+                          ``x_u8 = 255 mean_c``, so it cannot become a bias; SPEC section 3 item 3)
+``tgate_tfuse_merged``    tgate (96 -> 4) and tfuse.0 (96 -> 96) as one 96 -> 100 1x1 conv: W (s x) = s (W x) for
+                          the per-pixel gate s = 4 softmax(tgate)[0], so the gate scales the conv output before its
+                          bias and ReLU (SPEC 8.4)
+``merged_heads``          hm_head + reg_head = one 1x1 128 -> 8; hm2d + reg2d per stride = one 1x1 -> 14
+``paint_conv21``          the paint gather (seg classes 2..8, 13) + paint_proj 8 -> 96 = one 1x1 21 -> 96 conv on
+                          softmax(raw seg logits) with zero columns for the unpainted classes
+``traj_stem_live``        traj_stem.0 on cat(fused, mot = 0) = the conv of its first 96 input channels on fused
+``lift_from_tables``      the lift with the calibration-only geometry from ``host.calib`` (grid, validity and the
+                          dense bin-interpolation table): w = valid (0.05 + sum_b prob_s[b] table[b]); the device's
+                          functional fallback (per-camera ``grid_sample`` + a bin contraction + the camera sum)
+``lane_pre_merged``       dec.out.3 (64 -> 9) and lane_branch.6 (64 -> 3, added to channels 4..6) as one 1x1 conv over
+                          cat(dec.out.0 output, lane_branch.3 output) (a K-split conv, rows 4..6 of the second half)
+``traj_head_fold``        traj_head on cat(traj.feat, reg_pre[4:6]) with reg_pre = reg_head(det.feat): the reg path
+                          folds into a 1x1 conv on det.feat (W_t[:, 256:] W_reg[4:6]; bias W_t[:, 256:] b_reg[4:6])
+``mha_fold``              the attention pools with learned (constant) queries: q is a constant, so the key projection
+                          folds into the scores (S = G^T keys^T + c), the value projection, the out-projection and the
+                          consumer (ego_delta Gemm / mean over queries + agent_delta conv) fold into one matrix on the
+                          attention-weighted keys Z = softmax(S) keys (sum_k softmax = 1 moves every bias out)
+``range_bias_map``        the refiners' constant range channel through their stem conv (+ the conv bias) as a constant
+                          per-pixel bias map: stem(cat(x, range)) = conv(x; W[:, :-1]) + B (zero padding included)
+``sem_fc_nhwc``           sem_ego.11 with its 1280 inputs re-ordered from (c, i, j) to (i, j, c) (the device's NHWC
+                          flatten of the 5 x 4 pooled map)
+========================  ===========================================================================================
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Dict, Tuple
+
+import numpy as np
+
+from . import config as C
+from .weights import MeteorWeights
+
+__all__ = ["stem_input_fold", "tgate_tfuse_merged", "merged_heads", "paint_conv21", "traj_stem_live",
+           "fused_from_merged", "lift_from_tables", "lane_pre_merged", "traj_head_fold", "MhaFold", "mha_fold",
+           "mha_from_fold", "range_channel", "range_bias_map", "sem_fc_nhwc", "REFINER_STEMS"]
+
+F32 = np.float32
+
+
+def stem_input_fold(w: MeteorWeights, input_norm: str = "onnx") -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    """(W', b, offset): ``conv(normalised x, W) == conv(x_u8 - offset_c, W')`` with W' = W / (255 std_c), offset =
+    255 mean_c (``imagenet``) or W / 255, offset 0 (``onnx``). Folded in float64, rounded once."""
+    p = w.conv("stem/stem.0")
+    wt = np.asarray(p.weight, np.float64)
+    if input_norm == "onnx":
+        scale = np.full(3, 255.0)
+        offset = np.zeros(3)
+    elif input_norm == "imagenet":
+        scale = 255.0 * np.asarray(C.IMAGENET_STD, np.float64)
+        offset = 255.0 * np.asarray(C.IMAGENET_MEAN, np.float64)
+    else:
+        raise ValueError(input_norm)
+    return (wt / scale.reshape(1, 3, 1, 1)).astype(F32), np.asarray(p.bias, F32), offset.astype(F32)
+
+
+def tgate_tfuse_merged(w: MeteorWeights) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    """(W [100, 96, 1, 1], b_gate [4], b_tfuse [96]): rows 0..3 = tgate, rows 4..99 = tfuse3_slim.0 (no bias: it is
+    added after the gate scaling)."""
+    g, t = w.conv("tgate_slim"), w.conv("tfuse3_slim/tfuse3_slim.0")
+    return (np.concatenate([g.weight, t.weight], axis=0).astype(F32), np.asarray(g.bias, F32),
+            np.asarray(t.bias, F32))
+
+
+def fused_from_merged(conv_out, b_gate, b_tfuse):
+    """The merged conv's output [1, 100, H, W] -> (gate [1, 1, H, W], relu(gate * W_t x + b_t) [1, 96, H, W])."""
+    import torch
+
+    gate = torch.softmax(conv_out[:, :4] + torch.as_tensor(b_gate).reshape(1, 4, 1, 1), dim=1)[:, 0:1] * C.TGATE_SCALE
+    t0 = torch.relu(gate * conv_out[:, 4:] + torch.as_tensor(b_tfuse).reshape(1, -1, 1, 1))
+    return gate, t0
+
+
+def merged_heads(w: MeteorWeights) -> Dict[str, Tuple[np.ndarray, np.ndarray]]:
+    """``det``: hm (2) | reg (6) on det_feat; ``det2d_s0/s1/s2``: hm2d (10) | reg2d (4) per stride."""
+    out = {}
+    for name, parts in (("det", ("hm_head", "reg_head")), ("det2d_s0", ("hm2d_head", "reg2d_head")),
+                        ("det2d_s1", ("hm2d_head8", "reg2d_head8")), ("det2d_s2", ("hm2d_head16", "reg2d_head16"))):
+        ps = [w.conv(m) for m in parts]
+        out[name] = (np.concatenate([p.weight for p in ps], 0).astype(F32),
+                     np.concatenate([p.bias for p in ps], 0).astype(F32))
+    return out
+
+
+def paint_conv21(w: MeteorWeights) -> Tuple[np.ndarray, np.ndarray]:
+    """paint_proj [96, 8, 1, 1] scattered into [96, 21, 1, 1] at the painted seg classes (zeros elsewhere)."""
+    p = w.conv("paint_proj")
+    full = np.zeros((p.weight.shape[0], len(C.SEG2D_CLASSES), 1, 1), F32)
+    full[:, list(w.paint_classes())] = p.weight
+    return full, np.asarray(p.bias, F32)
+
+
+def traj_stem_live(w: MeteorWeights) -> Tuple[np.ndarray, np.ndarray]:
+    """traj_stem.0 restricted to the fused-BEV input channels (the 96 ``mot`` channels are the baked-out zeros)."""
+    p = w.conv("traj_stem/traj_stem.0")
+    return np.ascontiguousarray(p.weight[:, :C.BEV_C]).astype(F32), np.asarray(p.bias, F32)
+
+
+def lift_from_tables(ctx, prob, geom) -> "object":
+    """The lift with host tables (``host.calib.LiftGeometry``): ctx [8, 96, 108, 192], prob [8, 64, 108, 192] ->
+    lift.bev [1, 96, 400, 250]. Per camera: ``grid_sample`` of ctx and prob at the table grid (bilinear, zeros),
+    w = valid (0.05 + sum_b prob_s table), camera sums, the clamped division."""
+    import torch
+    F = torch.nn.functional
+
+    grid = torch.from_numpy(np.ascontiguousarray(geom.grid)).unsqueeze(2)              # [8, N, 1, 2]
+    ctx_s = F.grid_sample(ctx, grid, mode="bilinear", padding_mode="zeros", align_corners=False).squeeze(3)
+    prob_s = F.grid_sample(prob, grid, mode="bilinear", padding_mode="zeros", align_corners=False).squeeze(3)
+    table = torch.from_numpy(geom.bin_weights()).permute(0, 2, 1)                    # [8, 64, N]
+    valid = torch.from_numpy(geom.valid.astype(F32)).unsqueeze(1)                      # [8, 1, N]
+    w_ = valid * ((prob_s * table).sum(dim=1, keepdim=True) + C.LIFT_W_EPS)           # [8, 1, N]
+    num = (ctx_s * w_).sum(dim=0, keepdim=True)                                        # [1, 96, N]
+    den = torch.clamp(w_.sum(dim=0, keepdim=True), min=C.LIFT_DEN_EPS)
+    return (num / den).reshape(1, C.CTX_C, C.LIFT_H, C.LIFT_W)
+
+
+# ---- rewrites of the device graph (M2+) -----------------------------------------------------------------------------
+
+def lane_pre_merged(w: MeteorWeights) -> Tuple[np.ndarray, np.ndarray]:
+    """(W [9, 128, 1, 1], b [9]): ``lane_pre = conv1x1(cat(x_dec, x_lb); W) + b`` with x_dec the dec.out.0 output and
+    x_lb the lane_branch.3 output (64 channels each); rows 4..6 of the second half carry lane_branch.6, the bias is
+    dec.out.3's plus lane_branch.6's on channels 4..6 (``model.lane_decoder``: out[:, 4:7] + lb)."""
+    o, lb = w.conv("dec/out/out.3"), w.conv("lane_branch/lane_branch.6")
+    c, k = o.weight.shape[0], o.weight.shape[1]
+    wt = np.zeros((c, k + lb.weight.shape[1], 1, 1), F32)
+    wt[:, :k] = o.weight
+    wt[4:7, k:] = lb.weight
+    b = np.asarray(o.bias, F32).copy()
+    b[4:7] = b[4:7] + np.asarray(lb.bias, F32)
+    return wt, b
+
+
+def traj_head_fold(w: MeteorWeights) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    """(W_feat [39, 256, 1, 1], W_det [39, 128, 1, 1], b [39]): ``traj = conv(feat; W_feat) + conv(det_feat; W_det) +
+    b`` equals traj_head(cat(feat, reg_pre[:, 4:6])) with reg_pre = reg_head(det_feat) (both 1x1: the reg path is
+    linear in det_feat). Folded in float64, rounded once."""
+    t, r = w.conv("traj_head"), w.conv("reg_head")
+    wt = np.asarray(t.weight, np.float64)[:, :, 0, 0]                    # [39, 258]
+    nf = wt.shape[1] - 2
+    w2 = wt[:, nf:]                                                       # [39, 2]
+    wr = np.asarray(r.weight, np.float64)[4:6, :, 0, 0]                   # [2, 128]
+    w_det = w2 @ wr
+    b = np.asarray(t.bias, np.float64) + w2 @ np.asarray(r.bias, np.float64)[4:6]
+    return (wt[:, :nf, None, None].astype(F32), w_det[:, :, None, None].astype(F32), b.astype(F32))
+
+
+@dataclass(frozen=True)
+class MhaFold:
+    """An attention pool with constant queries in folded form: for keys K [L, E] (one token per row),
+    ``S = G^T K^T + c`` [R, L] with R = queries x heads (row r = query i * heads + head h), ``att = softmax(S)`` over
+    the keys, ``Z = att K`` [R, E], and the consumer output ``vec(Z) V + d0`` ([R * E] x [R * E, out])."""
+
+    G: np.ndarray            # [E, R] float32
+    c: np.ndarray            # [R] float32
+    V: np.ndarray            # [R * E, out] float32
+    d0: np.ndarray           # [out] float32
+    queries: int
+    heads: int
+
+    @property
+    def rows(self) -> int:
+        return self.queries * self.heads
+
+    @property
+    def embed(self) -> int:
+        return int(self.G.shape[0])
+
+
+def mha_fold(w: MeteorWeights, name: str) -> MhaFold:
+    """``ego_attn`` (3 queries, consumer ``ego_delta`` Gemm 288 -> 42 on the flattened [3, 96] output) or
+    ``agent_attn`` (4 queries, mean over the queries, consumer ``agent_delta`` 1x1 conv 128 -> 256): see
+    :class:`MhaFold`. ``model.mha_pool``: q = b_q + queries W_q^T, k|v = b_kv + keys W_kv^T, per head
+    softmax((q / div) k^T) v, out = b_o + o W_o^T. With q constant, the scores are keys . (W_k,h^T q_i,h / div) +
+    q_i,h . b_k,h / div; since the attention weights sum to 1, o_i,h = W_v,h Z_i,h + b_v,h and the consumer is linear,
+    so everything after the softmax is one matrix on Z. Float64, rounded once."""
+    m = w.mha(name)
+    qn = {"ego_attn": "ego", "agent_attn": "agent"}[name]
+    queries = np.asarray(w.queries(qn), np.float64)[0]                    # [Q, E]
+    e, h = m.embed_dim, m.num_heads
+    hd = e // h
+    nq = queries.shape[0]
+    win = np.asarray(m.in_weight, np.float64)
+    bin_ = np.asarray(m.in_bias, np.float64)
+    q = queries @ win[:e].T + bin_[:e]                                    # [Q, E]
+    wk, bk, wv, bv = win[e:2 * e], bin_[e:2 * e], win[2 * e:], bin_[2 * e:]
+    wo, bo = np.asarray(m.out_weight, np.float64), np.asarray(m.out_bias, np.float64)
+    div = float(np.float32(m.scale_divisor))
+    rows = nq * h
+    G = np.zeros((e, rows))
+    c = np.zeros(rows)
+    U = []
+    for hh in range(h):
+        sl = slice(hh * hd, (hh + 1) * hd)
+        U.append(wo[:, sl] @ wv[sl])                                      # [E, E]: out-proj of head hh's values
+    const_o = bo + wo @ bv                                                # attention output bias of every query
+    for i in range(nq):
+        for hh in range(h):
+            sl = slice(hh * hd, (hh + 1) * hd)
+            qs = q[i, sl] / div
+            G[:, i * h + hh] = wk[sl].T @ qs
+            c[i * h + hh] = qs @ bk[sl]
+    if name == "ego_attn":
+        g = w.gemm("ego_delta")
+        wd, bd = np.asarray(g.weight, np.float64), np.asarray(g.bias, np.float64)      # [42, Q * E]
+        mats = [[wd[:, i * e:(i + 1) * e] @ U[hh] for hh in range(h)] for i in range(nq)]
+        d0 = bd + sum(wd[:, i * e:(i + 1) * e] @ const_o for i in range(nq))
+    else:
+        p = w.conv("agent_delta")
+        wa = np.asarray(p.weight, np.float64)[:, :, 0, 0]                 # [256, 128]
+        mats = [[wa @ U[hh] / nq for hh in range(h)] for _ in range(nq)]
+        d0 = np.asarray(p.bias, np.float64) + wa @ const_o
+    out = mats[0][0].shape[0]
+    V = np.zeros((rows * e, out))
+    for i in range(nq):
+        for hh in range(h):
+            r = i * h + hh
+            V[r * e:(r + 1) * e] = mats[i][hh].T
+    return MhaFold(G.astype(F32), c.astype(F32), V.astype(F32), d0.astype(F32), nq, h)
+
+
+def mha_from_fold(fold: MhaFold, keys):
+    """torch emulation of the folded attention pool: keys [L, E] -> the consumer output [out] (CPU proofs)."""
+    import torch
+
+    k = torch.as_tensor(keys, dtype=torch.float32)
+    s = torch.as_tensor(fold.G).T @ k.T + torch.as_tensor(fold.c)[:, None]          # [R, L]
+    z = torch.softmax(s, dim=-1) @ k                                                # [R, E]
+    return z.reshape(-1) @ torch.as_tensor(fold.V) + torch.as_tensor(fold.d0)
+
+
+# the refiners' stems: module -> (input channels without the range channel, output channels)
+REFINER_STEMS = {"refiner/seg/stem/stem.0": (9, 48), "refiner/box/stem/stem.0": (8, 32)}
+
+
+def range_channel(h: int, w_: int) -> np.ndarray:
+    """The refiners' constant range channel [h, w] in float32, in the graph's operation order
+    (``model.MeteorNet._range_channel``)."""
+    from .model import MeteorNet
+
+    return MeteorNet._range_channel(h, w_).reshape(h, w_).numpy().astype(F32)
+
+
+def range_bias_map(w: MeteorWeights, module: str, h: int, w_: int) -> np.ndarray:
+    """[h * w, Cout] float32 (NHWC rows): the stem conv of the constant range channel (its last input channel, zero
+    padding of the graph) plus the conv bias, computed in float64: ``stem(cat(x, range)) = conv(x; W[:, :-1]) + B``."""
+    import torch
+    F = torch.nn.functional
+
+    p = w.conv(module)
+    wt = torch.from_numpy(np.asarray(p.weight, np.float64)[:, -1:])     # [Cout, 1, 3, 3]
+    r = torch.from_numpy(range_channel(h, w_).astype(np.float64)).reshape(1, 1, h, w_)
+    pad = int(p.pads[0])
+    y = F.conv2d(r, wt, torch.from_numpy(np.asarray(p.bias, np.float64)), stride=tuple(int(s) for s in p.strides),
+                 padding=pad)
+    return np.ascontiguousarray(y[0].permute(1, 2, 0).reshape(-1, y.shape[1]).numpy().astype(F32))
+
+
+def sem_fc_nhwc(w: MeteorWeights) -> Tuple[np.ndarray, np.ndarray]:
+    """sem_ego.11 (Gemm 1280 -> 256 on ``flatten`` of the [64, 5, 4] pooled map, index c * 20 + p) with its inputs
+    re-ordered to the NHWC flatten p * 64 + c: (W [256, 1280], b [256])."""
+    g = w.gemm("sem_ego/sem_ego.11")
+    wt = np.asarray(g.weight, F32)
+    out, k = wt.shape
+    c = 64
+    npos = k // c
+    return np.ascontiguousarray(wt.reshape(out, c, npos).transpose(0, 2, 1).reshape(out, k)), np.asarray(g.bias, F32)

code/tt_meteor/reference/weights.py

@@ -0,0 +1,236 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Weights of ``meteor_v157c3Z.onnx`` (AutowareFoundation/meteor@v1.0), read the way the TT port reads them.
+
+The ONNX is parsed as data with the vendored ``ttaw.weights.OnnxWeights`` (C04): nothing is executed, and every
+parameter is addressed through the graph node that consumes it, so convolution attributes (strides, pads, dilations,
+groups) come from the deployed graph itself, never from re-typed constants. Module names are the ONNX scopes without
+the ``/net/`` prefix and the op suffix, e.g. ``seg_head/d1/d1.3/d1.3.0`` for ``/net/seg_head/d1/d1.3/d1.3.0/Conv``.
+
+BatchNorm: the export folded every BatchNorm2d of the network into its convolution (SPEC section 4.2, "Every BN is
+folded into the conv in the ONNX"; the stem equals ``meteor_v157.pt`` conv1 + BN to 2e-8, S:113), and the graph has
+no BatchNormalization node left (checked by :meth:`MeteorWeights.check_graph`). The port therefore uses the conv
+weights as stored and folds nothing itself: BN folding happens exactly where the TT port folds it, i.e. nowhere.
+The exact rewrites the port applies (the /255 or ImageNet scale into the stem, merged sibling heads, the paint
+gather as a 21->96 1x1 conv, tgate + tfuse merge, the zero ``mot`` channels of traj_stem dropped) are proven on the
+CPU in ``reference.port_form``.
+
+Besides convolutions and Gemms the graph carries a few non-conv constants the reference reads by consuming node:
+the attention-pool parameters (``ego_attn``, ``agent_attn``: packed in-projections sliced by ``Slice`` nodes, learned
+queries expanded by ``Expand``), the e2e LayerNorm, the two learned gates (``risk_gate`` -> ``/net/Mul_19``,
+``dec_gate`` -> ``/net/Tanh``), the constant adaptive-pooling matrices (``MatMul`` operands), the 0.4 m lift ground
+points (``onnx::Expand_1643`` [1, 4, 100000]) and the painted seg2d classes (``/net/Gather`` indices).
+"""
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Dict, Iterator, List, Optional, Tuple
+
+import numpy as np
+
+from ..ttaw.weights import ConvParams, GemmParams, OnnxWeights, file_sha256
+from . import config as C
+
+__all__ = ["MeteorWeights", "MHAParams", "find_onnx", "conv_node", "gemm_node", "POOL_NODES"]
+
+NET = "/net/"
+
+
+def conv_node(module: str) -> str:
+    return f"{NET}{module}/Conv"
+
+
+def gemm_node(module: str) -> str:
+    return f"{NET}{module}/Gemm"
+
+
+# constant adaptive-pool / mean matrices: name -> (MatMul node, constant slot). Windows follow
+# adaptive_avg_pool2d: [floor(i n / m), ceil((i + 1) n / m)) (export_onnx.py:53-104; S:356-366)
+POOL_NODES: Dict[str, Tuple[str, int]] = {
+    "ego_stem_w": ("/net/ego_stem/ego_stem.15/MatMul", 1),       # [16, 1]   global mean over 25x16
+    "ego_stem_h": ("/net/ego_stem/ego_stem.15/MatMul_1", 0),     # [1, 25]
+    "tl_w": ("/net/tl_head/tl_head.8/MatMul", 1),                # [48, 1]   global mean over 27x48
+    "tl_h": ("/net/tl_head/tl_head.8/MatMul_1", 0),              # [1, 27]
+    "fused_pool_w": ("/net/MatMul_1", 1),                        # [500, 16] fused 800x500 -> 25x16 (ego_attn keys)
+    "fused_pool_h": ("/net/MatMul_2", 0),                        # [25, 800]
+    "sem_w": ("/net/sem_ego/sem_ego.9/MatMul", 1),               # [16, 4]   25x16 -> 5x4
+    "sem_h": ("/net/sem_ego/sem_ego.9/MatMul_1", 0),             # [5, 25]
+    "det_pool_w": ("/net/MatMul_3", 1),                          # [250, 16] det_feat 400x250 -> 25x16 (agent keys)
+    "det_pool_h": ("/net/MatMul_4", 0),                          # [25, 400]
+    "fused_mean_w": ("/net/MatMul_5", 1),                        # [500, 1]  mean(fused) for dec_head
+    "fused_mean_h": ("/net/MatMul_6", 0),                        # [1, 800]
+    "e2e_mean_w": ("/net/refiner/e2e/MatMul", 1),                # [500, 1]  the same mean for the e2e refiner
+    "e2e_mean_h": ("/net/refiner/e2e/MatMul_1", 0),              # [1, 800]
+}
+
+
+@dataclass(frozen=True)
+class MHAParams:
+    """``nn.MultiheadAttention`` exported by torch 2.1: packed in-projection (q | k | v rows), out-projection."""
+
+    in_weight: np.ndarray          # [3E, E]
+    in_bias: np.ndarray            # [3E]
+    out_weight: np.ndarray         # [E, E] (Gemm transB=1: y = x W^T + b)
+    out_bias: np.ndarray           # [E]
+    num_heads: int
+    scale_divisor: float           # q is divided by sqrt(E / heads) (the graph's Div constant)
+
+    @property
+    def embed_dim(self) -> int:
+        return int(self.out_weight.shape[0])
+
+
+def find_onnx(weights_dir: Optional[os.PathLike] = None) -> Path:
+    """The ONNX file in ``weights_dir`` (a snapshot of AutowareFoundation/meteor) or, for workspace runs,
+    ``$METEOR_WEIGHTS_DIR`` or ``assets/meteor/hf_meteor`` of the workspace."""
+    cands: List[Path] = []
+    if weights_dir is not None:
+        cands.append(Path(weights_dir))
+    if os.environ.get("METEOR_WEIGHTS_DIR"):
+        cands.append(Path(os.environ["METEOR_WEIGHTS_DIR"]))
+    here = Path(__file__).resolve()
+    for parent in here.parents:
+        if (parent / "assets" / "meteor" / "hf_meteor").is_dir():
+            cands.append(parent / "assets" / "meteor" / "hf_meteor")
+            break
+    for c in cands:
+        p = c if c.suffix == ".onnx" else c / C.ONNX_FILE
+        if p.is_file():
+            return p
+    raise FileNotFoundError(f"{C.ONNX_FILE} not found in {[str(c) for c in cands]}")
+
+
+class MeteorWeights:
+    """The deployed graph's parameters, addressed by consuming node (see the module docstring)."""
+
+    def __init__(self, path: Optional[os.PathLike] = None, *, verify_sha256: bool = False):
+        self.path = find_onnx(path) if (path is None or Path(path).is_dir()) else Path(path)
+        if verify_sha256 and file_sha256(self.path) != C.ONNX_SHA256:
+            raise ValueError(f"{self.path} is not the pinned {C.ONNX_FILE} (sha256 {C.ONNX_SHA256[:12]}...)")
+        self.onnx = OnnxWeights(self.path)
+
+    # ---- generic access ------------------------------------------------------------------------------------------
+    @property
+    def sha256(self) -> str:
+        return self.onnx.sha256
+
+    def conv(self, module: str) -> ConvParams:
+        return self.onnx.conv(conv_node(module))
+
+    def gemm(self, module: str) -> GemmParams:
+        return self.onnx.gemm(gemm_node(module))
+
+    def conv_modules(self) -> List[str]:
+        """Every Conv of the graph by module name, in graph order (190)."""
+        return [n.name[len(NET):-len("/Conv")] for n in self.onnx.nodes("Conv")]
+
+    def gemm_modules(self) -> List[str]:
+        return [n.name[len(NET):-len("/Gemm")] for n in self.onnx.nodes("Gemm")]
+
+    def iter_convs(self) -> Iterator[Tuple[str, ConvParams]]:
+        for m in self.conv_modules():
+            yield m, self.conv(m)
+
+    def const(self, node: str, slot: int) -> np.ndarray:
+        """The constant feeding input ``slot`` of ``node`` (a full node name, e.g. ``/net/Mul_19``)."""
+        return self.onnx.param(node, slot)
+
+    # ---- named non-conv parameters ---------------------------------------------------------------------------------
+    def pool_matrix(self, name: str) -> np.ndarray:
+        node, slot = POOL_NODES[name]
+        return self.const(node, slot)
+
+    def mha(self, name: str) -> MHAParams:
+        """``ego_attn`` (E 96, 4 heads, 3 queries) or ``agent_attn`` (E 128, 4 heads, 4 queries)."""
+        p = f"{NET}{name}/"
+        w_q = self.const(p + "Slice_1", 0)            # the packed [3E, E] in_proj_weight, sliced into q and k|v
+        w_kv = self.const(p + "Slice_2", 0)
+        b_q = self.const(p + "Slice_3", 0)
+        b_kv = self.const(p + "Slice_4", 0)
+        if w_q is not w_kv and not np.array_equal(w_q, w_kv):
+            raise ValueError(f"{name}: the q and k|v slices do not read one packed in-projection")
+        if not np.array_equal(b_q, b_kv):
+            raise ValueError(f"{name}: the q and k|v bias slices do not read one packed bias")
+        out = self.gemm(name)
+        div = float(self.const(p + "Div_1", 1))
+        heads = int(self.const(p + "Div", 1))         # head dim = E // heads (the graph's /Div by 4)
+        e = int(out.weight.shape[0])
+        if abs(div - np.float32(np.sqrt(e / heads))) > 1e-6:
+            raise ValueError(f"{name}: q scale divisor {div} != sqrt({e}/{heads})")
+        return MHAParams(np.asarray(w_q, np.float32), np.asarray(b_q, np.float32), np.asarray(out.weight, np.float32),
+                         np.asarray(out.bias, np.float32), heads, div)
+
+    def queries(self, name: str) -> np.ndarray:
+        """Learned attention-pool queries: ``ego`` [1, 3, 96] (``/net/Expand_1``), ``agent`` [1, 4, 128]
+        (``/net/Expand_2``)."""
+        node = {"ego": "/net/Expand_1", "agent": "/net/Expand_2"}[name]
+        return self.const(node, 0)
+
+    def layer_norm(self) -> Tuple[np.ndarray, np.ndarray, float]:
+        node = self.onnx.node(f"{NET}refiner/e2e/norm/LayerNormalization")
+        return (self.const(node.name, 1), self.const(node.name, 2), float(node.attrs.get("epsilon", 1e-5)))
+
+    def risk_gate(self) -> float:
+        """``net.risk_gate`` (2.2981195, S:120): mode logits -= risk_gate * mean_t sigmoid(risk)(path_t)."""
+        return float(self.const("/net/Mul_19", 0).reshape(()))
+
+    def dec_gate(self) -> float:
+        """``net.dec_gate`` (0.0336921, S:120): ego[:36] += tanh(dec_gate) * (wp_dec - ego[:36])."""
+        return float(self.const("/net/Tanh", 0).reshape(()))
+
+    def lift_ground_points(self) -> np.ndarray:
+        """The constant [1, 4, 100000] homogeneous ground points (x, y, 0, 1) of the 400 x 250 lift grid."""
+        return self.const("/net/Expand", 0)
+
+    def paint_classes(self) -> Tuple[int, ...]:
+        return tuple(int(i) for i in self.const("/net/Gather", 1).reshape(-1))
+
+    def zero_constants(self) -> Dict[str, np.ndarray]:
+        """The baked-out history (``mot``) constants that the graph subtracts / multiplies (both all-zero)."""
+        return {"hist": self.const("/net/Sub_5", 1), "mask": self.const("/net/Mul_14", 1)}
+
+    # ---- graph check ----------------------------------------------------------------------------------------------
+    def check_graph(self) -> Dict[str, int]:
+        """Fail loudly unless the file is the graph this port reproduces (ops, attributes, constants)."""
+        o = self.onnx
+        problems: List[str] = []
+        ops: Dict[str, int] = {}
+        for n in o.nodes():
+            ops[n.op_type] = ops.get(n.op_type, 0) + 1
+        expect = {"Conv": 190, "Gemm": 15, "GridSample": 3, "Resize": 16, "Softmax": 7, "ArgMax": 4,
+                  "LayerNormalization": 1, "AveragePool": 3, "MaxPool": 1, "GatherElements": 2, "CumSum": 2}
+        for op, cnt in expect.items():
+            if ops.get(op, 0) != cnt:
+                problems.append(f"{ops.get(op, 0)} {op} nodes, expected {cnt}")
+        if ops.get("BatchNormalization", 0):
+            problems.append("BatchNormalization nodes present (the reference assumes a fully folded export)")
+        for n in o.nodes("Resize"):
+            a = n.attrs
+            if a.get("mode") != "linear" or a.get("coordinate_transformation_mode") != "half_pixel":
+                problems.append(f"{n.name}: Resize {a.get('mode')}/{a.get('coordinate_transformation_mode')}")
+        for n in o.nodes("GridSample"):
+            want = "border" if n.name == "/net/GridSample_2" else "zeros"
+            a = n.attrs
+            if (a.get("mode"), a.get("padding_mode"), int(a.get("align_corners", 0))) != ("bilinear", want, 0):
+                problems.append(f"{n.name}: GridSample {a}")
+        for n in o.nodes("ArgMax"):
+            if int(n.attrs.get("select_last_index", 0)) != 0:
+                problems.append(f"{n.name}: select_last_index")
+        if self.paint_classes() != C.PAINT_CLASSES:
+            problems.append(f"paint classes {self.paint_classes()} != {C.PAINT_CLASSES}")
+        g, b, eps = self.layer_norm()
+        if g.shape != (139,) or abs(eps - C.E2E_LN_EPS) > 1e-9:
+            problems.append(f"e2e LayerNorm {g.shape} eps {eps}")
+        z = self.zero_constants()
+        if any(np.any(v != 0) for v in z.values()):
+            problems.append("the baked history constants are not all zero")
+        pts = self.lift_ground_points()
+        if pts.shape != (1, 4, C.LIFT_H * C.LIFT_W):
+            problems.append(f"lift points {pts.shape}")
+        for name, (node, slot) in POOL_NODES.items():
+            if not o.has(o.node(node).inputs[slot]):
+                problems.append(f"pool matrix {name}: {node} slot {slot} is not constant")
+        if problems:
+            raise ValueError(f"{self.path.name} is not the graph this port reproduces: " + "; ".join(problems))
+        return ops

code/tt_meteor/samples/README.md

@@ -0,0 +1,31 @@
+# Sample inputs shipped with meteor-p150
+
+Only REDISTRIBUTABLE data goes here (Apache-2.0 / MIT / CC-BY with attribution). Data whose license is unstated or
+non-commercial (the Autoware demo rosbag, nuScenes, Argoverse 2, ...) is NOT shipped: `code/scripts/fetch_samples.sh`
+downloads it (sha256-checked) for the tests and benchmarks.
+
+Never use the `.bin` suffix here (or `.pt`, `.pth`, `.ckpt`, `.safetensors`): tt-model's staging silently drops
+those suffixes from `code/` (`CODE_IGNORE`, tt-model-manager `src/tt_kernel/build.py:328-332`). Store point clouds as
+`.npy` / `.npz` / `.pcd`.
+
+Next to each sample, `<stem>.reference.json` is the `/predict` body of the fp32 CPU reference (`tt_meteor.reference`,
+`Output.to_dict()`) on it; `server/smoke_test.py` compares the served output with it (the smoke gate for synthetic
+samples, which may give no detections). When the output depends on the serve profile or the model variant, store
+one per profile or variant as `<stem>.<profile>.reference.json` / `<stem>.<variant>.reference.json`. Regenerate it
+whenever the reference, the weights or the post-processing changes.
+
+| file | content | source | license |
+|---|---|---|---|
+| `synthetic_8cam.json` | the request manifest: the eight camera PNGs (all present), calibration preset `synthetic_8cam`, `ego_speed` 8.0 m/s, stream id + ego pose; `tt_meteor.load_sample(path)` -> `model(**kwargs)`; `server/client.py --sample` / `server/smoke_test.py` build the `/predict` body from it | generated by `code/scripts/make_synthetic_sample.py` | Apache-2.0 (data generated by this repository) |
+| `synthetic_8cam/<CAM>.png` | eight 768x432 RGB images (lossless PNG, 1.4 MB in total): a ray-cast street (a straight two-lane road with lane lines, a stop line and a zebra crossing, kerbs, pavements, facades, box-shaped vehicles and pedestrians) seen by a generic METEOR-like rig; the pixels of the objects (only those) were then optimised against the fp32 CPU reference (gradient ascent through the image encoder, the lift, the BEV detector and the box refiner, 88 steps) until the 3D heads report six vehicles and two pedestrians with large margins (scores 0.889-0.993 against the thresholds 0.35 / 0.15) and nothing else (strongest other heatmap cell 0.179 vehicle / 0.072 VRU), so bf16 numerics cannot flip a published box. A test pattern, not a photograph: the objects carry noisy texture | as above | Apache-2.0 |
+| `synthetic_8cam.reference.json` | the `/predict` body of the fp32 CPU reference (`tt_meteor.reference.pipeline.MeteorReference`, PCC 1.0 vs ONNX Runtime) on the request above as a fresh stream (`timing_ms` emptied): 8 boxes (6 VEHICLE, 2 VRU), 10 2D boxes, plan mode 0 (the car ahead and a red light: the selected path brakes), traffic light red; `server/smoke_test.py` (the container smoke, `--expect VEHICLE:6,VRU:2` by default) and `tests/test_e2e_device.py` compare the served body with it | `code/scripts/make_synthetic_sample.py finalise` | Apache-2.0 |
+
+The calibration preset `../calib/synthetic_8cam.json` is the same generated rig (round numbers of our own: METEOR's
+slot layout, 98 deg wide cameras front and back, corner cameras pitched 25 deg down, 30.4 deg narrow cameras, fy =
+0.87 fx like METEOR's vertically squashed training images, 1.9 m above the road).
+
+A real-world sample (PandaSet 019 frame 40, CC BY 4.0 + PandaSet Dataset Terms, METEOR's 8-slot layout) is prepared
+but **not shipped** until the redistribution of PandaSet-derived samples is approved; the device tests read it from
+the git-ignored `staging_samples_pandaset/` of the development checkout (`tests/paths.py`). METEOR's own demo scenes
+(`AutowareFoundation/meteor-demo-scenes`, research / demonstration use only) and nuScenes (CC BY-NC-SA 4.0) are never
+shipped: `code/scripts/fetch_samples.sh` notes where to fetch them.