tt-model push frnet-p150 (container)

.gitattributes

@@ -33,3 +33,22 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/263c3393dbadf7dc0379d0fc3f574de2e1bf1f288d4cf6d258c862eb309fed88 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/2d8b1b67a1bfa51712efa01fd56489f97766c9d4439440d132b211edfefea304 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/2fcc025d0fe89554778cf89bcb74f1084552925ef59ab8520a3e20034f91cfe7 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/4d1d911054857424c0bbb8ffeac532a517210d4ce9dd8b458c995a934dc16c95 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/60918f793df50b25c4d69e6063db590866f48304ceb84738f3588a4ea3af1d39 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/7f8e824c081d26d72fc21097753f339930fd6fa33ee4f6e4f79ebdcfae6bb4a9 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/8a364a911fa5ef61e168db9466a9c1916109d07dd609a66102a077600a7928ce filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/955c696fdd4076cc41ef9b404f0f248af208a7058c33f685360e8ad5a8a7e96f filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/98c4455a98982b35380ec62f90d5cf0edff02eeb4b3bb18595d871fdeb0c0c67 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/9e742f886b889d3b324cb9e64252ac7b15c41dc59b900afea699edefda392ddc filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/ac4a0c0abcd00417126e1fdeb179cd923c8bd292a1c7a8b3091cdbfd79cf1a59 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/bffe9990635d2390c86310d6135a1f2fb109a288c2678108e6981ee0997f8d71 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/c241aa41a5566f12cb0d8c8d32dc73b0580c202137bdd5e6dd934fa7c51a13a4 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/d325fca6b89f9da7c9aec617b11eeae4e04520f0fb771d27e965e27ac7b660fb filter=lfs diff=lfs merge=lfs -text
+media/frnet_ot128_nuscenes_scene-0103_20_tt_NC.png filter=lfs diff=lfs merge=lfs -text
+media/frnet_ot128_pandaset_019_40_tt.png filter=lfs diff=lfs merge=lfs -text
+media/frnet_ot128_pandaset_019_40_tt_vs_cpu.png filter=lfs diff=lfs merge=lfs -text
+media/frnet_ot128_pandaset_019_bev_tt.gif filter=lfs diff=lfs merge=lfs -text
+media/frnet_ot128_pandaset_029_40_tt.png filter=lfs diff=lfs merge=lfs -text

OPT_BASELINE.md

@@ -0,0 +1,557 @@
+# frnet-p150 baseline on the p150 (before optimization)
+
+Date 2026-10-09. The model code is the port's final commit `14fa968` (M2-M4; VERIFY_PORT round 1: PASS), unchanged.
+The baseline commits add only the measurement scripts (`c246f92`: `code/scripts/bench.py`, `profile_ops.py`,
+`profile_summary.py`, `visualizer_report.py`; this commit: `visualizer_report.py --json-dir`) and this file.
+tt-metal `44d66500520` + `patches/tt-metal-eth-dispatch.patch`. Weights `AutowareFoundation/lidar_frnet` @
+`a53a1c11b0b` (tag `v2.0`), read from the sha256-pinned local copy through `FRNET_WEIGHTS_DIR`.
+
+Both serve profiles are measured: **`ot128`** (default; Hesai OT128, 128 × 1024 frustum grid, `N_cap` 160,000
+points, `M_cap` 60,000 cells) and **`qt128`** (Hesai QT128, 128 × 256 grid, `N_cap` 120,000, `M_cap` 60,000). The
+device graph has fixed shapes per profile (`N_cap`, `M_cap`). So the point-wise device work does not depend on the
+frame's point count, while the host rows and the data-dependent K1 segment reduction do.
+
+Inputs, batch 1:
+- **Synthetic scan** (generated by us, no dataset, Apache-2.0):
+  - made by `research/frnet/scripts/frnet_preproc_ref.py synthetic_scan` (ray cast against ground, cars, facades,
+    poles and pedestrians; seed 0; sensor 2 m above the ground) through
+    `logs/frnet/m5_baseline/scripts/make_bench_inputs.py`. Its fp32 CPU reference (ONNX Runtime on the pinned
+    deployed ONNX) sits next to it;
+  - OT128: 512 columns, 53,411 points, 52,613 after the ego crop, no interpolated point, N = M = 52,613;
+  - QT128: 600 columns, 45,783 points, 41,806 after the crop, N = 69,553 after interpolation, M = 18,926;
+  - it stands in for the release's synthetic shipped sample (the same generator), which is not built yet. Update at
+    the release: the shipped `code/tt_frnet/samples/synthetic_<variant>.npz` (`code/scripts/make_sample.py`) are
+    byte-identical to these bench inputs, so the synthetic rows below are the shipped sample's.
+- **PandaSet 019 frame 40** (CC BY 4.0 + PandaSet terms), today's `code/tt_frnet/samples/pandaset_019_40.npz`:
+  - 107,146 points, 106,501 after the ego crop;
+  - OT128: N = 145,961 after interpolation, M = 40,691; QT128: N = 106,585, M = 6,255.
+
+  The first release moved it to the git-ignored `staging_samples_pandaset/samples/` (not shipped until PandaSet
+  samples are approved); the commands below name its old path.
+
+Configuration of every number, unless a row says otherwise:
+- ETH dispatch, 1 CQ, 12×10 grid (`device.compute_with_storage_grid_size()`, printed in each log);
+- the shipped precision knobs of `tt-model.yaml` serve.env: `FRNET_POINT_TERMS=2` (two-term bf16 point island),
+  `FRNET_MAP_DTYPE=float32` (fp32 image-branch maps), `FRNET_CONV_TERMS=1`. The job unsets every other `FRNET_*`, so
+  the code defaults (= the serve.env pins) apply.
+
+## Environment
+
+| item | value |
+|---|---|
+| tt-metal | `44d66500520` (main) + `patches/tt-metal-eth-dispatch.patch` (sha256 `08d0ddf6…`, the same file as the workspace copy) |
+| ttaw | 0.20.0, vendored from `common` `89dec49` (`code/tt_frnet/ttaw/VENDORED.json`, `source_dirty: false`). `vendor.py --check`: 0 differences, so no re-vendor was needed |
+| device | Blackhole p150b, KMD 2.10.0. ETH dispatch, 1 CQ, 12×10 = 120 cores, `l1_small_size` 64 KiB, `trace_region_size` 256 MiB (8.7 MiB used by OT128, 5.6 MiB by QT128). One trace per process; 227 programs in the program cache (OT128), 174 (QT128) |
+| AICLK | 1350 MHz median (min 1343) over 4,568 (OT128) and 3,278 (QT128) samples, one every 50 ms, during the stage benches (`ttaw.profiling.AiclkSampler`, sysfs); the same in every matrix run. Board power median 78 W (max 209 W), ASIC 64.9–71.7 °C. The profiler CSV header reads CHIP_FREQ 1350 MHz |
+| numerics | point island (encoder, compression, point stem / fusions / fuse, head): fp32 activations, every linear as three bf16 matmuls (`x_hi W_hi + x_hi W_lo + x_lo W_hi`) with fp32 outputs summed in fp32, HiFi4. Image branch: fp32 weights, bf16 conv operands, fp32 maps (convs write fp32; HardSwish, adds and `addcmul` in fp32), HiFi4 + fp32 dest acc + packer L1 acc. Gathers and resize outputs bf16. The profile shows every matmul and conv as HiFi4 |
+| host | AMD EPYC-Rome VM, 8 vCPUs, 90 GB RAM, Linux 6.8. Shared with other agents' jobs: load average 4.1–6.6 during the device job. `OMP_NUM_THREADS=4`. Host rows move with that load; device rows do not (b2b rounds repeat to 0.02 ms) |
+
+## How to run
+
+```bash
+ROOT=/home/ubuntu/experiments/tt-models; cd $ROOT/bundles/frnet-p150; source $ROOT/bin/tt-env.sh
+export PYTHONPATH=$PWD/code:$PYTHONPATH HF_HUB_OFFLINE=1 OMP_NUM_THREADS=4 \
+       FRNET_WEIGHTS_DIR=$ROOT/assets/frnet/hf_lidar_frnet          # v2.0, sha256-pinned
+# accuracy gates (per-module PCC + end-to-end agreement vs the fp32 CPU reference, both profiles), one job
+$ROOT/bin/devrun -t 1800 -- env TTAW_GATES_READONLY=1 python -m pytest -p no:cacheprovider -q -s \
+    code/tt_frnet/tests/test_pcc_device.py code/tt_frnet/tests/test_e2e_device.py
+# stage breakdown (pre split, host staging, H2D split, trace, b2b, D2H split, unsort, post, e2e p50/p99), with a check
+# of each served output against its stored CPU reference (<stem>.<variant>.reference.json next to the input)
+$ROOT/bin/devrun -t 900 -- python code/scripts/bench.py --variant ot128 --iters 100 --warmup 10 --b2b-iters 50 \
+    --b2b-rounds 5 --input <synthetic_ot128.npz> --input code/tt_frnet/samples/pandaset_019_40.npz --json <out>.json
+# dispatch / CQ matrix: one process per configuration
+$ROOT/bin/devrun -t 900 -- python code/scripts/bench.py --variant qt128 --iters 50 --input <npz> --dispatch worker --cqs 2 --json <out>.json
+# device profile: one eager forward (module / layer signposts) + one traced replay between signposts (1,104 programs:
+# above the profiler's default 1,000-op buffer, hence --op-support-count; each section is flushed on its own)
+$ROOT/bin/devrun -t 3600 -- python -m tracy -r -p -v --no-web-server --op-support-count 4000 \
+    -o $ROOT/generated/profiler/frnet_baseline code/scripts/profile_ops.py --variant ot128 --input <npz>
+tt-perf-report <ops_perf_results_*.csv> --start-signpost trace --end-signpost trace_end --tracing-mode
+python code/scripts/profile_summary.py <ops_perf_results_*.csv> profile_summary.json --md profile_summary.md
+# ttnn-visualizer memory + graph report of one eager forward (slow: never used for timing; the 7.5 GB graph JSON goes to tmpfs)
+$ROOT/bin/devrun -t 2400 -- python code/scripts/visualizer_report.py $ROOT/generated/ttnn_visualizer/frnet_baseline \
+    --variant ot128 --input code/tt_frnet/samples/pandaset_019_40.npz --json-dir /dev/shm/frnet_visualizer
+ttnn-visualizer --profiler-path $ROOT/generated/ttnn_visualizer/frnet_baseline \
+    --performance-path $ROOT/generated/profiler/frnet_baseline/reports/2026_10_09_13_56_53
+```
+
+- All device measurements ran in two `bin/devrun` windows on 2026-10-09, every step rc 0 (job script
+  `logs/frnet/m5_baseline/scripts/job1_all.sh`, environment `env.sh`):
+  - 13:08:29–13:10:07 UTC: the device suite (step 1);
+  - 13:26:44–14:04:39 UTC: stage benches, matrix, both profiles and the visualizer report (steps 2–5).
+
+  The first window also started steps 2–5, which failed in 1 s each: the HF cache holds no snapshot of the pinned
+  revision, so `env.sh` now sets `FRNET_WEIGHTS_DIR` (logs of that attempt: `logs/frnet/m5_baseline/try0/`).
+- The OT128 replay is **1,104 programs**, above the device profiler's default buffer of 1,000. The run raises the
+  buffer with `--op-support-count 4000`, and `profile_ops.py` flushes each section with `ttnn.ReadDeviceProfiler`.
+  The replay's op sequence matches the eager one op code by op code (1,104 / 1,104; QT128 759 / 759), so no op was
+  dropped.
+
+## Accuracy (baseline)
+
+The device suite ran read-only (`TTAW_GATES_READONLY=1`, reports tagged `_m5`: `logs/frnet/{pcc,e2e,latency}_{ot128,qt128}_m5.json`,
+`logs/frnet/m5_baseline/try0/suite_r1.log`): **12 passed** in 94.5 s. The values are bit-identical to PORT_LOG M4 (j6)
+and VERIFY_PORT round 1 (the graph is deterministic). The gate JSONs are unchanged (`git status` clean).
+
+| check | gate | OT128 | QT128 |
+|---|---|---|---|
+| every gated module tap (PCC vs the fp32 reference, sample, replay outputs) | ≥ 0.999 | min 0.999729 (`point_fusion1`) | min 0.999649 (`attention4`) |
+| `pred_probs` PCC / raw-point argmax agreement, sample | ≥ 0.998 / ≥ 98.5 % | 0.999942 / 99.581 % | 0.999844 / 99.295 % |
+| public frames (OT128 17, QT128 5): min `pred_probs` PCC | ≥ 0.998 | 0.999688 | 0.999805 |
+| public frames: min raw-point argmax agreement | ≥ 98.5 % | **98.907 %** (nuScenes 0916 kf40; margin 0.41 pt) | 99.158 % (PandaSet 029 f40) |
+| replay == eager | bit-exact | 45 / 45 taps | 45 / 45 taps |
+| CPU reference vs ONNX Runtime on the deployed ONNX (the reference itself is right) | ≥ 0.9999 per tap | ≥ 0.99999999999 (VERIFY_PORT, research venv) | same |
+
+- **Every bench run also checks its served output** against the stored fp32 CPU reference with the container-smoke
+  gates (`server/smoke_test.py` `REFERENCE_GATES`); all passed:
+  - OT128: synthetic labels 99.55 % / filter 99.77 % (max |Δscore| 0.072); PandaSet 99.59 % / 99.76 % (0.109);
+  - QT128: synthetic 99.62 % / 99.87 % (0.144); PandaSet 99.29 % / 99.62 % (0.150);
+  - every matrix run (ETH / WORKER × 1 / 2 CQs, both profiles) passes the same check.
+- **The binding constraint of the optimization phase is nuScenes 0916 kf40 (OT128): 98.907 % against the 98.5 %
+  gate.** Every precision-reducing change measured in the port failed it: bf16 maps 98.28 %, an fp32 × fp32 point
+  island 97.50 % (PORT_LOG M3). See "The precision cost of the policy" below.
+- Host suite on the fake ttnn with the new scripts: **112 passed, 13 skipped** (the device files and the
+  onnxruntime-only test). Log: `logs/frnet/m5_baseline/host_suite.log`.
+
+## Performance (warm, batch 1, median of 100 iterations)
+
+Each stage is synchronised on its own, so the rows do not add up to e2e exactly. Values are p50 (p99) in ms; b2b is
+the median of 5 rounds of 50 back-to-back replays. `pre` includes the file decode (`pre.load`); an HTTP request
+decodes its points envelope inside the call.
+
+| stage | OT128 synthetic | OT128 PandaSet 019 f40 | QT128 synthetic | QT128 PandaSet 019 f40 |
+|---|---:|---:|---:|---:|
+| host preprocess as served (`model._prepare`) | 55.3 (59.1) | 107.2 (120.0) | 54.2 (69.7) | 83.5 (106.1) |
+| … decode (`load_points`) | 6.9 | 12.9 | 6.2 | 13.1 |
+| … ego crop / projection / unique cells | 1.6 / 3.4 / 1.8 | 3.2 / 8.0 / 2.4 | 1.5 / 4.7 / 0.9 | 3.3 / 5.9 / 1.0 |
+| … fp16 interpolation grid + interpolated points | 18.4 | 37.4 | 17.2 | 29.3 |
+| … device plan (`build_plan`: sort, CSR offsets, gather tables, 8-d features) | 22.5 | 41.8 | 23.3 | 31.4 |
+| `FrameStaging.write` (range checks + persistent host buffers) | 3.3 | 3.2 | 2.4 | 2.3 |
+| H2D, 9 inputs (8.6 MB OT128, 6.2 MB QT128) | 5.42 (5.57) | 5.41 (5.65) | 4.04 (5.59) | 4.00 (4.48) |
+| … of which `features` fp32 `[N_cap, 8]` | 4.13 | 4.13 | 3.14 | 3.13 |
+| device trace, one blocking forward | 210.02 (210.15) | 211.72 (211.89) | 133.53 (134.01) | 134.91 (135.12) |
+| **back-to-back trace replays, per forward** | **209.948** | **211.643** | **133.434** | **134.866** |
+| D2H as served (fp32 `pred_probs` `[N_cap, 32]` → numpy) | 7.65 (20.2) | 7.43 (19.8) | 5.80 (8.7) | 5.55 (7.6) |
+| … copy / host untilize (to numpy) | 2.25 / 4.95 | 2.29 / 5.01 | 1.77 / 4.15 | 1.78 / 3.75 |
+| unsort (first N rows / 27 columns, network-input order) | 7.1 (14.1) | 12.8 (23.3) | 6.1 (13.2) | 5.9 (13.7) |
+| host postprocess (first-max argmax, filter, output assembly) | 5.0 (6.8) | 14.8 (19.3) | 6.8 (9.8) | 10.5 (15.9) |
+| **e2e `model()`** | **292.6 (299.4)** | **355.7 (367.0)** | **221.1 (233.5)** | **244.3 (258.7)** |
+
+- **Throughput.**
+  - Back-to-back replays give **4.72 frames/s** of device throughput on OT128 (PandaSet) and **7.41 frames/s** on
+    QT128.
+  - A synchronous `model()` call gives 2.81 frames/s (OT128 PandaSet), 3.42 (OT128 synthetic), 4.09 (QT128
+    PandaSet) and 4.52 (QT128 synthetic).
+- **The API's own `timing_ms` medians** (OT128 PandaSet): preprocess 105.4 ms, device 235.7 ms, postprocess 14.6 ms,
+  total 355.7 ms. The API's `device` stage is staging + H2D + trace + D2H + unsort: 24 of its 236 ms are outside the replay.
+- **The device is 60 % of e2e on OT128** (211.6 of 355.7 ms, PandaSet) and 55 % on QT128 (134.9 of 244.3 ms). The
+  host preprocess is 30 % (OT128) and 34 % (QT128).
+- **The device time barely depends on the frame** (fixed `N_cap` / `M_cap`): the synthetic OT128 scan with 36 % of
+  PandaSet's points replays only 1.7 ms faster (209.95 vs 211.64 ms). The difference is in K1 and the gathers.
+- **Load** (warm JIT cache): `from_pretrained` 6.5 s OT128 / 6.9 s QT128 in the stage bench (build 0.28 s,
+  warm-up + capture 2.0 s). WORKER (11×10) compiles its own program set: 137 s OT128, 82 s QT128. A process with the
+  device profiler compiles its own kernels (146 s).
+
+## Dispatch / CQ matrix (D14)
+
+Setup:
+- synthetic scan, one process per configuration, all in one devrun job (13:34–13:53 UTC);
+- 50 iterations plus 5 × 30 back-to-back replays each;
+- host load 4.1–5.8 throughout.
+
+| profile | configuration | grid | programs | b2b replay ms | trace p50 ms | H2D p50 / min ms | D2H p50 / min ms | e2e p50 / min ms |
+|---|---|---|---:|---:|---:|---:|---:|---:|
+| OT128 | **ETH-1CQ** (shipped) | 12×10 | 227 | **209.948** | 210.00 | 5.39 / 5.35 | 7.61 / 7.33 | 292.9 / 291.1 |
+| OT128 | ETH-2CQ | 12×10 | 227 | 211.317 | 211.45 | 5.49 / 5.46 | 7.65 / 7.30 | 290.5 / 289.7 |
+| OT128 | WORKER-2CQ | 11×10 | 231 | 211.567 | 211.73 | 4.57 / 4.55 | 7.56 / 7.19 | 293.4 / 291.7 |
+| OT128 | WORKER-1CQ | 11×10 | 231 | 211.563 | 211.63 | 4.49 / 4.47 | 7.48 / 7.32 | 291.7 / 291.2 |
+| OT128 | ETH-1CQ again (drift check) | 12×10 | 227 | 209.959 | 210.02 | 5.40 / 5.34 | 7.46 / 7.17 | 291.7 / 290.8 |
+| QT128 | **ETH-1CQ** (shipped) | 12×10 | 174 | **133.437** | 133.51 | 4.02 / 3.98 | 5.51 / 5.33 | 208.8 / 207.5 |
+| QT128 | ETH-2CQ | 12×10 | 174 | 134.250 | 134.38 | 4.11 / 4.08 | 5.50 / 5.28 | 210.0 / 209.1 |
+| QT128 | WORKER-2CQ | 11×10 | 174 | 153.721 | 153.82 | 3.45 / 3.42 | 5.53 / 5.31 | 229.3 / 227.7 |
+| QT128 | WORKER-1CQ | 11×10 | 174 | 153.709 | 153.77 | 3.39 / 3.37 | 5.43 / 5.30 | 226.0 / 225.3 |
+| QT128 | ETH-1CQ again (drift check) | 12×10 | 174 | 133.433 | 133.50 | 4.01 / 3.98 | 5.42 / 5.31 | 206.3 / 205.5 |
+
+- **ETH stays the default: it is better on the device in both profiles.**
+  - OT128: ETH replays 1.6 ms faster than WORKER (209.95 vs 211.56 ms, −0.8 %).
+  - QT128: ETH replays **20.3 ms faster** (133.44 vs 153.71 ms, −13.2 %). On the 128 × 256 grid the 11×10
+    partitioning of the point matmuls and convs is clearly worse. This is the largest ETH/WORKER gap measured so
+    far in this workspace.
+  - WORKER's H2D is 0.6–0.9 ms faster, as in the other matrices. It does not offset the replay.
+- **1 CQ.** ETH with 2 CQs replays 0.8–1.4 ms slower (ETH-2CQ runs without dispatch_s, PLAN §0.2). A synchronous
+  request gains nothing from the second queue (e2e within host noise). Uploads are 5.4 ms and readbacks 7.5 ms per
+  frame, so pipelined 2CQ staging could hide at most about 13 ms of a 212 ms frame (−6 %).
+  - **Decision: ETH, 1 CQ stays pinned** (`serve.env FRNET_DISPATCH=eth`, `FRNET_NUM_CQS=1`, D14).
+  - Re-measure a pipelined 2CQ mode once the replay is no longer 15–30× the transfers.
+- The drift checks repeat to 0.01 ms (OT128) and 0.004 ms (QT128). One OT128 ETH-1CQ b2b round read 211.76 ms
+  (the other four 209.94–209.95), and the median is unaffected.
+
+## Host vs device, and every host<->device transfer per forward
+
+**Host.** Only the documented fallbacks of PLAN §2.8 run on the host; there is no host round trip and no host
+fallback op inside the device graph.
+
+Before the device (`host/preprocess.py`, numpy), in the node's order:
+- the ego crop (float32 `R p + t`);
+- the float32 projection;
+- the fp16 farthest-point interpolation grid and the interpolated points;
+- the unique cells and the profile checks;
+- the device plan: the sort by cell, CSR offsets, `point2pix_s` / `pix2vox_s` for s = 1, 2, 4, 8, and the 8-d
+  features with `pre_norm` folded.
+
+After the readback:
+- the un-permute (`plan.unsort`) of the first N rows / 27 columns;
+- the first-max argmax with the −1e9 start (never a device last-dim argmax, P10), the score, the class filter
+  (`prob ≥ 0.05` on `drivable_surface`) and the output assembly.
+
+**H2D, 9 per frame.** `copy_host_to_device_tensor` from the persistent `HostStaging` buffers into the trace inputs,
+all ROW_MAJOR in DRAM (OT128 bytes; QT128 in brackets):
+
+| input | shape, dtype | bytes | ms |
+|---|---|---:|---:|
+| `features` | `[1, 1, N_cap, 8]` fp32 (32 B pages) | 5,120,000 (3,840,000) | 4.13 (3.13) |
+| `offsets` (K1 CSR) | `[1, 60002]` uint32 | 240,008 | 0.17 |
+| `p2p_1/2/4/8` (pixel2point) | 4 × `[1, N_cap]` uint32 | 4 × 640,000 (480,000) | 0.27–0.31 each |
+| `v2p_1/2/4/8` (frustum2pixel) | `[1, (H/s)(W/s)]` uint32 | 524,288 + 131,072 + 32,768 + 8,192 | 0.06–0.26 |
+| **total** | | **8,616,328** (6,174,088) | **5.41** (4.00) |
+
+`features` moves at 1.24 GB/s: 160,000 pages of 32 bytes. The other inputs reach 1.4–2.1 GB/s.
+
+**D2H, 1 per frame.** The fp32 `pred_probs` TILE `[N_cap, 32]`:
+- 20,480,000 B (OT128) / 15,360,000 B (QT128);
+- of the 32 columns, 27 hold classes and 5 are padding (bias −1e30: probability 0);
+- rows beyond N are padding (PandaSet OT128 uses 145,961 of 160,000, the nuScenes frames 20–28 k);
+- copy 2.29 ms (8.9 GB/s), then host untilize 5.0 ms.
+
+**The served device path** (the API's `device` stage) is 236 ms on OT128 PandaSet. 212 ms of it run on the device.
+
+## Grid usage
+
+- **Nothing hard-codes 11×10 or 12×10** (VERIFY_PORT §3(b): the grid comes from `device.compute_with_storage_grid_size()`;
+  K1 splits its work on the device by two binary searches, so ETH and WORKER give the same outputs). On 11×10, WORKER
+  builds 231 instead of 227 programs (OT128).
+- **OT128: 456 of the 1,104 programs per frame run on all 120 cores, 648 on fewer** (26.3 ms of kernel time):
+  - every point-island op on `N_cap` rows, the K1 kernels and the softmax run on 120 cores. The `M_cap`-row
+    compression matmuls run on 118;
+  - the image-branch convs run on 96 (28 conv programs), 108 (22) and 40 cores (stage 4, `pixel_fusion4`,
+    `attention4`: the 16 × 128 stride-8 map). Their glue (halo, move, padded slice, slice write, resharding) runs on
+    40–114 cores. The height-sharded configurations use whole row groups;
+  - the convs reach a weighted 59.7 % of the HiFi4 FLOPs (tt-perf-report). The ten block-sharded conv programs reach
+    26.1 %, but take only 0.42 ms.
+- **QT128: the point matmuls run on 118 cores** (3,750 tile rows of 120,000 points). The convs of the narrow
+  128 × 256 grid use 32–108 cores.
+- **L1 is empty between ops.** The ttnn-visualizer report shows no L1 buffer at any op boundary.
+  - Every activation round-trips DRAM interleaved.
+  - DRAM peaks at **218.6 MB per bank (1.75 GB in all, weights included)** at the first partial-sum add of
+    `decoder.point_fuse1`. At that point the fp32 point features p0–p4 (`[160000, 128]`, 82 MB each) are still alive
+    next to the fp32 `[160000, 256]` partials (164 MB each).
+  - L1 for tensors is 167.5 MB (120 × 1,395,712 B, the visualizer's device record).
+- **What would fit in L1:**
+  - a 128 × 1024 × 128 image map: 33.5 MB in bf16, 67 MB in fp32;
+  - every point-island weight: the largest, `point_fuse1` 640 × 256, is 655 KB as two bf16 terms.
+
+  No fp32 point tensor at `N_cap` fits (82–164 MB each), so the point island needs row-block streaming.
+
+## What is already fused / traced
+
+**Done:**
+- **One trace per process** covers the whole device graph: encoder → K1 → compression → frustum2pixel → stem →
+  point stem → fusion stem → 4 stages → decoder → point fuse → head → softmax → one packed output.
+  - Capture takes 2.6 s (OT128) / 2.0 s (QT128).
+  - No eager op runs between replays; replay == eager bit for bit (a gate).
+  - Eagerly, under the profiler, the same 1,104 programs span 260.2 ms (op-to-op gaps 47.6 ms of host dispatch),
+    against 213.3 ms traced.
+- **K1 `segment_reduce`** (owned `ttnn.generic_op`, bit-exact). It replaces the log-step stock-op fallback, which
+  would take 3R + 1 programs per site, with one program per segmented-max site (6 per frame). It reads the
+  matmul's TILE output directly (no untilize). Sites 1–5 write the bf16 frustum2pixel gather table, zero row
+  included.
+- **Exact rewrites** (all from the port, each tested against ORT or an fp32 emulation):
+  - BN folded into the 14 point linears and the 53 convs;
+  - `pre_norm` folded into the host features;
+  - frustum2pixel / pixel2point as TILE + PADDED `ttnn.embedding` gathers with a zero sentinel row (C19);
+  - concat-free point layers (K-split linears) and K-split fusion convs. `fuse_layer1` is a 5 × 128 split-K
+    `KSplitConv` (P14), so the 160 MiB decoder concat never exists;
+  - align-corners resizes as interpolation matmuls (C18);
+  - the 27 logits padded to 32 with a −1e30 bias, so the softmax is exact;
+  - fused SIGMOID in the attention convs, ReLU in the point linears;
+  - a bf16 input of a two-term linear (the gathered map rows) takes two matmuls instead of three (`lo = 0`).
+- **Host side:** `HostStaging` writes the plan into persistent zero-copy host tensors (no per-frame `from_torch`),
+  and out-of-profile frames return before touching the device.
+
+**Not fused / not resident:**
+- **the two-term point island:**
+  - each point linear runs one split per fp32 input: typecast to bf16, typecast back, subtract, typecast. So
+    4 programs per input;
+  - then 2–3 bf16 matmuls per part, each writing a full fp32 `[N_cap, C_out]` partial to DRAM;
+  - then one fp32 add per partial.
+
+  The 5-part `point_fuse1` alone is 49 programs and 39.9 ms;
+- every activation in DRAM interleaved between ops. The 128 × 1024 convs run ttnn's automatic DRAM slicing, each
+  slice with its own padded slice, halo, move and slice write;
+- every HardSwish is its own fp32 `ttnn.hardswish` (P5: a fused HardSwish is silently skipped), 46 per frame;
+- the resizes (C18) run as transpose + matmul + reshape + tilize / untilize chains with bf16 outputs (C18 cannot
+  write fp32 rows at 1024 × 128);
+- pixel2point: untilize of the map, then the gather, then a typecast to fp32;
+- every point-wise op runs on all `N_cap` rows and K1 writes all `M_cap` segments, whatever the frame's N and M;
+- no megakernel. Projection, interpolation, the plan, the argmax and the filter run on the host.
+
+## Device profile (traced replay; kernel sum vs span, op-to-op gaps)
+
+One traced replay, ETH-1CQ, PandaSet 019 f40. Reports: OT128 `generated/profiler/frnet_baseline/reports/2026_10_09_13_56_53/`,
+QT128 `generated/profiler/frnet_baseline_qt128/reports/2026_10_09_13_59_21/`.
+
+| | OT128 | QT128 |
+|---|---:|---:|
+| programs per frame | **1,104** | **759** |
+| distinct programs (program cache) | 227 | 174 |
+| ttnn calls per frame (visualizer, OT128) | 491: 123 typecast, 84 add, 72 linear, 62 conv2d, 46 hardswish, 27 to_layout, 21 reshape, 20 subtract, 12 embedding, 6 generic_op (K1), 6 transpose, 6 matmul (resize), 4 addcmul, 1 pad, 1 softmax; plus 439 deallocate | – |
+| device kernel sum | **212.64 ms** | **134.81 ms** |
+| span (first FW start → last FW end at 1350 MHz) | 213.27 ms (b2b 211.64) | n/a (see below; b2b 134.87) |
+| op-to-op gaps | 0.628 ms in all, median 0.55 µs | 0.443 ms, median 0.57 µs |
+| conv + matmul kernels | 76.70 ms (36 %) | 51.62 ms (38 %) |
+| math fidelity | HiFi4 on all 677 compute ops | HiFi4 on all 539 |
+
+- **The trace is kernel-time bound: dispatch is not a sink** (0.3 % of the span). Cutting programs pays only through
+  the kernel time of the programs it removes. A megakernel's dispatch saving is at most 0.6 ms.
+- QT128's span is not usable: one replayed conv (row 3631 of the CSV, `backbone.layer4`) reports a FW duration of
+  3.18 s against a 41 µs kernel, a profiler artefact. The b2b replay (134.87 ms, five rounds within 0.01 ms) shows no
+  such stall. The kernel sum plus gaps (135.25 ms) is the QT128 reference.
+
+**By part of the graph** (`logs/frnet/m5_baseline/scripts/group_profile.py`; ops attributed through the eager
+module / layer signposts):
+
+| part | OT128 ops | OT128 kernel ms | share | QT128 ms | share | of which (OT128) |
+|---|---:|---:|---:|---:|---:|---|
+| **point island** (encoder ffe0–3, compression, point stem, point fusion 1–4, point fuse, head + softmax) | 215 | **133.66** | **62.9 %** | **101.12** | **75.0 %** | matmul 61.2, fp32 adds 52.6, typecasts (two-term splits) 19.5 |
+| … encoder ffe0–3 (8 → 64 → 128 → 256 → 256) | 36 | 22.14 | 10.4 % | 16.71 | 12.4 % | `ffe3` 9.9 (three 1.6 ms matmuls) |
+| … point stem + point fusion 1–4 (K-split 384 / 256 → 128) | 65 | 37.88 | 17.8 % | 28.36 | 21.0 % | 7.0 per fusion, 10.0 the stem |
+| … point fuse (640 → 256 → 128) | 58 | 47.40 | **22.3 %** | 35.77 | 26.5 % | `point_fuse1` 39.9: 15 matmuls 17.4, 15 adds 16.4, 20 split ops 6.1 |
+| … head (4 × linear + skips, logits, softmax) | 44 | 23.89 | 11.2 % | 17.95 | 13.3 % | |
+| … compression (256 → 16, on `M_cap` rows) | 12 | 2.35 | 1.1 % | 2.33 | 1.7 % | |
+| **K1 segment max** (6 sites) + tilize | 7 | **17.22** | 8.1 % | **15.18** | 11.3 % | encoder site 4.93 (fp32 × 256), 5 × 2.39 (× 128) |
+| pixel2point gathers (6 × untilize + embedding + typecast) | 18 | 6.15 | 2.9 % | 3.50 | 2.6 % | embedding 3.47 |
+| frustum2pixel gathers (6 embeddings) | 6 | 0.65 | 0.3 % | 0.15 | 0.1 % | |
+| decoder resizes (C18, 3 maps) | 39 | 9.03 | 4.2 % | 2.24 | 1.7 % | matmul 2.89, reshape 2.62, (un)tilize 2.30 |
+| image branch: stem, fusion stem, stages 1–4 (convs + glue) | 605 | 30.66 | 14.4 % | 8.63 | 6.4 % | conv 7.62, fp32 HardSwish 5.25, typecast 4.24, DRAM slicing 5.19, adds 3.21, halo 1.96 |
+| image branch: decoder `fuse1` (K-split 5 × 128 → 256) + `fuse2` | 214 | 15.26 | 7.2 % | 3.98 | 3.0 % | conv 4.96, K-split adds 3.62, slicing 3.89 |
+| **total** | **1,104** | **212.64** | | **134.81** | | |
+
+| sink (op type, OT128) | µs | % | where |
+|---|---:|---:|---|
+| Matmul (78) | 64,118 | 30.2 | 72 point-island matmuls at 1.4–7.9 % of the HiFi4 FLOPs and 130–200 GB/s (26–39 % of the DRAM peak): each writes a full fp32 partial `[160000, C_out]` (82–164 MB). The `ffe3` parts take 1.6 ms each, a `point_fuse1` part 1.16 ms, a point-fusion part 0.67 ms. Plus 6 resize matmuls (2.9 ms) |
+| BinaryNg (104) | 59,441 | 28.0 | point island 52.6 ms: 59 fp32 adds (the two-term / K-split partial sums and the head's skips; 1.09 ms each at `[160000, 256]`, about 450 GB/s: DRAM-bound) and the 20 subtracts of the two-term splits. Image branch 6.8 ms: fp32 residual adds and the decoder's K-split sums (3.6 ms) |
+| Typecast (123) | 26,795 | 12.6 | 61 fp32 ↔ bf16 casts of the two-term splits (19.5 ms); 53 casts of the fp32 maps to bf16 before the convs (5.2 ms); 6 pixel2point casts to fp32 (1.5 ms); 3 in the resizes (0.7 ms) |
+| GenericOp: K1 (6) | 16,903 | 7.9 | scalar RISC-V segment max. The encoder site reads 164 MB in 4.93 ms (33 GB/s); each 128-channel site takes 2.39 ms. It scales with `M_cap` (60,000 output rows each) more than with the rows: QT128 runs the same sites in 2.0–4.4 ms on 75 % of the rows |
+| Conv2d (138) | 12,582 | 5.9 | 53 convs in 2–4 DRAM slices each, weighted 59.7 % of the HiFi4 FLOPs |
+| SliceWrite + PaddedSlice (194) | 9,073 | 4.3 | ttnn's DRAM slicing of the 128 × 1024 convs |
+| Unary (46) | 6,269 | 2.9 | the 46 fp32 HardSwish |
+| Embedding (12) | 4,118 | 1.9 | pixel2point (3.47 ms) and frustum2pixel (0.65 ms) gathers |
+| Halo + Move + resharding (358) | 4,816 | 2.3 | conv glue |
+| Reshape + (Un)Tilize + Transpose (39) | 7,340 | 3.5 | resize chains, pixel2point untilize, the input tilize |
+| Ternary `addcmul` (4) | 852 | 0.4 | attention gating |
+
+Top 10 device ops of the OT128 replay:
+
+| # | op | where | µs | cores |
+|---:|---|---|---:|---:|
+| 1 | K1 `segment_reduce` | `segmax.encoder`: `[160000, 256]` fp32 → `[60000, 256]` fp32 (voxel max) | 4,928 | 120 |
+| 2–6 | K1 `segment_reduce` | `segmax.points` sites 1–5: `[160000, 128]` → the bf16 gather table `[60001, 128]` | 2,394–2,396 each | 120 |
+| 7–9 | Matmul | `encoder.ffe3` (256 → 256), the three two-term products `[160000, 256] × [256, 256]` → fp32 | 1,641 / 1,599 / 1,598 | 120 |
+| 10 | Matmul | `head.mlp2` part `[160000, 256] × [256, 128]` → fp32 | 1,221 | 120 |
+
+Next come the 15 `point_fuse1` matmuls (1.15–1.20 ms each), the 25 fp32 adds at `[160000, 256]` (1.08–1.11
+ms each) and the `point_stem` part matmuls (1.19 ms).
+
+**Most expensive layers:**
+- `decoder.point_fuse1`: 39.85 ms (18.7 % of the replay), of which 17.41 ms matmul;
+- `decoder.fuse1`: 12.55 ms in 157 programs, of which 3.92 ms conv;
+- K1 `segmax.points` (5 sites): 11.98 ms;
+- `backbone.point_stem`: 9.96 ms;
+- `encoder.ffe3`: 9.88 ms;
+- `head.mlp1` / `head.mlp2`: 8.23 / 8.09 ms;
+- each point fusion: 6.97–6.99 ms;
+- stage-1 convs: 1.43–1.90 ms each, of which 0.37 ms compute (the rest is the DRAM slicing, halo and the fp32
+  output).
+
+`logs/frnet/m5_baseline/profile_summary_{ot128,qt128}.{json,md}` have the per-module, per-layer and per-op tables,
+`tt_perf_report_{ot128,qt128}.txt` the tt-perf-report output (with its per-op advice), and
+`grp_{ot128,qt128}.txt` the part breakdown above.
+
+**The precision cost of the policy.** These are the port's same-session device A/Bs on the OT128 frames (PORT_LOG
+M1–M3, jobs j2–j5, gates read-only). They were not re-run for this baseline.
+
+| policy | replay ms | Δ vs shipped | OT128 frames min argmax (nuScenes 0916 kf40) | gate |
+|---|---:|---:|---|---|
+| fp32 × fp32 point island (`POINT_TERMS=1`), bf16 maps (j2) | 107 | −105 ms (−50 %) | **97.50 %** | fails |
+| two-term point island, bf16 maps (j3) | 201 | −11 ms (−5 %) | **98.28 %** | fails |
+| two-term point island, bf16 maps, `CONV_TERMS=2` (j4) | 244 | +32 ms | **98.35 %** | fails |
+| **two-term point island, fp32 maps (shipped; j5 / this baseline)** | **212** | – | **98.91 %** | passes, 0.41 pt margin |
+| two-term point island, fp32 maps fed to the convs directly (j5) | 217 | +5 ms | **98.13 %** | fails |
+| two-term point island, fp32 maps, `CONV_TERMS=2` (j5) | 249 | +37 ms | 99.00 % | passes, 0.50 pt margin |
+
+- **The two-term point island costs about 94 ms per OT128 frame** (j2 → j3: 107 → 201 ms, +88 %). That is 44 % of
+  today's replay, and it is how the port reaches the gate: the device's fp32 matmul is TF32-like (P12). The
+  profile shows that the cost is not the extra products (bf16 HiFi4 is exact) but the fp32 partial tensors, their
+  DRAM adds and the split typecasts. Item 1 below targets exactly that, with the same products.
+- **The fp32 maps cost about 11 ms** (201 → 212 ms): fp32 HardSwish, residual adds and `addcmul`, and a cast to bf16
+  before every conv. They buy 0.63 pt on the binding frame.
+- **Cheaper precision fails.** Every cheaper policy measured fails the binding nuScenes frame. HiFi2 / LoFi or
+  bfp8 were not measured: with a 0.41 pt margin and 98.28 % for bf16 maps, they need fresh device evidence, and the
+  FPU is not the bottleneck (the matmuls run at 1.4–7.9 % of the FLOPs).
+- **More margin costs time.** `CONV_TERMS=2` raises the margin to 0.50 pt for +37 ms. fp32 resize outputs (blocked
+  by C18 at 1024 × 128 rows) and fp32 gathers for the 128-channel maps are unmeasured options.
+
+## Ranked optimization opportunities
+
+How to read the estimates:
+- Gains are on the OT128 trace (211.6 ms, PandaSet) unless marked e2e (OT128 PandaSet e2e 355.7 ms p50). QT128
+  follows the same ranking (the point island is 75 % of its 134.9 ms).
+- Each estimate is the measured kernel or host time of the work the item removes, with a range for what replaces
+  it. Items overlap, so the gains do not add up.
+- Every item keeps the frozen gates and the precision policy (PLAN §9.3), and lands as one commit with one
+  `FRNET_*` A/B knob (PLAN §5.1).
+- **Judge numerics changes on the binding frame first: nuScenes 0916 kf40, OT128, 98.907 % against the 98.5 % gate.**
+  A change in the order of fp32 accumulation is not bit-identical to today, so items 1–2 must re-run the full
+  device suite.
+
+1. **Two-term point linears as one K-concatenated matmul per layer (trace −60..−80 ms, −28..−38 %; the largest
+   item).**
+   - Today each point linear writes one fp32 `[N_cap, C_out]` partial per product: 2–3 per input part, up to 15 for
+     `point_fuse1`. One fp32 add per partial then sums them. Matmuls take 61.2 ms and the adds 52.6 ms, at
+     130–450 GB/s: DRAM traffic, not FLOPs (1.4–7.9 % of HiFi4).
+   - The fix: one matmul per layer over the K-concatenated bf16 operands `[x_hi | x_hi | x_lo]` (all parts) ×
+     `[W_hi; W_lo; W_hi]`, with fp32 dest accumulation and packer L1 accumulation.
+     - The products are the same exact bf16 products. Only the fp32 summation order changes, so the gates must be
+       re-run.
+     - The fp32 output is written once. The bf16 inputs are a quarter of an fp32 partial's bytes.
+   - The concat of the bf16 operands costs about 0.3–1 ms per layer, unless the split kernel (item 3) writes them
+     straight into the K-slices of one tensor.
+   - Expected result: matmul about 61 → 25–35 ms, adds 52.6 → about 5 ms (the residual / skip adds stay).
+2. **MK-PointMLP: a row-block-streamed point MLP megakernel (trace −100..−115 ms beyond today, overlapping 1;
+   D19).**
+   - The point island is 133.7 ms (OT128) and 101.1 ms (QT128): 63–75 % of the replay. Its work is row-local
+     except the gathers and K1.
+   - A kernel per row block (for example 32–64 points per core) chains the linears of one segment in L1:
+     - the encoder `ffe0..ffe3`;
+     - point fuse `640 → 256 → 128`;
+     - the head's 4 layers + skips + logits + softmax.
+
+     It does the two-term splits on the fly, holds the bf16 weight terms in L1 (at most 655 KB for `point_fuse1`)
+     and writes only the segment's outputs. The head segment writes `pred_probs` directly.
+   - Its DRAM floor is a few tens of MB per frame, about 0.1–0.3 ms. Compute at the measured bf16 rates fits in
+     roughly 15–30 ms for the whole island.
+   - Stepping stones:
+     - item 1;
+     - then fuse each linear's split into the producer's epilogue (item 3);
+     - then one segment at a time, the head first (23.9 ms, no gather inside).
+3. **A fused two-term split (trace −10..−14 ms; on the way to 1–2).**
+   - Each fp32 input today runs 4 programs: typecast, typecast back, subtract, typecast. That is 19.5 ms for 61
+     casts, plus 20 subtracts (in the BinaryNg row), at `N_cap`.
+   - One `generic_op` (or SFPU eltwise) pass can read fp32 and write hi and lo bf16, about 0.3–0.5 ms per
+     `[160000, 256]` input. It can write into the K-slices of item 1's operand. Exact: the same RNE casts.
+4. **N_cap / M_cap buckets (trace −100..−160 ms on frames with N ≤ 32–64 k; e.g. every nuScenes frame).**
+   - Every point-wise op runs on all 160,000 rows (OT128) or 120,000 (QT128), and K1 writes all 60,000
+     segments. That covers the point island, K1, the gathers and the readback: about 157 ms of the OT128 replay.
+   - The catalogued nuScenes frames have N = 20–28 k (13–17 % of `N_cap`) and M = 17–25 k. The QT128 PandaSet
+     frame has M = 6,255 against `M_cap` = 60,000.
+   - The fix: two or three trace variants per profile (for example `N_cap` 32 k / 64 k / 160 k, `M_cap` 16 k /
+     60 k), sharing weights. The host picks the smallest bucket that covers the frame. Each variant costs about
+     6–9 MiB of trace region (256 MiB available) plus its activations.
+   - PandaSet-like frames (N = 146 k) gain nothing. The readback shrinks with N too (item 9).
+5. **K1 throughput (trace −10..−14 ms; owned shared kernel).**
+   - K1 takes 16.9 ms: 33 GB/s on the 164 MB encoder site, and 2.0–2.4 ms per 128-channel site almost
+     independent of the row count. So per-segment output overhead dominates: 60,000 segments of one row each
+     through an 8-row staging ring.
+   - Fixes, in order:
+     - batch output rows per worker (contiguous segments → one NOC write per block);
+     - move the accumulator from L1 into the RISC-V's local memory (PORT_LOG known issue);
+     - with item 4, write only M_cap-bucket segments;
+     - a compute-engine reduction is the long form.
+   - Bit-exactness must stay (max of integer keys). The shared-kernel rule (PLAN §5.1.5) applies: re-vendor and
+     re-run K1's host / device tests.
+6. **Image branch DRAM slicing and glue (trace −8..−14 ms; MK-ImageStage).**
+   - The 128 × 1024 convs spend 12.6 ms in conv kernels but 9.1 ms in DRAM slicing (padded slice + slice write) and
+     4.8 ms in halo / move / resharding. The fp32-map glue adds the following:
+     - 46 fp32 HardSwish: 6.3 ms;
+     - the casts to bf16 before each conv: 5.2 ms;
+     - fp32 residual adds and K-split sums: 6.8 ms.
+   - A 128 × 1024 × 128 map is 33.5 MB in bf16 / 67 MB in fp32 against 167.5 MB of L1. So the stride-1 stage chain
+     can stay height-sharded in L1 (C17 `slicing="l1"`), which drops the slicing.
+   - Fold the bf16 cast of the conv input and the HardSwish into one eltwise pass (exact: the same ops).
+   - The longer form is a BasicBlock megakernel with halo exchange (MK-ImageStage).
+7. **Decoder resizes and `fuse1` (trace −5..−8 ms).**
+   - Resizes take 9.0 ms: reshape 2.6, tilize / untilize 2.3, transpose 0.4 and matmul 2.9.
+   - `decoder.fuse1` takes 12.5 ms in 157 programs: a 5-part K-split over five 128 × 1024 maps, with 3.6 ms of
+     K-split adds and 3.9 ms of slicing.
+   - Fix: run the resize matmuls in TILE layout end to end, so no reshape / (un)tilize is needed. That also lifts
+     the C18 fp32 limitation, a precision-margin option. Accumulate the K-split parts in the conv (dest / L1
+     accumulation) instead of fp32 adds.
+8. **pixel2point gathers (trace −2..−4 ms).**
+   - Today: untilize of each map (1.2 ms), the bf16 embedding (3.5 ms), a typecast to fp32 (1.5 ms).
+   - The two-term linear already treats a bf16 input as exact (`lo = 0`). So feed the gather output as bf16 straight
+     into item 1's operand (no cast). Exact.
+   - Later: gather from the TILE map in the MK-PointMLP reader.
+9. **Host preprocess (e2e −60..−85 ms OT128).**
+   - Today 107 ms on PandaSet:
+     - plan 41.8 ms: sort, CSR, gather tables, features;
+     - fp16 interpolation 37.4 ms;
+     - decode 12.9 ms;
+     - projection 8.0 ms;
+     - crop and unique cells 5.6 ms.
+   - The fixes:
+     - a C++ / numba implementation of the node's CUDA kernels (projection + interpolation grid in one pass,
+       counting-sort plan): about 10–20 ms in all;
+     - keep the decode outside the timed request path.
+   - The numerics must stay bit-identical to today's host code (float32 / fp16 policies, the documented race
+     policies). The host tests and the research goldens check that bit for bit.
+10. **Host after the device (e2e −15..−25 ms OT128).**
+    - Today: unsort 12.8 ms + post 14.8 ms + host untilize 5.0 ms.
+    - Fixes: read back row-major (device untilize, or a ROW_MAJOR output), with only N rows (item 4). Fuse the
+      un-permute into the argmax pass: take the argmax over the 27 columns on the permuted rows, then scatter
+      int8 labels and float scores instead of the 27-column float rows.
+    - Exact: the same first-max rule on the same fp32 values.
+11. **H2D `features` layout (e2e −3.5 ms).** `features` moves 5.1 MB at 1.24 GB/s in 160,000 pages of 32 bytes. Upload
+    it as `[N_cap / 4, 32]` rows (128 B pages) and view or reshape on the device. A TILE upload would also drop the
+    input tilize (0.36 ms).
+12. **Throughput by pipelining (frames/s; latency unchanged).**
+    - Device b2b allows 4.72 frames/s (OT128) and 7.41 (QT128). A synchronous request gives 2.81 / 4.09 frames/s on
+      PandaSet.
+    - Overlap the host pre / post of frames k ± 1 with the device work of frame k in the server.
+    - 2CQ staging can hide the 5.4 ms upload (and, with double-buffered outputs, the 7.4 ms readback). It is worth
+      measuring only after items 1–2 shrink the replay.
+13. **Megakernel candidates (D19, PLAN §5.2 / §5.3).**
+    - The §5.2 gate is met through DRAM round trips and layout glue, not dispatch. 64 % of the replay is not conv /
+      matmul compute, while op-to-op gaps are 0.3 %.
+    - Candidates, by measured block time:
+      - **MK-PointMLP** (point island, 133.7 ms OT128 / 101.1 ms QT128; items 1–3, 8): first, by far. It is
+        row-local, and its weights fit L1;
+      - **MK-SegGather** (K1 + the frustum2pixel / pixel2point gathers, 24 ms; items 5, 8): fold the gather tables
+        into K1's writer and the pixel2point gather into the point reader;
+      - **MK-ImageStage** (stem + stages 1–4 + decoder convs, 46 ms, of which 12.6 ms conv compute; items 6–7):
+        band-fused BasicBlocks with halo exchange. The 128 × 1024 maps fit L1 in bf16.
+    - A single whole-model kernel is not practical: the fp32 point tensors at `N_cap` (82–164 MB each) do not fit L1.
+      MK-PointMLP's row-block streaming is the practical "single" form for 63–75 % of the work.
+14. **Precision margin (accuracy, not speed; optimization-phase precision item).**
+    - The margin on nuScenes 0916 kf40 is 0.41 pt. Items 1–2 change the fp32 accumulation order, so re-check the
+      margin after each step.
+    - Options to buy margin, measured: `CONV_TERMS=2` +0.09 pt for +37 ms. Unmeasured: fp32 resize outputs (after
+      item 7) and fp32 gathers of the 128-channel maps (`ttnn.embedding` is bf16-only; a K1-style generic_op gather).
+    - No precision relaxation (bf16 maps, a TF32 point island, HiFi2) is a candidate without new device evidence on
+      that frame: each one measured so far fails it.
+
+## Raw logs and files
+
+All under `/home/ubuntu/experiments/tt-models/`:
+- `logs/frnet/m5_baseline/job1.log` (+ `job1.rcs`): the device job (steps 2–5). `try0/`: the first attempt (suite log
+  `suite_r1.log`; the other steps failed on the weights path). `job1_prereboot.log`: the job queued before the server
+  reboot, which never started;
+- `logs/frnet/m5_baseline/bench_eth1cq_{ot128,qt128}.{json,log}`: the stage benches;
+- `logs/frnet/m5_baseline/matrix_{ot128,qt128}_{eth-1cq,eth-2cq,worker-2cq,worker-1cq,eth-1cq-repeat}.{json,log}`;
+- `logs/frnet/m5_baseline/profile_{ot128,qt128}.log`, `profile_summary_{ot128,qt128}.{json,md}`,
+  `tt_perf_report_{ot128,qt128}.txt`, `grp_{ot128,qt128}.txt`;
+- `logs/frnet/m5_baseline/visualizer.log`, `host_suite.log`; `logs/frnet/{pcc,pcc_small,e2e,latency}_{ot128,qt128}_m5.json`;
+- `logs/frnet/m5_baseline/inputs/`: the synthetic scans, their calibrations and CPU references;
+- device profiles: `generated/profiler/frnet_baseline/reports/2026_10_09_13_56_53/ops_perf_results_2026_10_09_13_56_53.csv`
+  (OT128) and `generated/profiler/frnet_baseline_qt128/reports/2026_10_09_13_59_21/ops_perf_results_2026_10_09_13_59_21.csv`
+  (QT128);
+- ttnn-visualizer report: `generated/ttnn_visualizer/frnet_baseline/` (`db.sqlite` 26 MB, `config.json`; OT128 PandaSet,
+  one eager forward. The 7.5 GB graph JSON went to `/dev/shm` and was deleted after the import).

OPT_REPORT.md

@@ -0,0 +1,157 @@
+# frnet-p150 optimization report (p150, ETH dispatch, 12×10 grid)
+
+**Status: baseline port; optimization pending.** This is the first public release: the functional port, measured
+once (`OPT_BASELINE.md`, baseline commit `0dd52b2`, 2026-10-09) and not optimized yet. No optimization round has run,
+so there is no step, no rejected attempt and no hang to report, and every number below is the baseline. The device
+graph runs as one metal trace per serve profile: 1,104 programs per frame for `ot128` (Autoware's default sensor
+model) and 759 for `qt128`, 211.64 / 134.87 ms per back-to-back replay on a PandaSet frame (4.72 / 7.41 frames/s of
+device throughput). A synchronous `model()` call takes 292.6 ms (ot128) / 221.1 ms (qt128) on the shipped synthetic
+sample and 355.7 / 244.3 ms on the PandaSet frame; the device is 60 % (ot128) / 55 % (qt128) of it. The ranked
+backlog at the end says where the time goes. The largest item is the precision island of the point-wise layers
+(63 % of the ot128 replay, 75 % of qt128), whose cost is DRAM traffic of fp32 partial sums, not compute; then the
+image-branch convs and glue (46 ms) and the custom K1 segment-max kernel (16.9 ms). The binding accuracy constraint
+of every step is the nuScenes 0916 key-frame 40 frame on ot128: 98.907 % argmax agreement against the 98.5 % gate,
+a 0.41-point margin.
+
+All numbers: `code/scripts/bench.py` (median of 100 warm iterations, batch 1, the shipped sample
+`code/tt_frnet/samples/synthetic_<variant>.npz` and the staged PandaSet 019 frame 40), ETH dispatch, 1 CQ, 12×10 grid,
+the serve.env precision knobs (`FRNET_POINT_TERMS=2`: two-term bf16 operands with fp32 accumulation in the point-wise
+layers; `FRNET_MAP_DTYPE=float32`: fp32 image-branch maps with bf16 conv operands; `FRNET_CONV_TERMS=1`), HiFi4 with
+fp32 accumulation everywhere, AICLK 1350 MHz, unless marked otherwise. Device profile: `code/scripts/profile_ops.py`
+under the device profiler (one traced replay between signposts). Accuracy gates: `OPT_BASELINE.md` "How to run".
+
+## Summary
+
+| | baseline `0dd52b2` (2026-10-09) | **final (= baseline: no round yet)** |
+|---|---|---|
+| device trace, one blocking forward (ot128 / qt128; shipped sample) | 210.02 / 133.53 ms | same |
+| back-to-back traces (ot128 / qt128; PandaSet 019 f40) | 211.64 / 134.87 ms (4.72 / 7.41 frames/s) | same |
+| e2e `model()` p50 (p99), shipped sample: ot128 / qt128 | 292.6 (299.4) / 221.1 (233.5) ms | same |
+| e2e `model()` p50 (p99), PandaSet 019 f40 (107,146 points): ot128 / qt128 | 355.7 (367.0) / 244.3 (258.7) ms | same |
+| host preprocess / H2D / D2H / un-permute / postprocess, ot128, PandaSet | 107.2 / 5.41 / 7.43 / 12.8 / 14.8 ms | same |
+| device programs per frame (distinct programs in the trace): ot128 / qt128 | 1,104 (227) / 759 (174) | same |
+| kernel sum / span / op-to-op gaps, ot128 | 212.64 / 213.27 / 0.63 ms | same |
+| kernel sum / gaps, qt128 (span unusable: a profiler artefact, see `OPT_BASELINE.md`) | 134.81 / 0.44 ms | same |
+| accuracy gates (vs the fp32 CPU reference / ONNX Runtime) | every frozen gate passes: module PCC min 0.999729 / 0.999649; public frames min `pred_probs` PCC 0.99969 / 0.99980, min argmax agreement **98.907 %** (nuScenes 0916 kf40, gate 98.5 %) / 99.158 % | same |
+| `from_pretrained` load, warm JIT cache | 6.5 / 6.9 s (capture 2.6 / 2.0 s) | same |
+
+Host-side rows move with the load of the shared host (load average 4.1–6.6 during the baseline job); the device rows
+repeat to 0.01 ms. The release re-check (`VERIFICATION_2026-10-09.md`) reproduced the device rows on the release
+commit.
+
+## Steps (chronological; each row is one commit)
+
+| step | commit | what | trace / b2b ms (ot128; qt128) | e2e ms (shipped sample: ot128; qt128) | accuracy | revert switch |
+|---|---|---|---|---|---|---|
+| baseline | `0dd52b2` | first correct port: one trace per profile, ETH 1CQ 12×10; two-term bf16 point layers, fp32 image-branch maps, K1 `segment_reduce` (bit-exact); host pre-processing, argmax and filter | 210.02 / 209.95; 133.53 / 133.43 | 292.6; 221.1 | all frozen gates (above) | – |
+
+The release commit after the baseline changed no model numerics and no device code: the shipped sample became a
+synthetic scan per serve profile (generated by `code/scripts/make_sample.py`, byte-identical to the baseline's
+synthetic bench inputs), the PandaSet sample moved out of the shipped `code/` (staged until PandaSet samples are
+approved), test hardening (`test_public_frames` fails on a missing golden and pins the frame counts; a new
+`test_shipped_sample` gate), docs and demo media. The device suite was re-run on it with identical values for every
+existing gate (`VERIFICATION_2026-10-09.md`).
+
+## Round 1
+
+Not started. The optimization phase follows the publication of every model of the collection (`research/PLAN.md`
+§4.1, §5): each step is one commit with one `FRNET_*` A/B knob, keeps the frozen gates, and the precision policy
+(PLAN.md §9.3) may be relaxed per module only with gate evidence. **Every numerics change is judged on the binding
+frame first: nuScenes 0916 kf40, ot128, 98.907 % against the 98.5 % gate.** A change of the fp32 accumulation order
+is not bit-identical to today, so it re-runs the full device suite.
+
+### Findings (measured)
+
+The baseline profile (`OPT_BASELINE.md` "Device profile") is the starting point:
+
+- The trace is **kernel-bound, not dispatch-bound**: op-to-op gaps total 0.63 ms of a 213.27 ms span (ot128, 0.3 %).
+  Cutting programs pays only through the kernel time of the programs removed; a megakernel's dispatch saving is at
+  most 0.6 ms.
+- **The point-wise layers kept at higher precision are 63 % of ot128 (133.7 ms) and 75 % of qt128 (101.1 ms).** Each
+  linear splits its fp32 input into two bf16 terms and runs 2–3 bf16 matmuls whose fp32 partials are summed in fp32.
+  The cost is DRAM traffic: 61.2 ms of matmuls at 1.4–7.9 % of the HiFi4 FLOPs, 52.6 ms of fp32 adds of the
+  partials, 19.5 ms of typecasts for the splits. `decoder.point_fuse1` alone is 39.9 ms.
+- **K1 `segment_reduce`: 16.9 ms** (8 % of ot128, 11 % of qt128): 2.39 ms per 128-channel site almost independent of
+  the row count, so per-segment output overhead dominates.
+- **Image-branch convs and glue: 46 ms on ot128**, of which 12.6 ms conv kernels; the rest is ttnn's DRAM slicing of
+  the 128 × 1024 convs (9.1 ms), halo / move / resharding (4.8 ms), 46 fp32 HardSwish (6.3 ms), bf16 casts before
+  every conv (5.2 ms), fp32 residual and K-split adds (6.8 ms), resizes (9.0 ms).
+- Every point-wise op runs on the profile's `N_cap` rows (160,000 / 120,000) and K1 writes all 60,000 segments,
+  whatever the frame: the nuScenes frames (N = 20–28 k) cost the same as PandaSet (N = 146 k).
+- L1 holds nothing between ops; DRAM use peaks at 218.6 MB per bank at the first partial-sum add of
+  `decoder.point_fuse1`.
+- Dispatch / CQ matrix (back-to-back): ot128 ETH-1CQ 209.95, ETH-2CQ 211.32, WORKER 211.56 ms; qt128 133.44, 134.25,
+  153.71 ms. ETH and 1 CQ stay pinned (D14): on qt128's narrow grid WORKER's 11×10 partitioning costs 20.3 ms.
+- The precision policy is how the port reaches the gate. Measured in the port (PORT_LOG M1–M3): an fp32 × fp32 point
+  island (the device's TF32-like matmul) replays in 107 ms but gives 97.50 % on the binding frame; bf16 image maps
+  98.28 %; `CONV_TERMS=2` buys +0.09 pt for +37 ms.
+
+### Megakernel / fusion work
+
+None yet. The candidates by measured block time are listed in the backlog (item 13).
+
+## Known hangs (all rounds)
+
+| when (UTC) | command | cause | status |
+|---|---|---|---|
+| – | – | none: no hang, device timeout, reset or FAULT marker in any device job of the port, its verification round, the baseline or the release runs (one build-time `TT_THROW` of the shared C18 resize at fp32 1024 × 128 rows, an L1 overflow reported at program build, is a known issue in `PORT_LOG.md`, not a hang) | – |
+
+## Rejected / not kept
+
+None yet in the optimization phase. Configurations measured during the port that are **not** the default, for
+accuracy (`PORT_LOG.md` M1–M3; `OPT_BASELINE.md` "The precision cost of the policy"; ot128, binding frame nuScenes
+0916 kf40):
+
+| configuration | replay | binding frame | why not |
+|---|---:|---:|---|
+| fp32 × fp32 point island (`POINT_TERMS=1`), bf16 maps | 107 ms | 97.50 % | fails the 98.5 % gate |
+| two-term point island, bf16 maps (`MAP_DTYPE=bfloat16`) | 201 ms | 98.28 % | fails |
+| two-term point island, bf16 maps, `CONV_TERMS=2` | 244 ms | 98.35 % | fails, slower |
+| two-term point island, fp32 maps fed to the convs directly (`MAP_DTYPE=float32_direct`) | 217 ms | 98.13 % | fails |
+| two-term point island, fp32 maps, `CONV_TERMS=2` | 249 ms | 99.00 % | passes, +37 ms for +0.09 pt of margin |
+
+## Profile at the end (trace replay)
+
+The baseline profile (`OPT_BASELINE.md`): ot128 1,104 programs, kernel sum 212.64 ms, span 213.27 ms; qt128 759
+programs, 134.81 ms. Top sinks by op type (ot128): Matmul 64.1 ms (30 %), BinaryNg 59.4 ms (28 %), Typecast 26.8 ms
+(13 %), K1 16.9 ms (8 %), Conv2d 12.6 ms (6 %), DRAM slicing 9.1 ms, HardSwish 6.3 ms, resize chains 7.3 ms.
+
+## Remaining backlog (gains on the ot128 trace of 211.6 ms, PandaSet, unless marked e2e; estimates, not measurements)
+
+Each gain is the measured kernel or host time of the work an item removes, with a range for what replaces it. Items
+overlap, so the gains do not add up. qt128 follows the same ranking. From `OPT_BASELINE.md` "Ranked optimization
+opportunities":
+
+1. **Two-term point linears as one K-concatenated matmul per layer (trace −60..−80 ms).** One matmul over
+   `[x_hi | x_hi | x_lo] × [W_hi; W_lo; W_hi]` with fp32 dest and packer L1 accumulation replaces 2–3 matmuls per part
+   and their fp32 partial adds. Same exact bf16 products; only the fp32 summation order changes (re-run the gates).
+2. **MK-PointMLP, a row-block-streamed point MLP megakernel (trace −100..−115 ms beyond today, overlapping 1; D19).**
+   The point layers are row-local except the gathers and K1; their bf16 weight terms fit L1 (at most 655 KB). Chain
+   the encoder, the point fuse and the head per row block in L1, with the two-term split on the fly.
+3. **A fused two-term split (trace −10..−14 ms; on the way to 1–2).** One pass reads fp32 and writes the hi / lo bf16
+   terms (exact: the same RNE casts) instead of 4 programs per input.
+4. **`N_cap` / `M_cap` buckets (trace −100..−160 ms on frames with N ≤ 32–64 k, e.g. every nuScenes frame).** Two or
+   three trace variants per profile sharing weights; the host picks the smallest that covers the frame. PandaSet-like
+   frames (N = 146 k) gain nothing.
+5. **K1 throughput (trace −10..−14 ms; owned shared kernel).** Batch output rows per worker, accumulator in the
+   RISC-V local memory, then a compute-engine reduction; bit-exactness stays.
+6. **Image-branch DRAM slicing and glue (trace −8..−14 ms; MK-ImageStage).** Keep the stride-1 stage chain
+   height-sharded in L1 (a 128 × 1024 × 128 map is 33.5 MB in bf16), fold the bf16 cast and the HardSwish into one pass.
+7. **Decoder resizes and `fuse1` (trace −5..−8 ms).** Resize matmuls in TILE layout end to end (also lifts the C18 fp32
+   limitation), K-split accumulation inside the conv.
+8. **pixel2point gathers (trace −2..−4 ms).** Feed the bf16 gather output straight into item 1's operand (exact).
+9. **Host pre-processing (e2e −60..−85 ms).** 107 ms on PandaSet: a C++ / numba implementation of the node's CUDA
+   kernels (projection + interpolation grid in one pass, counting-sort plan), bit-identical to today's host code.
+10. **Host after the device (e2e −15..−25 ms).** Row-major readback of only N rows; argmax over the permuted rows, then
+    scatter labels and scores instead of 27-column float rows (exact: the same first-max rule).
+11. **H2D `features` layout (e2e −3.5 ms).** 160,000 pages of 32 bytes at 1.24 GB/s: upload as 128-byte rows.
+12. **Throughput by pipelining (frames/s; latency unchanged).** Overlap the host stages of neighbouring frames with the
+    device work; 2 CQ staging once items 1–2 shrink the replay.
+13. **Megakernel candidates (D19, PLAN §5.2 / §5.3)** by measured block time: MK-PointMLP (133.7 / 101.1 ms; first),
+    MK-SegGather (K1 + gathers, 24 ms), MK-ImageStage (46 ms). A whole-model kernel is not practical: the fp32 point
+    tensors at `N_cap` (82–164 MB each) do not fit L1.
+14. **Precision margin (accuracy, an optimization-phase precision item).** The 0.41-point margin on nuScenes 0916 kf40
+    (ot128) is the narrowest of the release. Items 1–2 change the accumulation order, so re-check it after each step.
+    Measured option: `CONV_TERMS=2` (+0.09 pt, +37 ms). Unmeasured: fp32 resize outputs (after item 7), fp32 gathers of
+    the 128-channel maps. No relaxation (bf16 maps, a TF32 point island, HiFi2, bfp8) is a candidate without new device
+    evidence on that frame.

PYTHON.md

@@ -0,0 +1,220 @@
+# Python API: FRNet (Autoware lidar_frnet) on Blackhole
+
+Use this API from Python code (a pipeline, a notebook, a ROS 2 node wrapper). You do not need the HTTP server: the
+API and the server share the decoders, the device graph and the post-processing, so the outputs and the speed are
+the same.
+
+## Install
+
+Install the package on top of an environment that already has `ttnn` (a tt-metal `python_env` at `44d66500520`
+with `patches/tt-metal-eth-dispatch.patch`, or the tt-model container). From the root of the model repository (the
+directory that holds `pyproject.toml`, `README.md` and `code/`):
+
+```bash
+pip install -e .                        # the Python API
+pip install -e ".[server,test]"         # + the HTTP server and the tests
+```
+
+The pip project is the repository's top-level `pyproject.toml`; it installs the package from `code/tt_frnet` (there is no
+`pyproject.toml` inside `code/`, because the container build copies `code/` over the tt-metal tree).
+
+The package carries `tt_frnet.ttaw`, the shared code of the Autoware ports to Blackhole (device open, trace runner,
+decoders, model base class, HTTP app, the K1 `segment_reduce` kernel), vendored at the version recorded in
+`code/tt_frnet/ttaw/VENDORED.json` (0.20.0).
+
+| You want to run | Extras |
+|---|---|
+| the Python API | none |
+| the HTTP server (`tt_frnet.server.app`, see `SERVING.md`) | `server` |
+| host tests (no device; device tests are skipped): `TT_VISIBLE_DEVICES=none python -m pytest -q code/tt_frnet/tests` | `server,test` |
+| device tests: `python -m pytest -q -s code/tt_frnet/tests/test_pcc_device.py code/tt_frnet/tests/test_e2e_device.py` | `test` |
+
+## Quickstart
+
+Run from the repository root (the sample paths are relative to it):
+
+```python
+from tt_frnet import FRNet
+
+with FRNet.from_pretrained(device_id=0) as model:                  # variant="qt128" for the Hesai QT128 profile
+    out = model("code/tt_frnet/samples/synthetic_ot128.npz",       # one sweep in the sensor frame
+                calibration={"preset": "synthetic_ot128"})         # its sensor -> base_link transform (ego crop box)
+
+print(out.class_counts())                                          # points per class (raw + interpolated)
+labels, scores, xyz = out.label_ids, out.scores, out.points[:, :3]
+```
+
+`examples/quickstart.py` runs this snippet and writes `quickstart.json` (the `/predict` body) and `quickstart_bev.png`
+(a bird's-eye view of the p150 classes next to the stored CPU reference of the shipped sample).
+
+## `FRNet.from_pretrained(...)`
+
+```python
+FRNet.from_pretrained(
+    model_id=None,          # HF repo or a local directory with the weights files; default AutowareFoundation/lidar_frnet
+    *,
+    revision=None,          # default for the default repo: the validated commit a53a1c11b0b (tag v2.0)
+    variant=None,           # load-time variant: "ot128" (Hesai OT128, Autoware's default) | "qt128"; default $FRNET_VARIANT or "ot128"
+    device_id=None,         # chip to open; default $TT_DEVICE_ID or 0
+    device=None,            # an already-opened ttnn device (tt_frnet.device.open_device); close() does not close it
+    dispatch=None,          # "eth" (p150 target, 12x10 grid) | "worker" (A/B only, 11x10) | "auto"; default $FRNET_DISPATCH or "eth"
+    num_command_queues=None,  # default $FRNET_NUM_CQS or 1
+    weights_dir=None,       # explicit local weights directory; no Hub access
+    warmup_variants="default",  # trace variants to capture now; see "Warm-up"
+    verbose=False,
+) -> FRNet
+```
+
+What it does: resolves the weights first, so a Hub problem never claims the chip (`weights_dir` >
+`$FRNET_WEIGHTS_DIR` > a local `model_id` directory > the HF snapshot at the pinned revision, restricted to
+`frnet_*.onnx, ml_package_frnet_*.param.yaml, deploy_metadata.yaml`, with an offline fallback to the cache), checks
+the sha256 of the variant's ONNX against the pinned one, reads the variant's `ml_package_frnet_<variant>.param.yaml`
+(FOV, frustum and interpolation grids, the 27 class names, the palette, the point / cell profiles), opens the chip
+(ETH dispatch, 12×10, 1 CQ, `l1_small_size` 64 KiB, `trace_region_size` 256 MiB: `DEVICE_DEFAULTS` in
+`tt_frnet/device.py`, overridable with `FRNET_*`), builds the graph, then compiles and captures the metal trace. If
+ETH dispatch cannot open (tt-metal without the patch), it warns and falls back to WORKER dispatch.
+
+`variant` picks one of the two networks of the weights repo. Each is tied to its sensor (Autoware's `sensor_model`):
+
+| variant | sensor | FOV up / down | frustum grid (H × W) | interpolation grid | points (min / max) | unique cells (min / max) |
+|---|---|---|---|---|---|---|
+| `ot128` (default) | Hesai OT128 | +15° / −25° | 128 × 1024 | 128 × 4096 | 5,000 / 160,000 | 3,000 / 60,000 |
+| `qt128` | Hesai QT128 | +52.6° / −52.6° | 128 × 256 | 128 × 1024 | 5,000 / 120,000 | 3,000 / 60,000 |
+
+Precision knobs (read once at load from the environment, pinned in the package's `serve.env`; the defaults are the
+validated configuration and every number of the model card uses them): `FRNET_POINT_TERMS=2` (each point-wise linear
+as two-term bf16 operands, about 16 mantissa bits, fp32 accumulation), `FRNET_MAP_DTYPE=float32` (fp32 image-branch
+maps, bf16 conv operands), `FRNET_CONV_TERMS=1`. Other values are A/B switches for the optimization phase; they fail
+the frozen accuracy gates (`OPT_BASELINE.md`, "The precision cost of the policy").
+
+## Warm-up
+
+`from_pretrained` returns a warm model: it builds the graph of the variant, runs it once eagerly on an empty frame
+(this first run compiles the kernels and uploads the weights) and captures it as ONE metal trace, so no call compiles
+anything. There are no shape buckets: every frame is padded to the profile's fixed device shapes (`N_cap` points =
+160,000 / 120,000, `M_cap` cells = 60,000), so one trace serves every input of the profile.
+
+- Measured (2026-10-09, the shipped sample): with an empty JIT cache the load took 156.6 s for ot128 and
+  151.9 s for qt128 (firmware and every kernel compiled); with that cache warm, in a new process,
+  7.5 s and 6.6 s. The trace is captured during the load, so the first call costs about as
+  much as the next ones (320.0 / 304.7 ms for ot128, 241.1 / 214.9 ms for qt128, first / second call, on a shared host).
+- `warmup_variants="none"` (server: `FRNET_WARMUP=none`) skips the capture at load; call `model.warmup()` before the
+  first frame (it is idempotent).
+- The JIT cache is tt-metal's (`~/.cache/tt-metal-cache`, or `TT_METAL_CACHE`); inside the container it is
+  `~/.cache/tt-model/frnet-p150/cache` of the host.
+
+## Call: `model(points, calibration=..., **kwargs)`
+
+| Argument | Type | Description |
+|---|---|---|
+| `points` (first positional) | path, `bytes` + `fmt=`, (N, C) array / tensor, structured array, `PointCloud`, JSON envelope | ONE LiDAR sweep in the **sensor frame** (x forward, y left, z up, metres), fields `x, y, z, intensity` with the raw intensity the node casts (0-255 for uint8 drivers); see "Input types" |
+| `calibration` | `{"T_base_link_from_sensor": <transform>}`, `{"preset": name}` (a file of `code/tt_frnet/calib/`), optional `"ego_crop_box"` | the static sensor -> `base_link` transform the node looks up once for its ego crop box. `ego_crop_box`: `[min_x, min_y, min_z, max_x, max_y, max_z]` in `base_link` (default Autoware's `(-1.53, -1.15, -0.1)` .. `(6.1, 1.15, 3.1)`), or `None` (or all zeros) to disable the box, the node's own switch; a disabled box needs no transform |
+| `class_probability_threshold` | float in [0, 1], default 0.05 | the filtered cloud keeps a point unless its probability of a filter class is >= this value (Autoware's `filter.class_probability_threshold`) |
+| `filter_classes` | comma-separated class names, `""` = none, default `"drivable_surface"` | Autoware's `filter.classes` (at most 16); an unknown name is refused |
+
+Both knobs only change `filter_keep`; the class decision (`label_ids`, `scores`) never depends on them. They are
+host-only (no device shape changes), the defaults reproduce Autoware's `frnet.param.yaml`, and a bad value raises
+`tt_frnet.io.InputError` (HTTP 400) before the device runs. `model.info["runtime_params"]` (and the server's `/info`)
+lists them with their defaults.
+
+### Input types
+
+- Point clouds: a path (`.bin` raw float32 rows -- pass `fields=` if not `x, y, z, intensity`; `.npy`; `.npz`;
+  `.pcd` ascii / binary / binary_compressed), raw bytes with `fmt=`, an (N, C) numpy array or torch tensor, a
+  structured array, the JSON envelope of `/predict`, or `tt_frnet.io.PointCloud`. The frame is the **sensor** frame
+  (Autoware's `~/input/pointcloud` before any transform), one sweep, not motion-compensated.
+- Rows with a non-finite x / y / z are dropped by the decoder; `input_index` refers to the decoded rows.
+- Calibration transforms: a 4×4 (or 3×4) row-major matrix, `{"translation": [x, y, z], "rotation_wxyz": [w, x, y, z]}`,
+  or Autoware's `{x, y, z, roll, pitch, yaw}`. With the crop box on and no transform the call is refused (the node would
+  skip every frame because its TF lookup fails); the error names both fixes.
+- `sweeps`, `images`, `stream` and `inputs` are refused: FRNet takes one sweep and its calibration only.
+- A frame outside the variant's profile (input points, points after interpolation, or unique cells out of the table
+  above) is not an error: as the node, which logs and publishes nothing, the call returns an empty output whose
+  `meta["skipped"]` holds the node's message, and nothing runs on the device.
+
+## Output
+
+`model(...)` returns a `tt_frnet.FRNetSegmentation` (a `ttaw.outputs.Segmentation3D`); `out.to_dict()` is exactly the
+`POST /predict` body (SERVING.md section 3.2). The point order is the node's: the input points that survive the ego
+crop box, in input order, then the points Autoware's interpolation appends (D4: published as Autoware does).
+
+| field | type | meaning |
+|---|---|---|
+| `label_ids` | uint8 `[N]` | Autoware's `class_id` (first maximum of the 27 probabilities, as `fill_cloud_kernel`); names in `class_names` |
+| `scores` | float32 `[N]` | Autoware's `probability`: the softmax probability of that class |
+| `points` | float32 `[N, 4]` | x, y, z, intensity of every output point in the sensor frame (the segmentation cloud's coordinates; the interpolated points carry interpolated values) |
+| `filter_keep` | bool `[N]` | membership of Autoware's `~/output/pointcloud/filtered` cloud (False = removed by the class filter) |
+| `num_points_raw` | int | the first `num_points_raw` rows are input points, the rest are interpolated |
+| `input_index` | int32 `[num_points_raw]` | the decoded input row of every raw output point |
+| `class_names` | 27 str | `drivable_surface`, `other_flat_surface`, `sidewalk`, `manmade`, `vegetation`, `car`, `bus`, `emergency_vehicle`, `train`, `truck`, `tractor_unit`, `semi_trailer`, `construction_vehicle`, `forklift`, `kart`, `motorcycle`, `bicycle`, `pedestrian`, `personal_mobility`, `animal`, `pushable_pullable`, `traffic_cone`, `debris`, `stroller`, `other_stuff`, `noise+ghost_point`, `unknown` (Autoware's names; ids 22 / 23 were trained as stroller / debris, see Limits) |
+| `meta` | dict | `variant`; `num_points_input`; `ego_crop` (the box and transform used); the network-input counts (`num_points_raw`, `num_points_interpolated`, `num_points`, `num_unique_coors`, `cropped_by_ego_box`, `crop_applied`, `max_points_per_cell`); `plan` (`n`, `m`, `n_cap`, `m_cap`); `filter` (classes, threshold, points kept); `skipped` (the node's message) for an out-of-profile frame |
+| `timing_ms` | dict | `preprocess`, `device` (staging + H2D + trace + D2H + un-permute), `postprocess`, `total` of this call |
+
+`out.class_counts()` gives the points per class (raw + interpolated); `out.to_dicts()` one row per class
+(`label`, `label_id`, `points`); `len(out)` the number of output points. The node's visualization cloud is
+`model.config.palette` indexed by `label_ids` (`tt_frnet.host.postprocess.visualization_rgb`).
+
+## Lifetime and information
+
+- `model.close()` releases the trace and the device tensors and closes the chip if the model opened it; idempotent.
+  `with` calls it for you; an unclosed model is closed when Python exits.
+- `model.info`: weights (repo, tag, revision, path), device (`dispatch`, `grid`, `cores`, `num_command_queues`,
+  `fallback`), variant, warm variants, warm-up times, runtime parameter defaults, the variant config (`frnet`), the
+  trace and graph descriptions.
+- Calls from several threads are safe: the device calls are serialised by a lock. One model per process per chip.
+- `tt_frnet.viz.render_bev(points, label_ids, palette=...)` and `render_compare(...)` draw bird's-eye views (PIL, no
+  OpenCV).
+- `model.device_probs(plan)` returns the device's `pred_probs` `(N, 27)` of a prepared frame (tests and tools; call it
+  under `model._lock`).
+
+## Speed
+
+Warm calls, batch 1, ETH dispatch, 1 CQ, 12x10, the serve.env precision knobs (`code/scripts/bench.py`, 100
+iterations; the numbers of `OPT_BASELINE.md`, 2026-10-09, on a shared host whose load moves the host stages):
+
+| stage, p50 | ot128, shipped sample | ot128, PandaSet 019 f40 | qt128, shipped sample | qt128, PandaSet 019 f40 |
+|---|---:|---:|---:|---:|
+| `model(path)` end to end | 292.6 ms | 355.7 ms | 221.1 ms | 244.3 ms |
+| host pre-processing (decode, crop, projection, fp16 interpolation, unique cells, device plan) | 55.3 ms | 107.2 ms | 54.2 ms | 83.5 ms |
+| H2D (9 inputs, 8.6 / 6.2 MB) | 5.42 ms | 5.41 ms | 4.04 ms | 4.00 ms |
+| device trace, one forward | 210.02 ms | 211.72 ms | 133.53 ms | 134.91 ms |
+| D2H (fp32 `pred_probs`) | 7.65 ms | 7.43 ms | 5.80 ms | 5.55 ms |
+| un-permute + host post-processing (argmax, filter) | 7.1 + 5.0 ms | 12.8 + 14.8 ms | 6.1 + 6.8 ms | 5.9 + 10.5 ms |
+
+- Back-to-back replays take 211.64 ms (ot128) and 134.87 ms (qt128) per PandaSet frame: 4.72 and 7.41 frames/s of
+  device throughput. A synchronous call gives 2.81 / 4.09 frames/s on that frame.
+- The device time barely depends on the frame: every point-wise op runs on the profile's `N_cap` rows. The host
+  stages grow with the cloud.
+- The shipped samples are synthetic scans: ot128 53,411 points (N = M = 52,613 after the crop, no interpolated point),
+  qt128 45,783 points (N = 69,553 after interpolation, M = 18,926). PandaSet 019 frame 40 is a 107,146-point Pandar64
+  sweep (N = 145,961 / M = 40,691 on ot128); it is not shipped.
+- `OPT_REPORT.md` ranks what the optimization phase will change (the two-term point layers, K1, the image-branch glue,
+  shape buckets, the host pre-processing).
+
+## Tests
+
+From the repository root, with `code/` on `PYTHONPATH` (or the package installed):
+
+- Host (no device, no ttnn call): `TT_VISIBLE_DEVICES=none python -m pytest -q code/tt_frnet/tests`. Inside the
+  tt-models workspace the suite runs on its fake ttnn (`-p fake_ttnn_plugin`), which also enables `test_tt_host.py`;
+  tests that need workspace-only oracles or data skip elsewhere, and FAIL inside the workspace when that data is
+  missing. `test_reference_cpu.py` needs onnxruntime.
+- Device (one p150): `python -m pytest -q -s code/tt_frnet/tests/test_pcc_device.py
+  code/tt_frnet/tests/test_e2e_device.py` (per-module PCC against the fp32 CPU reference on replay outputs, the K1
+  segment-max sites included; end-to-end agreement with ONNX Runtime on 17 OT128 / 5 QT128 frames, the shipped samples
+  against their stored references, replay == eager, API == `/predict`). The public-frame gates run only inside the
+  tt-models workspace, where their data lives: PandaSet, nuScenes and Autoware's sample rosbag, of which only PandaSet
+  may be redistributed.
+- The gates are frozen in `tests/*.gates.json` at the first green run and are never loosened.
+
+## Limits
+
+- Batch 1 on the chip; one model per process; one frame (one sweep) per call; one variant per model.
+- The device input has a fixed size per profile (`N_cap` 160,000 / 120,000 points, `M_cap` 60,000 cells): a frame
+  outside Autoware's profile ranges is skipped exactly as the node skips it.
+- Label names 22 / 23: Autoware's `class_names` read `debris` / `stroller`, but the weights were trained with
+  stroller = 22 and debris = 23 (AWML config). The port keeps Autoware's names for parity.
+- Numerics: the device's per-point probabilities differ slightly from the fp32 reference, so points near a class
+  boundary can take the other class (98.9-99.6 % of the points agree on the public frames, see the model card).
+- Not ported: Autoware's `PointCloud2` layouts and the filtered cloud's `output_format` conversion; the API takes
+  arrays or files and returns arrays.

README.md

@@ -0,0 +1,202 @@
+---
+tags:
+- blackhole
+- p150
+- tt-dit-server
+- tt-model-cache
+- tt-model-container
+- tenstorrent
+- ttnn
+- tt-metal
+- tt-nn
+- autoware
+- autonomous-driving
+- lidar
+- point-cloud
+- semantic-segmentation
+- frnet
+- range-view
+- tt-model-catalog
+pipeline_tag: image-segmentation
+license: apache-2.0
+license_link: https://huggingface.co/AutowareFoundation/lidar_frnet
+base_model:
+- AutowareFoundation/lidar_frnet
+---
+
+# frnet-p150
+
+FRNet (Autoware lidar_frnet): the LiDAR semantic segmentation network of Autoware's opt-in `autoware_lidar_frnet` node, ported to one Tenstorrent Blackhole p150 with tt-nn. One LiDAR sweep in the sensor frame (x, y, z, raw 0-255 intensity) in; a class id and probability for every point (27 T4 classes, Autoware's interpolated points included) and the node's filtered-cloud mask out, with the node's exact pre- and post-processing. One image, two serve profiles: `ot128` (Hesai OT128, the default) and `qt128` (Hesai QT128).
+Weights: [AutowareFoundation/lidar_frnet `v2.0`](https://huggingface.co/AutowareFoundation/lidar_frnet/tree/a53a1c11b0b28d31fd50a03abef5db0e830a5b3c) · Paper: [arXiv:2312.04484](https://arxiv.org/abs/2312.04484) · Autoware package: [autoware_lidar_frnet](https://github.com/autowarefoundation/autoware_universe/tree/9ceaccf026c31ffc5319bc9eeb4bd7bede0af3fd/perception/autoware_lidar_frnet) · Training code: [tier4/AWML (projects/FRNet)](https://github.com/tier4/AWML) · Port: [`code/`](https://huggingface.co/changh95/frnet-p150/tree/main/code)
+
+Runs on **p150** (mesh `P150`). Configuration: dispatch on the ETH cores, 1 command queue, 12×10 compute grid. Precision: fp32 weights; the point-wise layers (point MLPs, fusions, head) take fp32 activations as two bf16 terms with fp32 accumulation; the frustum CNN runs fp32 feature maps with bf16 conv operands; HiFi4 math with fp32 accumulation everywhere; no bfp8. Both serve profiles run in this configuration, and all numbers on this card were measured in it.
+
+Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1).
+
+## Quickstart (Python)
+
+Prerequisite: a tt-metal / ttnn environment at tt-metal [`44d66500520`](https://github.com/tenstorrent/tt-metal/commit/44d66500520fda9f2c7060c0f6b41ec48f7ab37e) with [`patches/tt-metal-eth-dispatch.patch`](patches/tt-metal-eth-dispatch.patch) applied. ttnn is not on PyPI.
+
+```bash
+hf download changh95/frnet-p150 --exclude "image/*" --local-dir frnet-p150 && cd frnet-p150
+pip install -e .                        # adds numpy<2, pillow, pyyaml, onnx, huggingface_hub; ttnn and torch come from tt-metal
+pip install -e ".[server,test]"         # optional: the HTTP server and the tests
+```
+
+Run the snippet from the model repo root: the sample and its calibration preset are paths relative to it.
+
+```python
+from tt_frnet import FRNet
+
+with FRNet.from_pretrained(device_id=0) as model:                  # variant="qt128" for the Hesai QT128 profile
+    out = model("code/tt_frnet/samples/synthetic_ot128.npz",       # one sweep in the sensor frame: path, (N, 4) array, .npy/.npz/.pcd, or a PointCloud
+                calibration={"preset": "synthetic_ot128"})         # its sensor -> base_link transform, for the node's ego crop box
+
+print(out.class_counts())                                          # points per class (raw + interpolated)
+labels, scores, xyz = out.label_ids, out.scores, out.points[:, :3]
+```
+
+- `from_pretrained` downloads [`AutowareFoundation/lidar_frnet`](https://huggingface.co/AutowareFoundation/lidar_frnet) at the pinned commit `a53a1c11b0b` (tag `v2.0`, 85 MB) to your HF cache, checks the ONNX sha256, opens the chip, builds the graph of the variant and captures the metal trace. The first load compiles the kernels (156.6 s for ot128 and 151.9 s for qt128 with an empty JIT cache); later loads take 7.5 / 6.6 s.
+- The trace is captured during the load, so the first call compiles nothing: it took 320 ms (ot128) / 241 ms (qt128), against 305 / 215 ms for the second call.
+- The `with` block releases the trace and closes the chip. Without `with`, call `model.close()`.
+- For your own cloud, pass its transform: `calibration={"T_base_link_from_sensor": T}` (4×4, or `{x, y, z, roll, pitch, yaw}`), or `{"ego_crop_box": None}` to disable the ego crop box.
+
+| | |
+|---|---|
+| **Input** | ONE LiDAR sweep in the **sensor frame** (x forward, y left, z up; Autoware's `~/input/pointcloud`): path (`.bin` / `.npy` / `.npz` / `.pcd`), raw bytes with `fmt=`, an (N, C) float array or tensor, or a `PointCloud`; fields `x, y, z, intensity` (raw, 0-255). `calibration`: `{"T_base_link_from_sensor": ...}` or `{"preset": name}` for the node's ego crop box, optional `ego_crop_box` (bounds, or `None` to disable it). |
+| **Options** | `class_probability_threshold=0.05`, `filter_classes="drivable_surface"` (Autoware's filter: they change `filter_keep` only). `from_pretrained(device_id=0, variant="ot128" \| "qt128", dispatch="eth", weights_dir=None, device=None)`. |
+| **Output** | `FRNetSegmentation`: `label_ids` uint8 [N] (Autoware's `class_id`), `scores` float32 [N] (its `probability`), `points` float32 [N, 4], `filter_keep` bool [N] (the filtered cloud), `num_points_raw`, `input_index` [N_raw], `class_names` (27), `meta` (crop, counts, `skipped`), `timing_ms`. Order: the input points kept by the ego crop box (input order), then Autoware's interpolated points. |
+| **Methods** | `out.to_dict()` gives the `/predict` JSON. `out.class_counts()` gives the points per class, `out.to_dicts()` one row per class. `tt_frnet.viz.render_bev(points, label_ids)` draws a bird's-eye view. |
+
+- The API gives the same output as the HTTP server `/predict`: both share the decoders, the device trace and the host post-processing (checked on the device by `test_api_equals_server`, both profiles).
+- `variant` selects the network at load time. Each is tied to its sensor's FOV and frustum grid: `ot128` +15° / −25°, 128 × 1024 (Autoware's default `sensor_model`); `qt128` ±52.6°, 128 × 256.
+- A frame outside the variant's profile (5,000-160,000 / 120,000 points, 3,000-60,000 cells) returns an empty output with the node's message in `meta["skipped"]`, as the node publishes nothing; nothing runs on the device.
+- One model uses one chip; calls from several threads are serialised.
+- Full reference: [`code/PYTHON.md`](code/PYTHON.md). Runnable example: [`examples/quickstart.py`](examples/quickstart.py) (also writes `quickstart_bev.png`, the p150 classes from above next to the stored CPU reference).
+
+## Serving (HTTP)
+
+```bash
+tt-model pull  changh95/frnet-p150 --with-weights
+tt-model serve changh95/frnet-p150       # QT128: tt-model serve --profile qt128 changh95/frnet-p150 (options go before the repo id); tt-cli: tt serve changh95/frnet-p150
+python3 code/tt_frnet/server/client.py --points code/tt_frnet/samples/synthetic_ot128.npz --calib-preset synthetic_ot128 --out req.json
+curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
+tt model stop changh95/frnet-p150
+```
+
+- The image does not contain the weights. `--with-weights` puts them in your HF cache.
+- The server uses port 20000 (or the next free port). It is ready when the log shows `Application startup complete`.
+- Two serve profiles (`tt-model profiles changh95/frnet-p150`): `ot128` (the default) and `qt128`. Each serves one variant.
+- `client.py` builds the request with the standard library only; add `--url http://127.0.0.1:20000` to send it, `--calib my_calib.json` for your own transform.
+- `POST /predict`: `points` (base64 `.bin` / `.npy` / `.npz` / `.pcd`, sensor frame, fields x, y, z, intensity); `calibration` (`T_base_link_from_sensor`, or `{"preset": name}`; optional `ego_crop_box`, `null` disables the box); optional `params` (`class_probability_threshold`, `filter_classes`), `output_format`. Also `GET /health`, `GET /info` (with the effective default of every knob and the calibration presets), `GET /v1/models` (stub). Contract: [`SERVING.md`](SERVING.md) section 3.
+
+```json
+{"model": "frnet-p150", "frame_id": "sensor", "num_points": 52613, "num_points_raw": 52613,
+ "labels": {"format": "npz", "key": "labels", "dtype": "uint8", "shape": [52613], "data": "..."},
+ "scores": {"format": "npz", "key": "scores", "dtype": "float32", "shape": [52613], "data": "..."},
+ "points": {"format": "npz", "key": "points", "dtype": "float32", "shape": [52613, 4], "data": "..."},
+ "filter_keep": {"format": "npz", "key": "filter_keep", "dtype": "bool", "shape": [52613], "data": "..."},
+ "input_index": {"format": "npz", "key": "input_index", "dtype": "int32", "shape": [52613], "data": "..."},
+ "class_names": ["drivable_surface", "other_flat_surface", "sidewalk", "manmade", "vegetation", "car", "..."],
+ "class_counts": {"drivable_surface": 2913, "other_flat_surface": 20118, "sidewalk": 5, "manmade": 22898, "vegetation": 6526, "car": 13, "semi_trailer": 140, "...": 0},
+ "meta": {"variant": "ot128", "num_points_input": 53411, "cropped_by_ego_box": 798, "num_points_interpolated": 0, "num_unique_coors": 52613,
+          "ego_crop": {"enabled": true, "bounds": [-1.53, -1.15, -0.1, 6.1, 1.15, 3.1], "reference_frame": "base_link", "...": "..."},
+          "plan": {"n": 52613, "m": 52613, "n_cap": 160000, "m_cap": 60000, "...": "..."},
+          "filter": {"classes": ["drivable_surface"], "class_probability_threshold": 0.05, "kept": 44747}, "...": "..."},
+ "timing_ms": {"decode": 24.711, "preprocess": 47.664, "device": 229.164, "postprocess": 5.018, "model_call": 281.879, "total": 310.689}}
+```
+
+- `labels` / `scores` / `points` / `filter_keep` / `input_index` are base64 NPZ arrays (`tt_frnet.io.decode_array`, or numpy on the decoded bytes); `class_counts` and `meta` are plain JSON.
+- The first `num_points_raw` points are input points (`input_index` maps them back to the input rows); the rest are the points Autoware's interpolation appends, published as Autoware does.
+
+## Demo
+
+The p150 output on the shipped samples: synthetic scans generated by this repository (a ray cast against boxes; no dataset involved). Left: the p150 classes; middle: the stored fp32 CPU reference; right: the points whose class differs.
+
+| OT128 profile: p150 vs the CPU reference | QT128 profile: p150 vs the CPU reference |
+|:---:|:---:|
+| ![](media/frnet_synthetic_ot128_tt_vs_cpu.png) | ![](media/frnet_synthetic_qt128_tt_vs_cpu.png) |
+
+On public driving datasets (p150 outputs, OT128 profile; the frames themselves are not in this repository). The caption of each image gives its agreement with the fp32 CPU reference on that frame.
+
+| PandaSet 019, frame 40: front camera and bird's-eye view, p150 classes vs the mapped dataset labels | PandaSet 029, frame 40 |
+|:---:|:---:|
+| ![](media/frnet_ot128_pandaset_019_40_tt.png) | ![](media/frnet_ot128_pandaset_029_40_tt.png) |
+| **PandaSet 019, frame 40: p150 vs the CPU reference** | **PandaSet 019, frames 00-75 (every 5th frame)** |
+| ![](media/frnet_ot128_pandaset_019_40_tt_vs_cpu.png) | ![](media/frnet_ot128_pandaset_019_bev_tt.gif) |
+| **nuScenes v1.0-mini scene-0103, key-frame 20: non-commercial, CC BY-NC-SA 4.0** | **nuScenes v1.0-mini scene-0916, key-frame 40, p150 vs the CPU reference (the hardest frame of the release): non-commercial, CC BY-NC-SA 4.0** |
+| ![](media/frnet_ot128_nuscenes_scene-0103_20_tt_NC.png) | ![](media/frnet_ot128_nuscenes_scene-0916_40_tt_vs_cpu_NC.png) |
+
+PandaSet renders: contains data from PandaSet (Scale AI and Hesai), https://pandaset.org, licensed under CC BY 4.0 and the PandaSet Dataset Terms; converted to the sensor frame, rendered as bird's-eye views and downscaled camera images with projected points; Scale AI and Hesai do not endorse this work. The nuScenes renders are **non-commercial (CC BY-NC-SA 4.0)**: rendered from the nuScenes dataset, © Motional AD Inc., nuScenes Terms of Use; Motional does not endorse this work. Sources, changes and the full attributions: [`media/ATTRIBUTION.md`](media/ATTRIBUTION.md).
+
+## Demo & Performances
+
+Warm, batch 1, 2026-10-09. Latency: the stage bench of [`OPT_BASELINE.md`](OPT_BASELINE.md) (`code/scripts/bench.py`, 100 iterations per stage; re-checked on the release commit to within 0.05 ms on the device rows) on the shipped synthetic sample (OT128: 53,411 points, 52,613 after the ego crop; QT128: 45,783 points, 69,553 after Autoware's interpolation) and on a PandaSet frame (107,146 points; not shipped); the served rows from uvicorn on the host (the app the container runs) and a loopback client, 50 requests of the shipped sample. The host is shared with other jobs, so the host stages move with its load; the device rows repeat to 0.01 ms. Accuracy: the p150 output against the fp32 CPU reference of the same network (ONNX Runtime on the deployed ONNX, with the port's Autoware pre- and post-processing, which is bit-identical to the research reference), on every raw point the node scores (public frames); on the shipped samples, on every output point, Autoware's interpolated points included.
+
+| Metric | Performance |
+|---|---:|
+| Class agreement with the fp32 CPU reference, shipped sample | OT128 **99.55 %** of 52,613 points (filtered-cloud mask 99.77 %) · QT128 **99.62 %** of 69,553 (99.87 %) |
+| Class agreement, all 113 converted public frames per profile: PandaSet 019 / 029 (32 frames, every 5th) and nuScenes v1.0-mini mini_val (81 key-frames), production API | OT128: PandaSet **99.66 %** pooled (worst frame 99.35 %), nuScenes **99.19 %** (worst 98.37 %) · QT128: PandaSet 99.29 % (98.69 %), nuScenes 99.44 % (98.10 %) |
+| Frozen device gates, 17 OT128 / 5 QT128 frames (PandaSet, nuScenes, Autoware's sample rosbag; gates `pred_probs` PCC ≥ 0.998, agreement ≥ 98.5 %) | **all pass**: OT128 min PCC 0.99969, min agreement **98.907 %** (nuScenes 0916 kf40, the binding frame); QT128 0.99980 / 99.158 % |
+| Module PCC vs the fp32 reference (replay outputs, PandaSet 019 f40) | every gated module ≥ 0.9996: worst OT128 `point_fusion1` 0.999729, QT128 `attention4` 0.999649 (gate 0.999) |
+| Mapped-label mIoU, OT128 (a sanity check, not a paper metric: dataset labels mapped to the 27 T4 classes, see Caveats) | PandaSet 32 frames: coarse **p150 0.714** · CPU 0.715, fine (classes with ≥ 1,000 labelled points) 0.447 · 0.448 · nuScenes 81 frames: coarse **p150 0.483** · CPU 0.483, fine (same rule) 0.451 · 0.453 |
+| Python `model()` call, shipped sample (host pre-processing, H2D, trace, D2H, host post-processing) | OT128 **292.6 ms p50** (3.4 calls/s) · QT128 **221.1 ms p50** (4.5 calls/s) |
+| Python `model()` call, PandaSet frame (107,146 points; not shipped) | OT128 355.7 ms · QT128 244.3 ms p50 |
+| Served `/predict` `timing_ms.total` (uvicorn on the host, the shipped sample) | OT128 **310.4 ms** · QT128 **228.9 ms** median (including 24.3 / 20.7 ms of request decode) |
+| Served client round trip, loopback | OT128 328.1 ms · QT128 247.3 ms median |
+| Device trace, one blocking forward (shipped sample) | OT128 **210.02 ms** · QT128 **133.53 ms** |
+| Back-to-back trace replays | OT128 **209.95 ms** per forward (4.76 frames/s; 211.64 ms on PandaSet) · QT128 **133.43 ms** (7.49 frames/s; 134.87 ms) |
+| Host pre-processing · H2D · D2H · un-permute + post-processing (shipped sample) | OT128 55.3 · 5.42 · 7.65 · 7.1 + 5.0 ms; QT128 54.2 · 4.04 · 5.80 · 6.1 + 6.8 ms |
+| `from_pretrained` load: empty JIT cache / warm cache | OT128 156.6 s / 7.5 s · QT128 151.9 s / 6.6 s |
+
+All numbers in this table were measured with dispatch on the ETH cores, 1 command queue and a 12×10 compute grid on one p150 at 1350 MHz AICLK, with the published precision (`FRNET_POINT_TERMS=2`, `FRNET_MAP_DTYPE=float32`, `FRNET_CONV_TERMS=1`). The wider 113-frame agreement set is not gated: 3 of its 226 frame-profile pairs fall below 98.5 % (nuScenes scene-0916 key-frames 11 and 1 on OT128, 98.37 % and 98.499 %; key-frame 0 on QT128, 98.10 %), all on the out-of-domain Singapore scene (see Caveats). The mIoU rows are indicative only: approximate label mappings, small sets, sensors and taxonomy different from the training data, and no published number exists for these weights. Details: [`VERIFICATION_2026-10-09.md`](VERIFICATION_2026-10-09.md), [`OPT_BASELINE.md`](OPT_BASELINE.md), [`OPT_REPORT.md`](OPT_REPORT.md).
+
+No GPU comparison: no GPU was available on the host where this port was built and measured, so this card makes no GPU speed claim. The reference rows are the port's own fp32 CPU reference on the same host (a correctness baseline, not a speed target). Autoware runs this network as a TensorRT engine (fp16 by default, per the weights card); this port was validated against the fp32 ONNX Runtime reference, not against a TensorRT engine. p150 power was not measured, so no efficiency comparison is made.
+
+## Caveats
+
+- First release: **baseline port, optimization pending.** Each profile runs as one metal trace of 1,104 (ot128) / 759 (qt128) programs and is kernel-bound (op-to-op gaps 0.3 %). 63 % (ot128) / 75 % (qt128) of the device time is the point-wise layers kept at higher precision, whose cost is DRAM traffic of fp32 partial sums, not compute; every point-wise op also runs on the profile's full capacity (160,000 / 120,000 rows) whatever the frame. [`OPT_REPORT.md`](OPT_REPORT.md) ranks what comes next.
+- Deployment status in Autoware: FRNet is an **opt-in, standalone node**. `autoware_lidar_frnet` ships in autoware_universe with its own launch file (`lidar_frnet.launch.xml`, `sensor_model:=ot128 | qt128`), but no Autoware launch configuration wires it into the default perception stack. This bundle is not a ROS 2 node (Python API and HTTP) and not a certified Autoware component; do not use it for safety-critical driving decisions.
+- **Narrow accuracy margin on one frame.** On the hardest public frame of the release (nuScenes scene-0916 key-frame 40, OT128: a 32-beam sensor, out of the training domain, low-confidence outputs), the p150 agrees with the fp32 reference on 98.907 % of the points, 0.41 points above the 98.5 % gate. Every cheaper precision configuration measured during the port failed that frame (bf16 maps 98.28 %, an fp32 × fp32 point island 97.50 %), which is why the release keeps the slower two-term point layers and fp32 maps. Beyond the gated frames, 3 of the 226 frame-profile pairs of the wider public set fall just below 98.5 % (98.10-98.499 %), all on the same Singapore scene-0916. It is the binding constraint, and an explicit precision item, of the optimization phase.
+- Points near a class boundary can take the other class: the device's probabilities differ slightly from the fp32 reference (`pred_probs` PCC ≥ 0.9997 on every public frame), and the argmax turns a small difference into a different class. The figures are in the agreement rows above.
+- Documented deviations from the node:
+  - Autoware's CUDA races are replaced by fixed, documented policies: interpolated points are appended in row-major pixel order after the raw points (Autoware's atomic append order varies), the fp16 interpolation grid keeps the farthest point with ties to the later point, a duplicated cell keeps the last voxel (as ONNX Runtime); the output is deterministic;
+  - the projection uses numpy's float32 `arctan2` / `arcsin`, which may differ from CUDA's `atan2f` / `asinf` by an ulp, so a point exactly on a bin edge could change cell;
+  - the pre-processing, the argmax and the class filter run on the host (bit-identical to the CPU reference); the network on the device;
+  - inputs are arrays or files, not `PointCloud2` messages; the filtered cloud is returned as a mask (`filter_keep`), not converted to Autoware's `output_format`;
+  - label names 22 / 23: Autoware's `class_names` read `debris` / `stroller`, but the weights were trained with stroller = 22 and debris = 23 (AWML config). The port keeps Autoware's names for parity.
+- Precision policy of this release: fp32 weights; the point-wise linears split their fp32 inputs and weights into two bf16 terms (three exact bf16 products, fp32 accumulation; the device's own fp32 matmul is TF32-like and fails the gates); fp32 feature maps in the frustum CNN with bf16 conv operands and fp32 weights; HiFi4 with fp32 accumulation and packer L1 accumulation; K1 `segment_reduce` (a custom `generic_op` kernel) is bit-exact. Pinned in `serve.env` (`FRNET_POINT_TERMS=2`, `FRNET_MAP_DTYPE=float32`, `FRNET_CONV_TERMS=1`).
+- Domain: the weights were trained by TIER IV on about 16,000 frames of the T4 dataset from Hesai OT128 / QT128 sensors (not public), with a 27-class T4 taxonomy. No public dataset has these sensors or labels, so the card shows agreement with the fp32 reference, not accuracy. On the public frames the reference itself shows the gap: PandaSet's Hesai Pandar64 shares the OT128's vertical FOV and segments cleanly (coarse mapped mIoU 0.714 on the p150, 0.715 for the reference, OT128), nuScenes' 32-beam HDL-32E much less so (0.483 for both), and the QT128 network on these sensors is out of distribution (coarse 0.19 on PandaSet; used here only as a numerical test). Traffic cones are almost never found and pedestrians next to structures often become `manmade`, in the reference as on the p150.
+- Validation scope: agreement with the fp32 CPU reference (ONNX Runtime on the deployed ONNX, with the port's bit-exact Autoware pre- and post-processing) on public driving data (PandaSet, nuScenes v1.0-mini) and Autoware's sample rosbag, plus a mapped-label mIoU sanity check (not a paper metric: different taxonomy and sensors). The knobs were validated at their Autoware defaults.
+- Does not scale to multiple p150 in a mesh configuration. The build uses a 12×10 compute grid of Tensix cores: the dispatch functions move from one Tensix column to the ETH cores (`patches/tt-metal-eth-dispatch.patch`), so this build assumes that you do not need chip-to-chip ethernet communication.
+- `dispatch="worker"` (server: `FRNET_DISPATCH=worker`) is an A/B opt-in. On a p150 it gives an 11×10 grid (211.56 ms per ot128 replay instead of 209.95 ms; 153.71 ms instead of 133.44 ms for qt128); if ETH dispatch is not available (tt-metal without the patch), the model falls back to it with a warning. The numbers on this card do not apply to that mode.
+- Batch 1, one sweep per request; requests are serialised on the chip; one variant per process (the serve profile). The device input has a fixed size per profile (160,000 / 120,000 points, 60,000 cells), whatever the cloud.
+- Not an OpenAI-compatible API; `GET /v1/models` is a stub so the tt-model ready card does not 404.
+- p150 power was not measured, so no efficiency comparison is made.
+
+## Licensing
+
+- Weights: [AutowareFoundation/lidar_frnet](https://huggingface.co/AutowareFoundation/lidar_frnet) at tag `v2.0` (commit `a53a1c11b0b28d31fd50a03abef5db0e830a5b3c`), Apache-2.0 per its model card. Not redistributed here: the package only points to them. The upstream card's training data: the T4Dataset (TIER IV), about 16,000 frames (4,000 frames × 4 surrounding sensors), trained with AWML; it states no further data licence notice.
+- Pre- and post-processing ported from autoware_universe [`perception/autoware_lidar_frnet`](https://github.com/autowarefoundation/autoware_universe/tree/9ceaccf026c31ffc5319bc9eeb4bd7bede0af3fd/perception/autoware_lidar_frnet) (Apache-2.0); the FOV, grids, class names, palette and profiles are read from the weights repo's `ml_package_frnet_*.param.yaml` at load time.
+- Port and serving code (`code/`): Apache-2.0. `patches/tt-metal-eth-dispatch.patch` modifies tt-metal (Apache-2.0).
+- Sample data: `code/tt_frnet/samples/synthetic_ot128.npz` and `synthetic_qt128.npz` are synthetic scans generated by `code/scripts/make_sample.py` (no dataset involved), with their calibration presets and stored CPU-reference outputs: Apache-2.0. Only these samples ship; the public-dataset frames of the accuracy rows are not in this repository.
+- Demo media (`media/`, sources and changes in [`media/ATTRIBUTION.md`](media/ATTRIBUTION.md)):
+  - PandaSet renders: Contains data from PandaSet (Scale AI and Hesai), https://pandaset.org, licensed under CC BY 4.0 and the PandaSet Dataset Terms. Changes: the Pandar64 sweep re-expressed in an estimated sensor frame, bird's-eye views rendered, the front-camera image downscaled and dimmed, points and labels overlaid. Scale AI and Hesai do not endorse this work. Cite: P. Xiao et al., *PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving*, ITSC 2021.
+  - nuScenes renders (`media/*_NC.png`), **non-commercial, CC BY-NC-SA 4.0**: Rendered from the nuScenes dataset, © Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of Use (https://www.nuscenes.org/terms-of-use). Non-commercial use only; adaptations under the same license. Motional does not endorse this work. Cite: H. Caesar et al., *nuScenes: A Multimodal Dataset for Autonomous Driving*, CVPR 2020; W. K. Fong et al., *Panoptic nuScenes*, RA-L 2022 (lidarseg labels).
+  - `frnet_synthetic_*` renders: Apache-2.0.
+  - The agreement and mapped-mIoU rows use PandaSet (CC BY 4.0) and nuScenes v1.0-mini with nuScenes-lidarseg (CC BY-NC-SA 4.0); only the resulting numbers are on this card.
+
+## Provenance
+
+These are the exact sources the container image was built from:
+
+| component | built from |
+| --- | --- |
+| tt-metal | [`44d66500520fda9f2c7060c0f6b41ec48f7ab37e`](https://github.com/tenstorrent/tt-metal/commit/44d66500520fda9f2c7060c0f6b41ec48f7ab37e) + [`patches/tt-metal-eth-dispatch.patch`](patches/tt-metal-eth-dispatch.patch) (sha256 `08d0ddf6…`; dirty tree: the image includes the patch) |
+| weights | [`AutowareFoundation/lidar_frnet@a53a1c11b0b28d31fd50a03abef5db0e830a5b3c`](https://huggingface.co/AutowareFoundation/lidar_frnet/tree/a53a1c11b0b28d31fd50a03abef5db0e830a5b3c) (tag `v2.0`), files `frnet_*.onnx, ml_package_frnet_*.param.yaml, deploy_metadata.yaml` (ONNX sha256 ot128 `d29bf595…`, qt128 `647af927…`) |
+| Autoware reference | autoware_universe [`9ceaccf026c31ffc5319bc9eeb4bd7bede0af3fd`](https://github.com/autowarefoundation/autoware_universe/tree/9ceaccf026c31ffc5319bc9eeb4bd7bede0af3fd) (`perception/autoware_lidar_frnet`, package 0.53.0) |
+| shared package | `ttaw` 0.20.0, vendored as `code/tt_frnet/ttaw` from the Autoware ports' shared `common` repository at commit `89dec49` (`code/tt_frnet/ttaw/VENDORED.json`: version, commit and per-file sha256); it holds the K1 `segment_reduce` kernel |
+| `code/` digest (image) | `f0b62790446a51a8` (sha256, first 16 hex digits; `built.code_sha256` of `tt_kernel_manifest.json`) |
+| image | `tt-model/frnet-p150:85cbf6cc72b9` (`sha256:85cbf6cc72b9c535c2f44cec2b2145c8fb940d214874c23c918adfc66a51769c`) |
+| base images | build stage `ghcr.io/tenstorrent/tt-metal/tt-metalium/ubuntu-22.04-dev-amd64:latest` @ `sha256:df9d279c7f85c17c6fad982d196802682d669cca1b7ced9cbaad8181339cd5fc`; runtime stage `docker.io/library/ubuntu:22.04` @ `sha256:5ec03bb3441e8b0bf3b4f9cd4629a1ae763010dc3035bb8da3ae6cf026486401` (tt-model's `FROM` tags float; these are the digests this build resolved, see [`build_info.json`](build_info.json)) |
+| built | 2026-10-09T17:04:34+00:00 by tt-model 0.1.0 |

SERVING.md

@@ -0,0 +1,211 @@
+# Serving FRNet (Autoware lidar_frnet) on Blackhole with tt-model-manager
+
+This repo is the authoring source of the **tt-model container package** `changh95/frnet-p150`
+(`kind: tt-dit-server`, schema 5.1): `tt-model.yaml` is the manifest, `code/` the port, and
+`code/tt_frnet/server/app.py` the ASGI app uvicorn runs. Weights are a pinned pointer, never baked into the image.
+The device open, the decoders, the Python-API contract and the HTTP contract come from `code/tt_frnet/ttaw/`, the
+shared package of the Autoware ports vendored into this repo (`ttaw/VENDORED.json` records its version and file
+hashes; it is not edited here).
+
+| item | value |
+|---|---|
+| tt-metal tree | `/home/ubuntu/experiments/tt-models/tt-metal` (main `44d66500520`, `v0.80.0-dev20261006-78-g44d66500520`, + `patches/tt-metal-eth-dispatch.patch`; torch 2.11.0+cpu) |
+| weights | `AutowareFoundation/lidar_frnet` @ `a53a1c11b0b28d31fd50a03abef5db0e830a5b3c` (tag `v2.0`; files `frnet_*.onnx, ml_package_frnet_*.param.yaml, deploy_metadata.yaml`; 85 MB (frnet_ot128.onnx + frnet_qt128.onnx, 42.4 MB each); Apache-2.0; public, ungated) |
+| Autoware reference | `perception/autoware_lidar_frnet` @ autoware_universe `9ceaccf026c31ffc5319bc9eeb4bd7bede0af3fd` |
+| app | `tt_frnet.server.app:app` (uvicorn; the ASGI lifespan does weights -> device -> graph -> trace capture) |
+| device recipe | `ttnn.open_device(device_id, dispatch_core_config=DispatchCoreConfig(ETH), l1_small_size=65536, trace_region_size=256 MiB, num_command_queues=1)` (`DEVICE_DEFAULTS` in `code/tt_frnet/device.py`; the trace uses 8.7 MiB (ot128) / 5.6 MiB (qt128)) |
+| hardware | one Blackhole p150 (`hardware: p150`, `mesh_device: P150`, `TT_MESH_SHAPE=1x1`), 12×10 compute grid |
+| serve profiles | `ot128` (default; Hesai OT128, `FRNET_VARIANT=ot128`) and `qt128` (Hesai QT128); one variant per process. Both pin `FRNET_DISPATCH=eth`, `FRNET_NUM_CQS=1` and the precision knobs `FRNET_POINT_TERMS=2`, `FRNET_MAP_DTYPE=float32`, `FRNET_CONV_TERMS=1` |
+
+## 1. Run on the HOST (hardware validation, no Docker)
+
+The tt-metal `python_env` has ttnn and torch; put the HTTP stack in a side directory instead of installing into it.
+On the shared workspace box every command that opens the chip goes through the device lock (`bin/devrun`).
+
+```bash
+ROOT=/home/ubuntu/experiments/tt-models
+source $ROOT/bin/tt-env.sh                     # TT_METAL_HOME, PYTHONPATH, tt-metal python_env
+mkdir -p /tmp/frnet-http && uv pip install --python $TT_METAL_HOME/python_env/bin/python \
+    --target /tmp/frnet-http fastapi uvicorn onnx
+cd $ROOT/bundles/frnet-p150
+export PYTHONPATH=$PWD/code:$TT_METAL_HOME:$TT_METAL_HOME/ttnn:/tmp/frnet-http
+export HF_MODEL=AutowareFoundation/lidar_frnet TT_MODEL_WEIGHTS_REVISION=a53a1c11b0b28d31fd50a03abef5db0e830a5b3c TT_MESH_SHAPE=1x1
+
+# host tests (no device; the device tests are skipped; ttnn is the fake of common/tests/host: a real ttnn host
+# tensor opens the chip, so only devrun jobs may build one)
+PYTHONPATH=$ROOT/research/packaging/scripts:$PYTHONPATH TT_VISIBLE_DEVICES=none \
+    python -m pytest -q -p no:cacheprovider -p fake_ttnn_plugin code/tt_frnet/tests
+# device tests and the server, under the lock
+$ROOT/bin/devrun -t 1800 -- python -m pytest -q -s code/tt_frnet/tests/test_pcc_device.py code/tt_frnet/tests/test_e2e_device.py
+# (FRNET_VARIANT=qt128 for the second profile; the smoke test posts the shipped sample of the served variant)
+$ROOT/bin/devrun -t 1800 -- bash -c 'python -m uvicorn --host 127.0.0.1 --port 20000 --lifespan on tt_frnet.server.app:app & \
+    S=$!; python code/tt_frnet/server/smoke_test.py --url http://127.0.0.1:20000 --wait 1200; R=$?; kill -TERM $S; wait $S; exit $R'
+```
+
+Boot log landmarks (they drive `tt-model serve`'s progress view, `boot_progress.py` TT_DIT_PHASES):
+`Loading weights` -> `Opening device` -> `Warming up: capturing trace ...` -> `Warmup complete` -> uvicorn
+`Application startup complete`. A cold JIT cache compiles kernels for minutes; later boots take seconds. Startup
+failures raise and uvicorn exits non-zero (no CPU fallback). SIGTERM: the lifespan releases the traces and closes the
+chip (within `tt-model stop`'s 120 s budget).
+
+Offline override (no Hub): `FRNET_WEIGHTS_DIR=<dir holding frnet_*.onnx, ml_package_frnet_*.param.yaml, deploy_metadata.yaml>`.
+
+## 2. Package, serve, push (Docker)
+
+Docker 29 on this box is **rootful** (`/var/lib/docker`), and the user's session may not carry the `docker` group:
+wrap every docker-using command in `sg docker -c "..."` (no `docker-env.sh` is needed here; the reference bundles'
+`docker-env.sh` was the authors' rootless-Docker box). Run every `tt-model` command **from this directory**
+(`source.tt_metal` and `extra_code[].root: code` are CWD-relative) and point `--out` outside it.
+
+```bash
+ROOT=/home/ubuntu/experiments/tt-models; cd $ROOT/bundles/frnet-p150
+TTM=$HOME/.local/share/uv/tools/tt-model/bin/python
+
+# offline validation (seconds; no docker, no device) -- must print VALID
+$TTM -c "from tt_kernel.container_manifest import load_container_manifest as L; m=L('tt-model.yaml', check_sources=True); p=m.resolve_profile(); print('VALID', m.name, m.kind, p.hardware, p.mesh_device, m.weights_ref)"
+
+# build (one at a time on the box: the absolute lock file): ~35 min cold, ~12 min after a code/ change (the C++
+# build re-runs, ccache-warm, whenever code/ changes), ~1 min for a manifest- or lock-only change
+flock $ROOT/.package.lock sg docker -c "tt-model package --container tt-model.yaml --out $ROOT/build"   # log: ~/.cache/tt-model/build/frnet-p150.log
+# provenance: the base-image digests this build resolved (tt-model's FROM tags float) -> build_info.json (published)
+sg docker -c "python3 $ROOT/research/packaging/scripts/record_build_info.py --staged $ROOT/build/frnet-p150 --bundle ."
+
+# serve + smoke + stop, all inside ONE device-lock window, once per serve profile (`tt-model profiles
+# $ROOT/build/frnet-p150/tt_kernel_manifest.json` lists them; serve returns once READY and leaves the container
+# running; the script's trap saves the container log and always runs `tt-model stop`, also when devrun's timeout
+# fires: -k 150 > the 120 s grace). No token reaches the public weights download. Evidence: logs/smoke/ (--log-dir).
+for prof in ot128 qt128; do
+  $ROOT/bin/devrun -t 3600 -k 150 -- env -u HF_TOKEN -u HUGGING_FACE_HUB_TOKEN \
+      sg docker -c "bash code/scripts/container_smoke.sh $ROOT/build/frnet-p150 20000 --profile $prof"
+done
+
+# publish: overlay the hand-written docs, the pip project and the build record onto the staged dir, then push
+# (HF_TOKEN only from the environment)
+rsync -a README.md SERVING.md OPT_BASELINE.md OPT_REPORT.md VERIFICATION_*.md tt-model.yaml pyproject.toml build_info.json \
+    examples media patches $ROOT/build/frnet-p150/
+cp code/PYTHON.md $ROOT/build/frnet-p150/PYTHON.md 2>/dev/null || true
+tt-model push $ROOT/build/frnet-p150 --publish
+```
+
+The smoke test fails unless `/info` reports ETH dispatch, the 12x10 grid and what the staged package pins for the
+profile (`FRNET_DISPATCH`, `FRNET_NUM_CQS`, `FRNET_VARIANT`, the weights revision), and it compares the
+served output of the shipped sample of the profile (`samples/synthetic_<variant>.npz`) with its stored CPU-reference
+output (`samples/synthetic_<variant>.<variant>.reference.json`; >= 98.5 % label and filter agreement).
+
+`serve` publishes the first free port from 20000 and exports into the container: `HF_MODEL=AutowareFoundation/lidar_frnet`,
+`MESH_DEVICE=P150`, `TT_MESH_SHAPE=1x1`, `TT_MODEL_WEIGHTS_REVISION=a53a1c11b0b28d31fd50a03abef5db0e830a5b3c`, then `serve.env`. The HF cache is
+mounted at `/hf` (rw), the JIT cache at `/cache` (host `~/.cache/tt-model/frnet-p150/cache`), and the container
+sees only its own chip (`--device /dev/tenstorrent/<n>`). tt-cli users: `tt serve changh95/frnet-p150` /
+`tt model stop changh95/frnet-p150` (point tt at this tt-model with
+`tt config set tools.override.tt-model ~/.local/bin/tt-model` or `TT_TOOL_BIN_TT_MODEL`).
+
+What `push` does to the repo: `code/` and `image/` on the Hub become exactly the staged trees (`extra_code.paths` +
+`models/common/lightweightmodule.py`); every top-level file of the staged dir is uploaded (the overlaid README
+replaces the generated card; `pyproject.toml` and `build_info.json` arrive the same way, never through `code/`);
+top-level files already on the Hub and not in the staged dir are kept.
+
+## 3. The request / response contract
+
+| route | purpose |
+|---|---|
+| `GET /health` | `{"status": "ok" \| "starting" \| "error", "model", "device": {"dispatch", "grid", "cores", "device_id"}, "error"}`, always 200; `ok` only after warm-up |
+| `GET /v1/health` | same as `/health` (tt-model's hint for non-chat packages points here) |
+| `GET /info` | model, task, io, Autoware package + commit, weights {repo, tag, revision, path, license}, device (dispatch, grid), input limits (point formats, fields, camera order, calibration presets), labels, variant, warm variants, runtime params + defaults, compile params, warm-up and boot times |
+| `GET /v1/models` | `{"object": "list", "data": [{"id": "AutowareFoundation/lidar_frnet", "object": "model", "owned_by": "changh95"}]}` -- a stub so OpenAI-shaped probes do not 404; NOT a chat API |
+| `POST /predict` | one frame -> the model output (below) |
+
+### 3.1 Request (`application/json`; unknown fields are a 422)
+
+| field | type | meaning |
+|---|---|---|
+| `points` | object | `{"format": "bin" \| "npy" \| "npz" \| "pcd" \| "list", "data": <base64 file bytes>, "fields": [...], "dtype", "key", "frame_id"}`. `bin` = raw little-endian float32 rows, N × len(`fields`) (default fields `x, y, z, intensity`); `npy` / `npz` = (N, C) float or structured arrays; `pcd` = PCD v0.7 ascii / binary / binary_compressed; `list` = `"values": [[x, y, z, i], ...]` (small clouds). ONE sweep in the **sensor frame** (x forward, y left, z up, metres), raw intensity (0-255 for uint8 drivers), not motion-compensated: Autoware's `~/input/pointcloud`. Rows with a non-finite x / y / z are dropped |
+| `calibration` | object | `{"T_base_link_from_sensor": <transform>}` (the static transform the node looks up once for its ego crop box) or `{"preset": "<name>"}` (presets in `/info`: `synthetic_ot128`, `synthetic_qt128`, the shipped samples' transforms); optional `"ego_crop_box"`: `[min_x, min_y, min_z, max_x, max_y, max_z]` in `base_link` (default Autoware's `(-1.53, -1.15, -0.1)` .. `(6.1, 1.15, 3.1)`) or `null` to disable the box (then no transform is needed). Required while the box is on: without it the node would skip every frame, here it is a 400 |
+| `params` | object | per-request knobs (host post-processing only; they change `filter_keep`, never the classes): `class_probability_threshold` (float in [0, 1], default 0.05), `filter_classes` (comma-separated class names, `""` = none, default `"drivable_surface"`, at most 16) |
+| `output_format` | str | `json` (default) or `npz` (adds lossless base64 NPZ arrays) |
+| `sweeps`, `images`, `stream`, `inputs` | – | fields of the collection's common envelope that FRNet does not take: a request with any of them is a 400 |
+
+Transforms (`T_<to>_from_<from>`): a 4×4 (or 3×4) row-major matrix, `{"translation": [x, y, z], "rotation_wxyz": [w, x, y, z]}`
+(or `rotation_xyzw`), or `{"x", "y", "z", "roll", "pitch", "yaw"}` (Autoware `sensor_kit_calibration.yaml`, tf2 RPY).
+Intrinsics: 3×3 K, 3×4 P, or `{"fx", "fy", "cx", "cy"}`. Base64 may be standard or URL-safe, wrapped, with or without a
+`data:` prefix.
+
+### 3.2 Response (200)
+
+The shipped OT128 sample (`synthetic_ot128.npz` with the preset `synthetic_ot128`), abridged (`data` fields cut):
+
+```json
+{"model": "frnet-p150", "frame_id": "sensor", "num_points": 52613, "num_points_raw": 52613,
+ "labels": {"format": "npz", "key": "labels", "dtype": "uint8", "shape": [52613], "data": "..."},
+ "scores": {"format": "npz", "key": "scores", "dtype": "float32", "shape": [52613], "data": "..."},
+ "points": {"format": "npz", "key": "points", "dtype": "float32", "shape": [52613, 4], "data": "..."},
+ "filter_keep": {"format": "npz", "key": "filter_keep", "dtype": "bool", "shape": [52613], "data": "..."},
+ "input_index": {"format": "npz", "key": "input_index", "dtype": "int32", "shape": [52613], "data": "..."},
+ "class_names": ["drivable_surface", "other_flat_surface", "sidewalk", "manmade", "vegetation", "car", "..."],
+ "class_counts": {"drivable_surface": 2913, "other_flat_surface": 20118, "sidewalk": 5, "manmade": 22898, "vegetation": 6526, "car": 13, "semi_trailer": 140, "...": 0},
+ "meta": {"variant": "ot128", "num_points_input": 53411, "cropped_by_ego_box": 798, "num_points_interpolated": 0, "num_unique_coors": 52613,
+          "ego_crop": {"enabled": true, "bounds": [-1.53, -1.15, -0.1, 6.1, 1.15, 3.1], "reference_frame": "base_link", "...": "..."},
+          "plan": {"n": 52613, "m": 52613, "n_cap": 160000, "m_cap": 60000, "...": "..."},
+          "filter": {"classes": ["drivable_surface"], "class_probability_threshold": 0.05, "kept": 44747}, "...": "..."},
+ "timing_ms": {"decode": 24.711, "preprocess": 47.664, "device": 229.164, "postprocess": 5.018, "model_call": 281.879, "total": 310.689}}
+```
+
+- Output point order (the node's): the input points that survive the ego crop box, in input order (after non-finite
+  rows are dropped), then the interpolated points Autoware appends; the first `num_points_raw` rows are raw points and
+  `input_index` maps them back to the decoded input rows. The interpolated points are published, as Autoware does.
+- `labels` is Autoware's `class_id`, `scores` its `probability` (the softmax probability of that class), `points` the
+  x, y, z, intensity of every output point in the sensor frame, `filter_keep` the membership of Autoware's
+  `~/output/pointcloud/filtered` cloud.
+- An out-of-profile frame (input points, points after interpolation or unique cells outside the variant's ranges:
+  ot128 5,000-160,000 points, qt128 5,000-120,000, both 3,000-60,000 cells) answers 200 with `num_points: 0` and the
+  node's message in `meta.skipped`: the node publishes nothing for such a frame. Nothing runs on the device.
+- `meta` also has the crop and network-input counts (`num_points_input`, `cropped_by_ego_box`, `num_points_interpolated`,
+  `num_unique_coors`, ...), the device plan sizes and the applied filter.
+
+`timing_ms`: `decode` (base64 + parsing), `preprocess`, `device` (staging + H2D + trace + D2H + un-permute),
+`postprocess`, `model_call`, `total` (server side, after the body arrived).
+
+### 3.3 Errors
+
+**400** undecodable / malformed input (bad base64, wrong row size, no calibration while the ego crop box is on, unknown
+or out-of-range `params` such as an unknown filter class, `sweeps` / `images` / `stream` / `inputs`, no `points`), **422** schema violation (unknown field, wrong type), **503** while starting
+(or when the boot failed), **500** `inference failed: <ExceptionType>: <message>` if the device call raises.
+
+### 3.4 Environment the app reads (lifespan only, never at import)
+
+| var | set by | meaning / default |
+|---|---|---|
+| `HF_MODEL` | launcher (`weights.repo`) | weights repo id; default `AutowareFoundation/lidar_frnet` |
+| `TT_MODEL_WEIGHTS_REVISION` | launcher (`weights.revision`) | pinned commit; `TT_WEIGHTS_REVISION` (`serve.env`) is the same for older tt-model |
+| `FRNET_WEIGHTS_DIR` | you (offline) | local weights directory; overrides the Hub |
+| `TT_MESH_SHAPE` | launcher (`runtime.mesh_shape_env`) | `1x1`; anything else -> RuntimeError at startup |
+| `TT_DEVICE_ID` | you | chip to open, default 0 |
+| `FRNET_DISPATCH` | `serve.env` | `eth` (default) \| `worker` (A/B only) \| `auto` (ETH if the patch is present) |
+| `FRNET_NUM_CQS` | `serve.env` | `1` |
+| `FRNET_VARIANT` | `serve.env` / profile | `ot128` (profile `ot128`) or `qt128` (profile `qt128`) |
+| `FRNET_POINT_TERMS`, `FRNET_MAP_DTYPE`, `FRNET_CONV_TERMS` | `serve.env` | the precision knobs: `2`, `float32`, `1` (the validated configuration; other values are optimization-phase A/B switches that fail the frozen accuracy gates) |
+| `FRNET_WARMUP` | you | JSON list of warm-up variants, `default` or `none` |
+| `FRNET_TRACE_REGION`, `FRNET_L1_SMALL`, `FRNET_WORKER_L1_SIZE` | you | device-open overrides (validated values: `DEVICE_DEFAULTS` in `device.py`) |
+| `FRNET_MAX_BODY_MB` | you | request size guard, default 256 |
+| `TT_METAL_VISIBLE_DEVICES`, `MESH_DEVICE` | `serve.env` / launcher | `0`, `P150` (informational) |
+
+Server-side pipeline: JSON -> `tt_frnet.io` decoders -> `FRNet.__call__` under one lock: host pre-processing ported
+from `perception/autoware_lidar_frnet` (ego crop, float32 projection, fp16 interpolation grid, unique cells, profile
+checks, the device plan: sorted points, CSR offsets, gather tables, 8-d features) -> 9 H2D copies into persistent
+device inputs -> `execute_trace` (point MLP, K1 segment max, frustum CNN, point head, softmax) -> one D2H of the fp32
+probabilities -> un-permute -> host post-processing (first-max argmax, probability, class filter) ->
+`Output.to_dict()`.
+
+## 4. Caveats
+
+- Batch 1, one sweep per request; concurrent clients queue on the lock.
+- Fixed device shapes per profile: `N_cap` 160,000 (ot128) / 120,000 (qt128) points and `M_cap` 60,000 cells; every
+  point-wise op runs on all `N_cap` rows, so the device time hardly depends on the frame. Frames outside Autoware's
+  profile ranges are skipped as the node skips them (200, empty output, `meta.skipped`).
+- One variant per process: serve `--profile qt128` for the Hesai QT128 network. The profiles share the image.
+- Warm-up captures every trace variant inside the lifespan, so READY means warm; the first cold boot pays the ttnn JIT.
+- Weights are pinned by sha in `weights.revision`, exported by the launcher and repeated in `serve.env`;
+  `snapshot_download(..., revision=<sha>)` is a cache hit after `serve`'s pre-download and falls back to
+  `local_files_only=True` if the Hub is unreachable.
+- The runtime image has no host C/C++ compiler: device kernels JIT-compile (sfpi ships in the image), host-side C
+  helpers must have a numpy fallback.
+- `tt-model curl` and the ready card's `/v1/models` hint are OpenAI-shaped and are not this API; use the routes above.

VERIFICATION_2026-10-09.md

@@ -0,0 +1,166 @@
+# frnet-p150: independent verification, 2026-10-09
+
+## Verdict
+
+**PASS.** This is the light verification pass of the first publish (`research/PLAN.md` §6.3): the frozen gates
+re-run, the baseline numbers re-checked, and every device-open path audited for ETH dispatch. It was made by the agent
+that wrote the release docs, which is neither the porter, nor the adversarial verifier of `VERIFY_PORT.md`, nor the
+baseline agent, on the release-code commit `04c39b5` (the model code of the baseline `0dd52b2`; the shipped sample
+became a synthetic scan per serve profile, the PandaSet sample is staged and not shipped, test hardening; the docs
+commit that follows changes `make_demo.py`'s render captions and the docs only). The device runs ran on 2026-10-09
+15:53-16:10 UTC in one `bin/devrun` window (a first queueing attempt died while waiting for the lock, when the
+workspace's `bin/devrun` was edited under it; it had not touched the device). Against the baseline commit `0dd52b2`:
+
+- **Frozen gates:** the device suite (`test_pcc_device.py` + `test_e2e_device.py`, one pytest) passed **14 / 14**
+  tests. Every existing gate passes with values **bit-identical** to the baseline run (the `_m5` and `_m5docs` reports:
+  2,144 leaves compared, 0 differ; timings excluded), including the binding frame: OT128 min agreement 98.907 %
+  (nuScenes 0916 kf40) against the 98.5 % gate. The two new tests (`test_shipped_sample`, and `test_public_frames`
+  now requiring all 17 / 5 frames) passed; the four new shipped-sample gates were frozen at this first green run
+  (label agreement OT128 0.99550, QT128 0.99623; filtered-cloud agreement 0.99772 / 0.99872; threshold 0.985).
+- **Baseline re-check:** the device rows of `OPT_BASELINE.md` repeat to within 0.03 ms (OT128 210.02 / 209.95 ms
+  trace / b2b on the synthetic sample vs 210.00 / 209.95 ms now; QT128 133.53 / 133.43 vs 133.50 / 133.43 ms). Host
+  rows within the load noise (OT128 `model()` 292.6 -> 289.3 ms p50 on the synthetic sample, 355.7 -> 360.1 ms on
+  PandaSet, at load average 4.1).
+- **Shipped samples:** the shipped `synthetic_<variant>.npz` are byte-identical to the baseline's synthetic bench
+  inputs (`cmp`), so the baseline's synthetic rows are the shipped sample's. p150 vs the stored CPU reference:
+  OT128 99.550 % of 52,613 points, QT128 99.623 % of 69,553, the same through `bench.py`, the device test, the
+  quickstart, `make_demo.py` and the served smoke test.
+- **Public data (113 frames per profile, production API):** pooled class agreement OT128 PandaSet 99.66 % / nuScenes
+  99.19 %, QT128 99.29 % / 99.44 %; identical scored rows to the CPU reference on all 226 frame-profile pairs, no
+  profile skip. 3 pairs fall below 98.5 % (not gated; see "Review findings"). Mapped-label mIoU on the p150 equals the
+  CPU reference's within 0.002.
+- **ETH dispatch:** every device-open path opens ETH / 1 CQ / 12×10 by default; every run of this pass printed
+  `dispatch eth`, `grid 12x10`, `cores 120`, `fallback null`, `eth_patch true` (both serve profiles, API and server).
+- **No numerics change** and no new bug: nothing in the model code was changed by this pass.
+
+## Setup
+
+| item | value |
+|---|---|
+| Hardware | one Blackhole p150b, ETH dispatch, 1 CQ, 12×10 compute grid (120 cores), AICLK 1350 MHz median (min 1343) during the benches |
+| tt-metal | `44d66500520` with `patches/tt-metal-eth-dispatch.patch` (sha256 `08d0ddf6…`, byte-identical to the workspace copy) |
+| Weights | `AutowareFoundation/lidar_frnet` @ `a53a1c11b0b28d31fd50a03abef5db0e830a5b3c` (tag `v2.0`): the device suite and the demo run read the sha256-pinned workspace copy; the quickstart, the served runs and the load-time runs resolved the HF cache at the pinned revision offline (downloaded without a token for this pass; every file identical to the workspace copy, ONNX sha256 ot128 `d29bf595…`, qt128 `647af927…`) |
+| Verified commit | `04c39b5` (ttaw 0.20.0, `common` `89dec49`, `source_dirty: false`) |
+| Baseline commit | `0dd52b2` (`OPT_BASELINE.md`) |
+| Workload | the shipped samples `code/tt_frnet/samples/synthetic_{ot128,qt128}.npz` and the staged PandaSet 019 f40 (107,146 points, local), batch 1, warm; the 17 / 5 gate frames; 113 converted public frames per profile (local only) |
+| Bench method | `code/scripts/bench.py --iters 100 --warmup 10 --b2b-iters 50 --b2b-rounds 5`, one run per serve profile (the baseline's settings), with each output checked against its stored CPU reference |
+| Served method | uvicorn on the host with the serve pins of `tt-model.yaml` (the container's app; `FRNET_VARIANT` per profile), the bundle's `smoke_test.py` against it, then 5 warm-up + 50 requests of the shipped sample from a loopback client, the README's `client.py` + `curl` recipe, a knobs request and a request without calibration |
+| Accuracy reference | the fp32 CPU reference (`code/tt_frnet/reference/`, ONNX Runtime on the deployed ONNX; VERIFY_PORT: within 1e-11 of an independent ORT run); for the 113-frame sets the research ORT runs (`research/frnet/public_data/runs`) |
+| Host | AMD EPYC VM, 8 vCPUs, shared with other agents' jobs (load average 4.1-6.1 during the runs) |
+| Logs | `logs/frnet/m5_docs/` of the workspace (job scripts in `scripts/`) |
+
+## Measured baseline vs re-check
+
+All times in ms, p50 of 100 iterations (b2b: 5 rounds of 50 replays). Baseline: `OPT_BASELINE.md` (`0dd52b2`);
+re-check: `code/scripts/bench.py` with the baseline's settings on `04c39b5`
+(`logs/frnet/m5_docs/bench_recheck_{ot128,qt128}.json`).
+
+| row | baseline | re-check |
+|---|---:|---:|
+| OT128, shipped sample: trace, one replay / back-to-back | 210.02 / 209.948 | 210.00 / 209.946 |
+| OT128, PandaSet 019 f40: trace / back-to-back | 211.72 / 211.643 | 211.73 / 211.639 |
+| QT128, shipped sample: trace / back-to-back | 133.53 / 133.434 | 133.50 / 133.429 |
+| QT128, PandaSet 019 f40: trace / back-to-back | 134.91 / 134.866 | 134.92 / 134.856 |
+| OT128 H2D / D2H, shipped sample | 5.42 / 7.65 | 5.42 / 7.56 |
+| OT128 host pre-processing, shipped sample / PandaSet | 55.3 / 107.2 | 54.7 / 107.4 |
+| OT128 `model()` p50, shipped sample / PandaSet | 292.6 / 355.7 | 289.3 / 360.1 |
+| QT128 `model()` p50, shipped sample / PandaSet | 221.1 / 244.3 | 205.4 / 241.5 |
+| served `/predict` `timing_ms.total`, OT128 / QT128 (median of 50) | – | 310.4 / 228.9 |
+| served client round trip, loopback | – | 328.1 / 247.3 |
+| `from_pretrained`, empty JIT cache / warm cache, OT128 | – / 6.5 s | 156.6 / 7.5 s |
+| `from_pretrained`, empty JIT cache / warm cache, QT128 | – / 6.9 s | 151.9 / 6.6 s |
+
+The served and cold-load rows are new for the card (the baseline measured neither). The first two calls after a load
+took 320.0 / 304.7 ms (OT128) and 241.1 / 214.9 ms (QT128): the trace is captured during the load. The cold JIT cache
+holds 682 MB (OT128) / 644 MB (QT128) after the load.
+
+## Accuracy gate results
+
+`logs/frnet/m5_docs/suite.log`; reports `logs/frnet/{pcc,pcc_small,e2e,latency}_{ot128,qt128}_m5docs.json`.
+
+| check | gate | OT128 | QT128 |
+|---|---|---|---|
+| every gated module tap (PCC vs the fp32 reference, PandaSet sample, replay outputs) | ≥ 0.999 | min 0.999729 (`point_fusion1`) | min 0.999649 (`attention4`) |
+| `pred_probs` PCC / raw-point agreement, PandaSet sample (staged) | ≥ 0.998 / ≥ 98.5 % | 0.999942 / 99.581 % | 0.999844 / 99.295 % |
+| public frames (17 / 5): min `pred_probs` PCC | ≥ 0.998 | 0.999688 | 0.999805 |
+| public frames: min raw-point agreement | ≥ 98.5 % | **98.907 %** (nuScenes 0916 kf40) | 99.158 % (PandaSet 029 f40) |
+| shipped sample vs its stored reference body: labels / filtered-cloud mask (new) | ≥ 98.5 % | 99.550 % / 99.772 % | 99.623 % / 99.872 % |
+| replay == eager | bit-exact | 45 / 45 taps | 45 / 45 taps |
+| API == `/predict` handler | identical | yes | yes |
+
+Host suite (fake ttnn, gates read-only): **117 passed, 15 skipped** (the 14 device tests and the onnxruntime-only
+test), `logs/frnet/m5_docs/host_suite.log`. Research venv (ONNX Runtime + torch): **85 passed, 2 skipped** (by design),
+`logs/frnet/m5_docs/research_venv.log`. The counts differ from VERIFY_PORT's (112 / 13 host, 83 / 2 research venv) by the tests the release added or
+changed (release hygiene of the shipped samples, the top-level `PYTHON.md` copy, the shipped-sample device test).
+
+### Public-frame agreement beyond the gates
+
+`code/scripts/make_demo.py run` (the production API, `model(points, calibration=...)`) on every converted public frame
+of `research/frnet/public_data/inputs`, both profiles, against the research ORT runs
+(`logs/frnet/m5_docs/demo/agreement.json`):
+
+| set | frames | scored raw points | OT128 pooled (worst frame) | QT128 pooled (worst frame) |
+|---|---:|---:|---:|---:|
+| PandaSet 019 + 029, every 5th frame | 32 | 3,442,737 | 99.66 % (99.35 %) | 99.29 % (98.69 %) |
+| nuScenes v1.0-mini mini_val, all key-frames | 81 | 2,067,589 | 99.19 % (98.37 %) | 99.44 % (98.10 %) |
+
+Scored rows equal the reference's on all 226 frame-profile pairs (same crop, same profile decisions); no frame was
+skipped. Mapped-label mIoU (`research/frnet/public_data/scripts/frnet_public_eval.py`, not a paper metric;
+`logs/frnet/m5_docs/miou/`): OT128 PandaSet coarse 0.7136 (p150) vs 0.7146 (CPU), fine 0.4471 vs 0.4479; nuScenes
+coarse 0.4827 vs 0.4831, fine 0.4515 vs 0.4531 (fine = mean over the classes with at least 1,000 labelled points,
+`miou_classes_ge_1000_gt_points`; over all labelled classes: PandaSet 0.3229 vs 0.3235, nuScenes 0.3285 vs 0.3292); QT128 PandaSet coarse 0.1869 vs 0.1873, nuScenes 0.3255 vs 0.3266.
+
+## Disclosed numerics changes
+
+None: the model code, the precision knobs and the kernels are those of the baseline, and every existing gate value is
+bit-identical to it. The release changed what ships (samples), the tests (stricter) and the docs.
+
+## Review findings
+
+- **VERIFY_PORT round 1, low findings** (all resolved or disclosed):
+  1. stale PORT_LOG header and latency: fixed (ttaw 0.20.0 @ `89dec49`; the superseded first-cut latency replaced by
+     the current numbers; the duplicated known-issue paragraphs removed);
+  2. `test_public_frames` skipped missing goldens silently: inside the workspace a missing golden, research frame or
+     staged sample now FAILS (`tests/_data.py need()`), and the test asserts 17 / 5 frames;
+  3. narrow precision margin (0.41 pt on nuScenes 0916 kf40): disclosed on the card (Caveats) and in `OPT_REPORT.md`
+     as the binding constraint and backlog item 14;
+  4. end-to-end latency depends on the host load: the card quotes the stage bench with its load conditions and the
+     device rows separately;
+  5. `test_stage_pcc` replays on its capture inputs: unchanged (hardening note); input rewriting is covered by
+     `test_small_graph`, `test_public_frames` (17 / 5 frames through one trace per profile) and the new `test_shipped_sample`.
+- **New, low:** on the wider (ungated) public set, 3 of 226 frame-profile pairs agree below 98.5 %: nuScenes
+  scene-0916 key-frames 11 (98.374 %) and 1 (98.499 %) on OT128, key-frame 0 (98.105 %) on QT128. They are on the same
+  out-of-domain scene (32-beam HDL-32E, Singapore) as the binding gated frame, are consistent with its 0.41-pt margin,
+  and are disclosed on the card. They do not touch a frozen gate. The optimization phase should add them to the
+  frames it judges precision changes on.
+- **Gates:** no threshold was changed; the gate JSON gained only the four shipped-sample entries (0.985, the PLAN.md 2.8
+  argmax gate). No test compares TT with TT except replay == eager.
+- **Samples and data:** `code/` ships only the two synthetic scans with their presets and references
+  (`test_only_redistributable_samples_ship`); no PandaSet, nuScenes or rosbag data is tracked; the PandaSet sample
+  lives in the git-ignored `staging_samples_pandaset/`. nuScenes appears only as two renders labelled non-commercial.
+
+## p150 ETH-dispatch compliance
+
+Every path that opens the device goes through `ttaw.device.open_device`, whose default is
+`DispatchCoreConfig(ETH)` (falling back to WORKER only with a `RuntimeWarning`, or never with `allow_fallback=False`):
+
+| path | how it opens | dispatch | evidence |
+|---|---|---|---|
+| `FRNet.from_pretrained` (API, quickstart, bench, profile, make_demo) | `ModelBase.device_config` -> `DeviceConfig` (`DEVICE_DEFAULTS` < `FRNET_*` env < args) -> `open_device` | `eth` default; `FRNET_DISPATCH` / `dispatch=` switch | every run printed `dispatch eth, grid 12x10, fallback None` |
+| HTTP server (`tt_frnet.server.app`, the container) | the lifespan calls `FRNet.from_pretrained` with the env; `serve.env` pins `FRNET_DISPATCH=eth`, `FRNET_NUM_CQS=1` | `eth` (pinned; `check_bundle.py` refuses another value) | served runs: `/info` `dispatch eth`, `grid 12x10`; `smoke_test.py` fails on anything else |
+| device tests (`conftest.py` `device` fixture) | `tt_frnet.device.open_device(allow_fallback=False)` | `eth`, no fallback | suite log |
+| `code/scripts/visualizer_report.py` | `open_device(allow_fallback=False)` | `eth` | baseline log |
+| `code/scripts/diag_conv_precision.py`, `diag_frame_taps.py` | `tt_frnet.device.device_session()` | `eth` default | code read |
+| `tests/test_tt_host.py` | `ttaw.device.open_device(0, dispatch="eth")` on the fake ttnn only | – (no device) | host suite |
+
+The grid is never hard-coded: it comes from `device.compute_with_storage_grid_size()` (`tt/layers.py`, K1
+`segment.py`); the only 12×10 literal is the expected-grid warning in `ttaw/device.py` and the smoke test's assertion.
+The image's `verify:` line asserts the patch marker in `tt_metal/impl/dispatch/topology.cpp`.
+
+## Not covered by this pass
+
+- The container image: it is built and smoke-tested per serve profile (`ot128`, `qt128`) at packaging
+  (`code/scripts/container_smoke.sh`), after this pass.
+- A full adversarial verification of the release (that is `VERIFY_PORT.md` for the port; the next full verification
+  follows the first optimization round).
+- TensorRT parity: the reference is the fp32 ONNX Runtime run of the deployed ONNX, not Autoware's fp16 TensorRT engine.

build_info.json

@@ -0,0 +1,48 @@
+{
+ "schema": "ttaw-build-info/1",
+ "bundle": "frnet-p150",
+ "recorded_at": "2026-10-09T17:15:08Z",
+ "image": {
+  "tag": "tt-model/frnet-p150:85cbf6cc72b9",
+  "digest": "sha256:85cbf6cc72b9c535c2f44cec2b2145c8fb940d214874c23c918adfc66a51769c",
+  "built_at": "2026-10-09T17:04:34+00:00",
+  "code_sha256": "f0b62790446a51a86fe941bb4367561fecd9c560adc4d02b9b25b1e364358ede",
+  "size_bytes": 4071619125,
+  "layers": 23
+ },
+ "base_images": {
+  "build": {
+   "stage": "prep",
+   "ref": "ghcr.io/tenstorrent/tt-metal/tt-metalium/ubuntu-22.04-dev-amd64:latest",
+   "digest": "sha256:df9d279c7f85c17c6fad982d196802682d669cca1b7ced9cbaad8181339cd5fc",
+   "source": "docker build log",
+   "local_tag_matches": false,
+   "local_id": "sha256:3fd1e6013e658c65df1bc6b084543e7174d854c15af97373e493df0c1787c393",
+   "created": "2026-10-07T01:17:29.186036377Z",
+   "note": "the local tag moved after this build; the digest above is the one the image was built from"
+  },
+  "runtime": {
+   "stage": "runtime",
+   "ref": "docker.io/library/ubuntu:22.04",
+   "digest": "sha256:5ec03bb3441e8b0bf3b4f9cd4629a1ae763010dc3035bb8da3ae6cf026486401",
+   "source": "docker build log",
+   "local_tag_matches": null,
+   "note": "not in the local image store (BuildKit pulled it into its cache)"
+  }
+ },
+ "tt_metal": {
+  "sha": "44d66500520fda9f2c7060c0f6b41ec48f7ab37e",
+  "describe": "v0.80.0-dev20261006-78-g44d6650052-dirty",
+  "dirty": true,
+  "scm_version": "0.65.2.dev11169+g44d66500520",
+  "mode": "local",
+  "remote": "https://github.com/tenstorrent/tt-metal.git",
+  "branch": "main",
+  "pushed": true
+ },
+ "tools": {
+  "tt_model": "0.1.0",
+  "docker": "29.8.2"
+ },
+ "notes": "tt-model's FROM tags float (build: tt-metalium dev image :latest, runtime: ubuntu:<version>); these are the digests this image was built from. A moved build base costs a cold C++ build."
+}

code/PYTHON.md

@@ -0,0 +1,220 @@
+# Python API: FRNet (Autoware lidar_frnet) on Blackhole
+
+Use this API from Python code (a pipeline, a notebook, a ROS 2 node wrapper). You do not need the HTTP server: the
+API and the server share the decoders, the device graph and the post-processing, so the outputs and the speed are
+the same.
+
+## Install
+
+Install the package on top of an environment that already has `ttnn` (a tt-metal `python_env` at `44d66500520`
+with `patches/tt-metal-eth-dispatch.patch`, or the tt-model container). From the root of the model repository (the
+directory that holds `pyproject.toml`, `README.md` and `code/`):
+
+```bash
+pip install -e .                        # the Python API
+pip install -e ".[server,test]"         # + the HTTP server and the tests
+```
+
+The pip project is the repository's top-level `pyproject.toml`; it installs the package from `code/tt_frnet` (there is no
+`pyproject.toml` inside `code/`, because the container build copies `code/` over the tt-metal tree).
+
+The package carries `tt_frnet.ttaw`, the shared code of the Autoware ports to Blackhole (device open, trace runner,
+decoders, model base class, HTTP app, the K1 `segment_reduce` kernel), vendored at the version recorded in
+`code/tt_frnet/ttaw/VENDORED.json` (0.20.0).
+
+| You want to run | Extras |
+|---|---|
+| the Python API | none |
+| the HTTP server (`tt_frnet.server.app`, see `SERVING.md`) | `server` |
+| host tests (no device; device tests are skipped): `TT_VISIBLE_DEVICES=none python -m pytest -q code/tt_frnet/tests` | `server,test` |
+| device tests: `python -m pytest -q -s code/tt_frnet/tests/test_pcc_device.py code/tt_frnet/tests/test_e2e_device.py` | `test` |
+
+## Quickstart
+
+Run from the repository root (the sample paths are relative to it):
+
+```python
+from tt_frnet import FRNet
+
+with FRNet.from_pretrained(device_id=0) as model:                  # variant="qt128" for the Hesai QT128 profile
+    out = model("code/tt_frnet/samples/synthetic_ot128.npz",       # one sweep in the sensor frame
+                calibration={"preset": "synthetic_ot128"})         # its sensor -> base_link transform (ego crop box)
+
+print(out.class_counts())                                          # points per class (raw + interpolated)
+labels, scores, xyz = out.label_ids, out.scores, out.points[:, :3]
+```
+
+`examples/quickstart.py` runs this snippet and writes `quickstart.json` (the `/predict` body) and `quickstart_bev.png`
+(a bird's-eye view of the p150 classes next to the stored CPU reference of the shipped sample).
+
+## `FRNet.from_pretrained(...)`
+
+```python
+FRNet.from_pretrained(
+    model_id=None,          # HF repo or a local directory with the weights files; default AutowareFoundation/lidar_frnet
+    *,
+    revision=None,          # default for the default repo: the validated commit a53a1c11b0b (tag v2.0)
+    variant=None,           # load-time variant: "ot128" (Hesai OT128, Autoware's default) | "qt128"; default $FRNET_VARIANT or "ot128"
+    device_id=None,         # chip to open; default $TT_DEVICE_ID or 0
+    device=None,            # an already-opened ttnn device (tt_frnet.device.open_device); close() does not close it
+    dispatch=None,          # "eth" (p150 target, 12x10 grid) | "worker" (A/B only, 11x10) | "auto"; default $FRNET_DISPATCH or "eth"
+    num_command_queues=None,  # default $FRNET_NUM_CQS or 1
+    weights_dir=None,       # explicit local weights directory; no Hub access
+    warmup_variants="default",  # trace variants to capture now; see "Warm-up"
+    verbose=False,
+) -> FRNet
+```
+
+What it does: resolves the weights first, so a Hub problem never claims the chip (`weights_dir` >
+`$FRNET_WEIGHTS_DIR` > a local `model_id` directory > the HF snapshot at the pinned revision, restricted to
+`frnet_*.onnx, ml_package_frnet_*.param.yaml, deploy_metadata.yaml`, with an offline fallback to the cache), checks
+the sha256 of the variant's ONNX against the pinned one, reads the variant's `ml_package_frnet_<variant>.param.yaml`
+(FOV, frustum and interpolation grids, the 27 class names, the palette, the point / cell profiles), opens the chip
+(ETH dispatch, 12×10, 1 CQ, `l1_small_size` 64 KiB, `trace_region_size` 256 MiB: `DEVICE_DEFAULTS` in
+`tt_frnet/device.py`, overridable with `FRNET_*`), builds the graph, then compiles and captures the metal trace. If
+ETH dispatch cannot open (tt-metal without the patch), it warns and falls back to WORKER dispatch.
+
+`variant` picks one of the two networks of the weights repo. Each is tied to its sensor (Autoware's `sensor_model`):
+
+| variant | sensor | FOV up / down | frustum grid (H × W) | interpolation grid | points (min / max) | unique cells (min / max) |
+|---|---|---|---|---|---|---|
+| `ot128` (default) | Hesai OT128 | +15° / −25° | 128 × 1024 | 128 × 4096 | 5,000 / 160,000 | 3,000 / 60,000 |
+| `qt128` | Hesai QT128 | +52.6° / −52.6° | 128 × 256 | 128 × 1024 | 5,000 / 120,000 | 3,000 / 60,000 |
+
+Precision knobs (read once at load from the environment, pinned in the package's `serve.env`; the defaults are the
+validated configuration and every number of the model card uses them): `FRNET_POINT_TERMS=2` (each point-wise linear
+as two-term bf16 operands, about 16 mantissa bits, fp32 accumulation), `FRNET_MAP_DTYPE=float32` (fp32 image-branch
+maps, bf16 conv operands), `FRNET_CONV_TERMS=1`. Other values are A/B switches for the optimization phase; they fail
+the frozen accuracy gates (`OPT_BASELINE.md`, "The precision cost of the policy").
+
+## Warm-up
+
+`from_pretrained` returns a warm model: it builds the graph of the variant, runs it once eagerly on an empty frame
+(this first run compiles the kernels and uploads the weights) and captures it as ONE metal trace, so no call compiles
+anything. There are no shape buckets: every frame is padded to the profile's fixed device shapes (`N_cap` points =
+160,000 / 120,000, `M_cap` cells = 60,000), so one trace serves every input of the profile.
+
+- Measured (2026-10-09, the shipped sample): with an empty JIT cache the load took 156.6 s for ot128 and
+  151.9 s for qt128 (firmware and every kernel compiled); with that cache warm, in a new process,
+  7.5 s and 6.6 s. The trace is captured during the load, so the first call costs about as
+  much as the next ones (320.0 / 304.7 ms for ot128, 241.1 / 214.9 ms for qt128, first / second call, on a shared host).
+- `warmup_variants="none"` (server: `FRNET_WARMUP=none`) skips the capture at load; call `model.warmup()` before the
+  first frame (it is idempotent).
+- The JIT cache is tt-metal's (`~/.cache/tt-metal-cache`, or `TT_METAL_CACHE`); inside the container it is
+  `~/.cache/tt-model/frnet-p150/cache` of the host.
+
+## Call: `model(points, calibration=..., **kwargs)`
+
+| Argument | Type | Description |
+|---|---|---|
+| `points` (first positional) | path, `bytes` + `fmt=`, (N, C) array / tensor, structured array, `PointCloud`, JSON envelope | ONE LiDAR sweep in the **sensor frame** (x forward, y left, z up, metres), fields `x, y, z, intensity` with the raw intensity the node casts (0-255 for uint8 drivers); see "Input types" |
+| `calibration` | `{"T_base_link_from_sensor": <transform>}`, `{"preset": name}` (a file of `code/tt_frnet/calib/`), optional `"ego_crop_box"` | the static sensor -> `base_link` transform the node looks up once for its ego crop box. `ego_crop_box`: `[min_x, min_y, min_z, max_x, max_y, max_z]` in `base_link` (default Autoware's `(-1.53, -1.15, -0.1)` .. `(6.1, 1.15, 3.1)`), or `None` (or all zeros) to disable the box, the node's own switch; a disabled box needs no transform |
+| `class_probability_threshold` | float in [0, 1], default 0.05 | the filtered cloud keeps a point unless its probability of a filter class is >= this value (Autoware's `filter.class_probability_threshold`) |
+| `filter_classes` | comma-separated class names, `""` = none, default `"drivable_surface"` | Autoware's `filter.classes` (at most 16); an unknown name is refused |
+
+Both knobs only change `filter_keep`; the class decision (`label_ids`, `scores`) never depends on them. They are
+host-only (no device shape changes), the defaults reproduce Autoware's `frnet.param.yaml`, and a bad value raises
+`tt_frnet.io.InputError` (HTTP 400) before the device runs. `model.info["runtime_params"]` (and the server's `/info`)
+lists them with their defaults.
+
+### Input types
+
+- Point clouds: a path (`.bin` raw float32 rows -- pass `fields=` if not `x, y, z, intensity`; `.npy`; `.npz`;
+  `.pcd` ascii / binary / binary_compressed), raw bytes with `fmt=`, an (N, C) numpy array or torch tensor, a
+  structured array, the JSON envelope of `/predict`, or `tt_frnet.io.PointCloud`. The frame is the **sensor** frame
+  (Autoware's `~/input/pointcloud` before any transform), one sweep, not motion-compensated.
+- Rows with a non-finite x / y / z are dropped by the decoder; `input_index` refers to the decoded rows.
+- Calibration transforms: a 4×4 (or 3×4) row-major matrix, `{"translation": [x, y, z], "rotation_wxyz": [w, x, y, z]}`,
+  or Autoware's `{x, y, z, roll, pitch, yaw}`. With the crop box on and no transform the call is refused (the node would
+  skip every frame because its TF lookup fails); the error names both fixes.
+- `sweeps`, `images`, `stream` and `inputs` are refused: FRNet takes one sweep and its calibration only.
+- A frame outside the variant's profile (input points, points after interpolation, or unique cells out of the table
+  above) is not an error: as the node, which logs and publishes nothing, the call returns an empty output whose
+  `meta["skipped"]` holds the node's message, and nothing runs on the device.
+
+## Output
+
+`model(...)` returns a `tt_frnet.FRNetSegmentation` (a `ttaw.outputs.Segmentation3D`); `out.to_dict()` is exactly the
+`POST /predict` body (SERVING.md section 3.2). The point order is the node's: the input points that survive the ego
+crop box, in input order, then the points Autoware's interpolation appends (D4: published as Autoware does).
+
+| field | type | meaning |
+|---|---|---|
+| `label_ids` | uint8 `[N]` | Autoware's `class_id` (first maximum of the 27 probabilities, as `fill_cloud_kernel`); names in `class_names` |
+| `scores` | float32 `[N]` | Autoware's `probability`: the softmax probability of that class |
+| `points` | float32 `[N, 4]` | x, y, z, intensity of every output point in the sensor frame (the segmentation cloud's coordinates; the interpolated points carry interpolated values) |
+| `filter_keep` | bool `[N]` | membership of Autoware's `~/output/pointcloud/filtered` cloud (False = removed by the class filter) |
+| `num_points_raw` | int | the first `num_points_raw` rows are input points, the rest are interpolated |
+| `input_index` | int32 `[num_points_raw]` | the decoded input row of every raw output point |
+| `class_names` | 27 str | `drivable_surface`, `other_flat_surface`, `sidewalk`, `manmade`, `vegetation`, `car`, `bus`, `emergency_vehicle`, `train`, `truck`, `tractor_unit`, `semi_trailer`, `construction_vehicle`, `forklift`, `kart`, `motorcycle`, `bicycle`, `pedestrian`, `personal_mobility`, `animal`, `pushable_pullable`, `traffic_cone`, `debris`, `stroller`, `other_stuff`, `noise+ghost_point`, `unknown` (Autoware's names; ids 22 / 23 were trained as stroller / debris, see Limits) |
+| `meta` | dict | `variant`; `num_points_input`; `ego_crop` (the box and transform used); the network-input counts (`num_points_raw`, `num_points_interpolated`, `num_points`, `num_unique_coors`, `cropped_by_ego_box`, `crop_applied`, `max_points_per_cell`); `plan` (`n`, `m`, `n_cap`, `m_cap`); `filter` (classes, threshold, points kept); `skipped` (the node's message) for an out-of-profile frame |
+| `timing_ms` | dict | `preprocess`, `device` (staging + H2D + trace + D2H + un-permute), `postprocess`, `total` of this call |
+
+`out.class_counts()` gives the points per class (raw + interpolated); `out.to_dicts()` one row per class
+(`label`, `label_id`, `points`); `len(out)` the number of output points. The node's visualization cloud is
+`model.config.palette` indexed by `label_ids` (`tt_frnet.host.postprocess.visualization_rgb`).
+
+## Lifetime and information
+
+- `model.close()` releases the trace and the device tensors and closes the chip if the model opened it; idempotent.
+  `with` calls it for you; an unclosed model is closed when Python exits.
+- `model.info`: weights (repo, tag, revision, path), device (`dispatch`, `grid`, `cores`, `num_command_queues`,
+  `fallback`), variant, warm variants, warm-up times, runtime parameter defaults, the variant config (`frnet`), the
+  trace and graph descriptions.
+- Calls from several threads are safe: the device calls are serialised by a lock. One model per process per chip.
+- `tt_frnet.viz.render_bev(points, label_ids, palette=...)` and `render_compare(...)` draw bird's-eye views (PIL, no
+  OpenCV).
+- `model.device_probs(plan)` returns the device's `pred_probs` `(N, 27)` of a prepared frame (tests and tools; call it
+  under `model._lock`).
+
+## Speed
+
+Warm calls, batch 1, ETH dispatch, 1 CQ, 12x10, the serve.env precision knobs (`code/scripts/bench.py`, 100
+iterations; the numbers of `OPT_BASELINE.md`, 2026-10-09, on a shared host whose load moves the host stages):
+
+| stage, p50 | ot128, shipped sample | ot128, PandaSet 019 f40 | qt128, shipped sample | qt128, PandaSet 019 f40 |
+|---|---:|---:|---:|---:|
+| `model(path)` end to end | 292.6 ms | 355.7 ms | 221.1 ms | 244.3 ms |
+| host pre-processing (decode, crop, projection, fp16 interpolation, unique cells, device plan) | 55.3 ms | 107.2 ms | 54.2 ms | 83.5 ms |
+| H2D (9 inputs, 8.6 / 6.2 MB) | 5.42 ms | 5.41 ms | 4.04 ms | 4.00 ms |
+| device trace, one forward | 210.02 ms | 211.72 ms | 133.53 ms | 134.91 ms |
+| D2H (fp32 `pred_probs`) | 7.65 ms | 7.43 ms | 5.80 ms | 5.55 ms |
+| un-permute + host post-processing (argmax, filter) | 7.1 + 5.0 ms | 12.8 + 14.8 ms | 6.1 + 6.8 ms | 5.9 + 10.5 ms |
+
+- Back-to-back replays take 211.64 ms (ot128) and 134.87 ms (qt128) per PandaSet frame: 4.72 and 7.41 frames/s of
+  device throughput. A synchronous call gives 2.81 / 4.09 frames/s on that frame.
+- The device time barely depends on the frame: every point-wise op runs on the profile's `N_cap` rows. The host
+  stages grow with the cloud.
+- The shipped samples are synthetic scans: ot128 53,411 points (N = M = 52,613 after the crop, no interpolated point),
+  qt128 45,783 points (N = 69,553 after interpolation, M = 18,926). PandaSet 019 frame 40 is a 107,146-point Pandar64
+  sweep (N = 145,961 / M = 40,691 on ot128); it is not shipped.
+- `OPT_REPORT.md` ranks what the optimization phase will change (the two-term point layers, K1, the image-branch glue,
+  shape buckets, the host pre-processing).
+
+## Tests
+
+From the repository root, with `code/` on `PYTHONPATH` (or the package installed):
+
+- Host (no device, no ttnn call): `TT_VISIBLE_DEVICES=none python -m pytest -q code/tt_frnet/tests`. Inside the
+  tt-models workspace the suite runs on its fake ttnn (`-p fake_ttnn_plugin`), which also enables `test_tt_host.py`;
+  tests that need workspace-only oracles or data skip elsewhere, and FAIL inside the workspace when that data is
+  missing. `test_reference_cpu.py` needs onnxruntime.
+- Device (one p150): `python -m pytest -q -s code/tt_frnet/tests/test_pcc_device.py
+  code/tt_frnet/tests/test_e2e_device.py` (per-module PCC against the fp32 CPU reference on replay outputs, the K1
+  segment-max sites included; end-to-end agreement with ONNX Runtime on 17 OT128 / 5 QT128 frames, the shipped samples
+  against their stored references, replay == eager, API == `/predict`). The public-frame gates run only inside the
+  tt-models workspace, where their data lives: PandaSet, nuScenes and Autoware's sample rosbag, of which only PandaSet
+  may be redistributed.
+- The gates are frozen in `tests/*.gates.json` at the first green run and are never loosened.
+
+## Limits
+
+- Batch 1 on the chip; one model per process; one frame (one sweep) per call; one variant per model.
+- The device input has a fixed size per profile (`N_cap` 160,000 / 120,000 points, `M_cap` 60,000 cells): a frame
+  outside Autoware's profile ranges is skipped exactly as the node skips it.
+- Label names 22 / 23: Autoware's `class_names` read `debris` / `stroller`, but the weights were trained with
+  stroller = 22 and debris = 23 (AWML config). The port keeps Autoware's names for parity.
+- Numerics: the device's per-point probabilities differ slightly from the fp32 reference, so points near a class
+  boundary can take the other class (98.9-99.6 % of the points agree on the public frames, see the model card).
+- Not ported: Autoware's `PointCloud2` layouts and the filtered cloud's `output_format` conversion; the API takes
+  arrays or files and returns arrays.

code/conftest.py

@@ -0,0 +1,59 @@
+# SPDX-License-Identifier: Apache-2.0
+"""pytest fixtures of frnet-p150 (no dependency on tt-metal's own conftest).
+
+- ``device`` (session): one chip opened like the published numbers (``tt_frnet.device.open_device``:
+  ETH dispatch, 12x10 grid, the port's validated sizes and CQs). ``--device-id N`` or ``TT_DEVICE_ID`` selects
+  the chip (default 0); ``FRNET_DISPATCH=worker`` is the A/B opt-in, and the other ``FRNET_*``
+  variables apply as for the server. A failing ETH open is an error here, never a silent WORKER fallback: gates
+  are only valid on the published setup.
+- Tests marked ``device`` are skipped when ttnn is missing or ``TT_VISIBLE_DEVICES=none`` (host-only runs).
+  On the shared workspace box run them through the lock and name the test files:
+  ``bin/devrun -t 1800 -- python -m pytest -q -s code/tt_frnet/tests/test_pcc_device.py``.
+"""
+from __future__ import annotations
+
+import gc
+import importlib.util
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+    parser.addoption("--device-id", action="store", default=None, help="chip id (default $TT_DEVICE_ID or 0)")
+
+
+def pytest_configure(config):
+    config.addinivalue_line("markers", "device: needs a Tenstorrent chip (skipped on host-only runs)")
+
+
+def _no_device_reason():
+    if os.environ.get("TT_VISIBLE_DEVICES", "").lower() == "none":
+        return "TT_VISIBLE_DEVICES=none (host-only run)"
+    if importlib.util.find_spec("ttnn") is None:
+        return "ttnn is not installed"
+    return None
+
+
+def pytest_collection_modifyitems(config, items):
+    reason = _no_device_reason()
+    if reason:
+        skip = pytest.mark.skip(reason=reason)
+        for item in items:
+            if "device" in item.keywords:
+                item.add_marker(skip)
+
+
+@pytest.fixture(autouse=True)
+def _gc_between_tests():
+    gc.collect()
+
+
+@pytest.fixture(scope="session")
+def device(request):
+    from tt_frnet.device import close_device, open_device
+
+    cli = request.config.getoption("--device-id")
+    dev = open_device(int(cli) if cli is not None else None, allow_fallback=False)
+    yield dev
+    close_device(dev)

code/models/common/lightweightmodule.py

@@ -0,0 +1,12 @@
+# SPDX-FileCopyrightText: © 2023 Tenstorrent USA, Inc.
+
+# SPDX-License-Identifier: Apache-2.0
+
+
+class LightweightModule:
+    """Torch modules add a surprising amount of host overhead for attribute
+    access and method calls. This class is a lightweight alternative that
+    just wraps a forward function for now."""
+
+    def __call__(self, *args, **kwargs):
+        return self.forward(*args, **kwargs)

code/scripts/README.md

@@ -0,0 +1,19 @@
+# code/scripts
+
+| script | purpose | device |
+|---|---|---|
+| `make_sample.py` | generates the SHIPPED samples (`samples/synthetic_<variant>.npz`, a deterministic synthetic scan per serve profile; no dataset involved), their calibration presets and their stored fp32 CPU-reference outputs (ONNX Runtime) | no |
+| `bench.py` | stage breakdown of a warm forward (pre / H2D / trace / back-to-back / D2H / unsort / post / e2e; AICLK; a check of each output against its stored CPU reference); the card / OPT numbers | yes, via `bin/devrun` |
+| `profile_ops.py` | one eager forward with module / layer signposts + one traced replay between signposts under the device profiler (`python -m tracy -r -p -v ...`) | yes |
+| `profile_summary.py` | per-module / per-layer / per-op tables from the profiler CSV (`OPT_BASELINE.md`) | no |
+| `visualizer_report.py` | ttnn-visualizer memory + graph report of one eager forward (not for timing) | yes |
+| `container_smoke.sh` | serve ONE serve profile of the built package (`--profile NAME`; default profile otherwise), run `server/smoke_test.py` against it (asserts ETH dispatch, the 12x10 grid and the profile's pins; compares with the stored CPU reference), keep the evidence (container log, `/info`, the `/predict` output and the result, in `logs/smoke/` of the repo or `--log-dir DIR`), always stop it; one `bin/devrun -t 3600 -k 150` window per profile | yes |
+| `make_demo.py` | workspace tool: runs the production API on the shipped samples and the public-dataset frames of `research/frnet/public_data` (`run`, device) and renders the card's `media/` (`render`, CPU) | `run`: yes |
+| `make_samples.py` | workspace tool: rebuilds the STAGED PandaSet 019 frame 40 sample (git-ignored `staging_samples_pandaset/`, not shipped) and its CPU references | no |
+| `make_goldens.py` | workspace tool: the large module-tap goldens and the host pre-processing hashes of the tests | no |
+| `precision_study.py`, `diag_conv_precision.py`, `diag_frame_taps.py` | precision studies of the port: the CPU emulation of the TT numerics, and device diagnostics (per-unit conv precision; per-module device-vs-emulation error on one frame) | the `diag_*` scripts: yes |
+| `fetch_samples.sh` | downloads public sample data that may not be redistributed (sha256-checked); no entry is active in this release | no |
+
+Everything here ships in `code/` on the Hub and inside the image (`source.extra_code` lists `scripts`). The workspace
+tools need the tt-models research tree (`research/frnet/`) and the raw datasets; they document how the card's
+numbers and renders were made but do not run from a Hub clone.

code/scripts/bench.py

@@ -0,0 +1,261 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Stage breakdown of warm forwards: the numbers OPT_BASELINE.md / OPT_REPORT.md / the card quote.
+
+    bin/devrun -t 900 -- python code/scripts/bench.py --iters 100 --json logs/baseline_bench.json
+    bin/devrun -t 900 -- python code/scripts/bench.py --variant qt128 --input a.npz --input b.npz --dispatch worker --cqs 2
+
+For every ``--input`` cloud (default: the shipped synthetic sample of the variant), batch 1, after ``--warmup`` calls: ``--iters``
+iterations of the served path cut into stages, each synchronised on its own (p50 / p99 / mean / min in ms, so the
+rows do not add up to ``e2e`` exactly), then ``--iters`` plain ``model(points, calibration=...)`` calls, then
+back-to-back replays:
+
+=====================  ================================================================================================
+``pre``                host pre-processing as served (``model._prepare``): decode, ego crop, float32 projection, fp16
+                       interpolation, unique cells, profile checks, the device plan (sort, CSR offsets, gather tables,
+                       8-d features with ``pre_norm`` folded)
+``pre.load``           the file decode alone (``ttaw.io.load_points``)
+``pre.crop``           the ego crop box (``crop_keep_mask``: float32 ``R p + t``)
+``pre.interp``         the fp16 interpolation grid and the interpolated points (``interpolation_grid`` +
+                       ``interpolate_points``)
+``pre.project``        float32 projection of every point to the frustum grid (``project2d``)
+``pre.unique``         unique cells and the inverse map (``unique_coors``)
+``pre.plan``           ``build_plan``: the sort, CSR offsets, ``point2pix_s`` / ``pix2vox_s`` and the 8-d features
+``host_in``            ``FrameStaging.write``: range checks + the plan written into the persistent host buffers
+``h2d``                upload of the 9 trace inputs + device synchronise (``TraceRunner.upload``)
+``h2d.<input>``        the upload split per input (``features`` fp32, ``offsets``, ``p2p_<s>``, ``v2p_<s>`` uint32)
+``trace``              one blocking replay of the whole-graph trace
+``d2h``                ``TraceRunner.read`` as served: the fp32 ``pred_probs`` ``[N_cap, 32]`` -> numpy
+``d2h.copy``           the readback split: the device-to-host copy into a preallocated host tensor ...
+``d2h.to_numpy``       ... and its host conversion to numpy (host untilize of the TILE rows)
+``unsort``             the first N rows / 27 columns back in the network-input order (``plan.unsort``)
+``post``               host post-processing (first-max argmax, probability, class filter, output assembly)
+``e2e``                ``model(points, calibration=...)`` as served (its ``timing_ms`` medians are reported too)
+``b2b``                back-to-back replays, one synchronise per round of ``--b2b-iters``: device time per forward
+=====================  ================================================================================================
+
+AICLK / power / temperature are sampled from sysfs during the run (``ttaw.profiling.AiclkSampler``), and the host
+load average is recorded (the host stages move with it). ``--check`` (default) compares the served output of each
+input with a stored fp32 CPU-reference ``/predict`` body (``--reference PATH`` per input, ``-`` for none; default:
+``<stem>.<variant>.reference.json`` next to the input) with the container-smoke gates (``server/smoke_test.py``
+``REFERENCE_GATES``). The calibration of an input (``--calib`` per input): ``preset:<name>`` (``calib/``), a JSON
+file holding the calibration dict, ``none`` (crop disabled); default: the preset named like the input's stem, else
+``<stem>.calib.json`` next to the input, else the crop disabled. Always quote the configuration line (dispatch /
+CQs / grid) with the numbers.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import platform
+import statistics
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+import numpy as np
+
+from tt_frnet import FRNet
+from tt_frnet.ttaw.profiling import AiclkSampler, StageBench, time_b2b
+
+CODE = Path(__file__).resolve().parents[1]
+SAMPLES = CODE / "tt_frnet" / "samples"           # the shipped synthetic_<variant>.npz
+CALIB_DIR = CODE / "tt_frnet" / "calib"
+
+
+def calibration_for(path: Path, spec: Optional[str] = None) -> Dict[str, Any]:
+    """The ``calibration`` argument for ``path`` (module docstring)."""
+    if spec:
+        if spec == "none":
+            return {"ego_crop_box": None}
+        if spec.startswith("preset:"):
+            return {"preset": spec[len("preset:"):]}
+        return json.loads(Path(spec).read_text())
+    stem = path.name.split(".")[0]
+    if (CALIB_DIR / f"{stem}.json").is_file():
+        return {"preset": stem}
+    side = path.parent / f"{stem}.calib.json"
+    if side.is_file():
+        return json.loads(side.read_text())
+    return {"ego_crop_box": None}
+
+
+def check_output(model, path: Path, calib: Dict[str, Any], reference: Optional[Path]) -> Dict[str, Any]:
+    """Served output vs the stored fp32 CPU-reference ``/predict`` body (the container-smoke comparison)."""
+    from tt_frnet.server import smoke_test as st
+    from tt_frnet.ttaw.server import smoke
+
+    out = model(str(path), calibration=calib).to_dict()
+    res: Dict[str, Any] = {"num_points": out.get("num_points"), "num_points_raw": out.get("num_points_raw"),
+                           "skipped": (out.get("meta") or {}).get("skipped")}
+    if reference is not None and reference.is_file():
+        metrics, fails = smoke.compare_with_reference(out, smoke.load_json(reference), gates=st.REFERENCE_GATES)
+        res["reference"] = {"file": str(reference), "metrics": metrics, "failures": fails}
+    return res
+
+
+def bench_input(model, path: Path, calib: Dict[str, Any], *, iters: int, warmup: int, b2b_iters: int,
+                b2b_rounds: int, name: str) -> Dict[str, Any]:
+    import ttnn
+
+    from tt_frnet.host.preprocess import (build_plan, crop_keep_mask, interpolate_points, interpolation_grid,
+                                          project2d, unique_coors)
+    from tt_frnet.host.frame import POINT_FIELDS
+    from tt_frnet.ttaw import io as tio
+    from tt_frnet.ttaw.tensors import to_numpy
+
+    tr = model.runner                       # ttaw.trace.TraceRunner, variant "default"
+    dev, pre, cfg = model.device, model.pre, model.config
+    params = model.validate_params({})
+    src = str(path)
+    bench = StageBench(name)
+    for _ in range(warmup):
+        model(src, calibration=calib)
+    out_dev = tr.outputs("default")         # the persistent trace output (device)
+    host_out = ttnn.allocate_tensor_on_host(out_dev.spec, dev)
+    names = list(model.graph.input_specs())
+    slots = {k: tr._slot(k, "input") for k in names}
+    T12, bounds, _ = pre.crop_setup(calib)
+    prepared = None
+    for _ in range(iters):
+        t0 = time.perf_counter()
+        prepared = model._prepare(points=src, calibration=calib)
+        t1 = time.perf_counter()
+        plan = prepared.plan
+        inputs = model.staging.write(plan)
+        t2 = time.perf_counter()
+        tr.upload(inputs)
+        ttnn.synchronize_device(dev)
+        t3 = time.perf_counter()
+        tr.replay("default")
+        ttnn.synchronize_device(dev)
+        t4 = time.perf_counter()
+        rows = tr.read("default")
+        t5 = time.perf_counter()
+        probs = np.asarray(rows, np.float32).reshape(-1, rows.shape[-1])
+        raw = plan.unsort(probs[:, : cfg.num_classes])
+        t6 = time.perf_counter()
+        model._postprocess(raw, prepared, params)
+        t7 = time.perf_counter()
+        for stage, a, b in (("pre", t0, t1), ("host_in", t1, t2), ("h2d", t2, t3), ("trace", t3, t4),
+                            ("d2h", t4, t5), ("unsort", t5, t6), ("post", t6, t7)):
+            bench.add(stage, (b - a) * 1e3)
+        # the split rows (outside the served path; the same data and functions as autoware_preprocess)
+        with bench.stage("pre.load"):
+            xyzi = np.ascontiguousarray(tio.load_points(src).select(POINT_FIELDS), np.float32)
+        with bench.stage("pre.crop"):
+            keep = (crop_keep_mask(xyzi[:, :3], T12, np.asarray(bounds, np.float64).astype(np.float32))
+                    if T12 is not None else np.ones(len(xyzi), bool))
+            rawp = xyzi[keep]
+        with bench.stage("pre.interp"):
+            grid, filled = interpolation_grid(rawp, cfg)
+            interp = interpolate_points(grid, filled, cfg)
+            allpts = np.concatenate([rawp, interp], axis=0) if len(interp) else rawp
+        with bench.stage("pre.project"):
+            fx, fy = project2d(allpts[:, :3], cfg.frustum_w, cfg.frustum_h, cfg.fov)
+            coors = np.stack([np.zeros_like(fy), fy, fx], axis=1).astype(np.int64)
+        with bench.stage("pre.unique"):
+            unique_coors(coors, cfg.frustum_w, cfg.frustum_h)
+        with bench.stage("pre.plan"):
+            build_plan(prepared.inputs, cfg, pre_norm=pre.pre_norm)
+        for k, slot in slots.items():
+            with bench.stage(f"h2d.{k}"):
+                ttnn.copy_host_to_device_tensor(inputs[k], slot.buffers[0], cq_id=0)
+                ttnn.synchronize_device(dev)
+        with bench.stage("d2h.copy"):
+            ttnn.copy_device_to_host_tensor(out_dev, host_out, blocking=True)
+        with bench.stage("d2h.to_numpy"):
+            to_numpy(host_out)
+    timing: List[Dict[str, float]] = []
+    for _ in range(iters):
+        t0 = time.perf_counter()
+        out = model(src, calibration=calib)
+        bench.add("e2e", (time.perf_counter() - t0) * 1e3)
+        timing.append(dict(out.timing_ms))
+    rounds = [time_b2b(lambda: tr.replay("default"), lambda: ttnn.synchronize_device(dev), n=b2b_iters)
+              for _ in range(b2b_rounds)]
+    for r in rounds:
+        bench.add("b2b", r)
+    b2b = statistics.median(rounds)
+    elem = {"float32": 4, "uint32": 4, "bfloat16": 2}
+    in_bytes = {k: int(np.prod(shape)) * elem[dtype] for k, (shape, dtype) in model.graph.input_specs().items()}
+    return {"stages_ms": bench.summary(), "table": bench.table(),
+            "timing_ms_median": {k: statistics.median(t[k] for t in timing) for k in timing[0]},
+            "b2b_rounds_ms": rounds, "b2b_iters": b2b_iters, "device_fps_b2b": 1e3 / b2b,
+            "e2e_fps_sync": 1e3 / bench.summary()["e2e"]["p50"],
+            "frame": {k: v for k, v in prepared.stats().items() if k != "ego_crop"},
+            "bytes": {**{f"h2d.{k}": v for k, v in in_bytes.items()}, "h2d": sum(in_bytes.values()),
+                      "d2h": int(np.prod(out_dev.shape)) * 4}}
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    ap.add_argument("--input", action="append", default=None, help="cloud .npz / .pcd / .bin; repeatable; "
+                    "default: the shipped synthetic sample of the variant")
+    ap.add_argument("--calib", action="append", default=None,
+                    help="per --input: preset:<name> | <calibration .json> | none (default: see the docstring)")
+    ap.add_argument("--reference", action="append", default=None,
+                    help="fp32 CPU-reference /predict body per --input ('-' for none; default <stem>.<variant>"
+                         ".reference.json next to the input)")
+    ap.add_argument("--iters", type=int, default=100)
+    ap.add_argument("--warmup", type=int, default=10)
+    ap.add_argument("--b2b-iters", type=int, default=50)
+    ap.add_argument("--b2b-rounds", type=int, default=5)
+    ap.add_argument("--variant", default=None, help="ot128 | qt128 (default: FRNET_VARIANT or ot128)")
+    ap.add_argument("--dispatch", default=None, choices=["eth", "worker"], help="default: FRNET_DISPATCH or eth")
+    ap.add_argument("--cqs", type=int, default=None, choices=[1, 2], help="command queues (default: FRNET_NUM_CQS or 1)")
+    ap.add_argument("--allow-fallback", action="store_true", help="accept a WORKER fallback of an ETH open")
+    ap.add_argument("--no-check", dest="check", action="store_false")
+    ap.add_argument("--tag", default="", help="label of this configuration in the output")
+    ap.add_argument("--json")
+    a = ap.parse_args()
+    inputs = [Path(p) for p in (a.input or [str(SAMPLES / f"synthetic_{a.variant or os.environ.get('FRNET_VARIANT', 'ot128')}.npz")])]
+    calibs = list(a.calib or []) + [None] * len(inputs)
+    refs = [None if r in (None, "-") else Path(r) for r in (a.reference or [])] + [None] * len(inputs)
+    load0 = os.getloadavg()
+    t0 = time.perf_counter()
+    with FRNet.from_pretrained(variant=a.variant, dispatch=a.dispatch, num_command_queues=a.cqs) as model:
+        load_s = time.perf_counter() - t0
+        info = dict(model.device_info)
+        want = a.dispatch or os.environ.get("FRNET_DISPATCH", "eth")
+        if info.get("dispatch") != want and not a.allow_fallback:
+            raise SystemExit(f"device opened with dispatch {info.get('dispatch')!r}, asked for {want!r}")
+        tag = a.tag or f"{info.get('dispatch')}-{info.get('num_command_queues')}cq {info.get('grid')}"
+        res: Dict[str, Any] = {"tag": tag, "config": info, "variant": model.variant,
+                               "graph": model.graph.describe(), "trace": model.runner.describe(),
+                               "load_s": load_s, "warmup_ms": dict(model.warmup_ms), "iters": a.iters,
+                               "host": {"cpus": os.cpu_count(), "machine": platform.processor() or platform.machine(),
+                                        "loadavg_start": load0}, "inputs": {}}
+        with AiclkSampler(interval_s=0.05) as clk:
+            for k, (path, cspec, ref) in enumerate(zip(inputs, calibs, refs)):
+                calib = calibration_for(path, cspec)
+                if ref is None and a.reference is None:
+                    cand = path.parent / f"{path.name.split('.')[0]}.{model.variant}.reference.json"
+                    ref = cand if cand.is_file() else None
+                r = bench_input(model, path, calib, iters=a.iters, warmup=a.warmup, b2b_iters=a.b2b_iters,
+                                b2b_rounds=a.b2b_rounds, name=f"{path.name} | {tag} | {model.variant}")
+                r["input"], r["calibration"] = str(path), calib
+                if a.check:
+                    r["check"] = check_output(model, path, calib, ref)
+                res["inputs"][f"{k}:{path.name}"] = r
+                print(r["table"], flush=True)
+                print(f"b2b rounds ms: {[round(x, 3) for x in r['b2b_rounds_ms']]}  "
+                      f"timing_ms medians: { {k2: round(v, 2) for k2, v in r['timing_ms_median'].items()} }  "
+                      f"frame: {r['frame']}  bytes: {r['bytes']}", flush=True)
+                if a.check:
+                    print("check:", json.dumps(r["check"], default=str), flush=True)
+        res["aiclk"] = clk.summary()
+        res["trace_after"] = model.runner.describe()
+        res["host"]["loadavg_end"] = os.getloadavg()
+    print(json.dumps({"tag": tag, "variant": res["variant"], "config": info, "aiclk": res["aiclk"],
+                      "load_s": round(load_s, 2), "program_cache_entries":
+                      res["trace_after"].get("program_cache_entries"),
+                      "loadavg": [load0, res["host"]["loadavg_end"]]}, default=str), flush=True)
+    if a.json:
+        Path(a.json).parent.mkdir(parents=True, exist_ok=True)
+        Path(a.json).write_text(json.dumps(res, indent=1, default=str) + "\n")
+
+
+if __name__ == "__main__":
+    main()

code/scripts/container_smoke.sh

@@ -0,0 +1,137 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: Apache-2.0
+# Serve ONE serve profile of the BUILT container package, run the smoke test against it, keep the evidence, and always
+# stop it -- in one device-lock window. Run it once per serve profile (`tt-model profiles
+# <staged dir>/tt_kernel_manifest.json` lists them; without --profile the package's default profile is served):
+#
+#   ROOT=/home/ubuntu/experiments/tt-models
+#   $ROOT/bin/devrun -t 3600 -k 150 -- env -u HF_TOKEN -u HUGGING_FACE_HUB_TOKEN sg docker -c \
+#       "bash code/scripts/container_smoke.sh $ROOT/build/frnet-p150 [port] [--profile NAME] [--log-dir DIR]"
+#
+# The smoke test FAILS unless /info reports ETH dispatch and the 12x10 grid and runs what the package pins for the
+# profile (dispatch, CQs, variant, weights revision); it also compares the output with the stored CPU reference of the
+# sample when one exists (code/tt_frnet/server/smoke_test.py). Exit code: the smoke test's (0 = PASS); 1 when
+# serve fails, 2 on a usage error.
+#
+# Evidence, kept whatever the outcome (--log-dir, default: logs/smoke/ of this repo, next to code/), named
+# <name>[-<profile>]-<UTC time>.*:
+#   .container.log  the container's whole log (boot, requests, shutdown), followed while the container stops
+#   .info.json      GET /info as soon as the server is READY (device: dispatch, grid, cores; pins; versions)
+#   .smoke.json     the smoke test's /predict output (the SMOKE_OUT environment variable overrides this path)
+#   .result.json    profile, port, exit code, start / end times
+#
+# `tt-model serve` returns once the server is READY and leaves the container running, so the stop must happen before
+# the lock is released, also when devrun's timeout TERMs this script: the EXIT trap saves the log, then stops the
+# container cleanly with SIGTERM (120 s grace, hence devrun -k 150), never `docker kill`, which would leave the chip
+# dirty. Needs docker access (sg docker) and python3.
+set -u
+usage() { echo "usage: container_smoke.sh <staged package dir> [port] [--profile NAME] [--log-dir DIR]" >&2; }
+PROFILE=""
+LOG_DIR=""
+POSITIONAL=()
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --profile) [ $# -ge 2 ] || { usage; exit 2; }; PROFILE="$2"; shift 2 ;;
+    --profile=*) PROFILE="${1#--profile=}"; shift ;;
+    --log-dir) [ $# -ge 2 ] || { usage; exit 2; }; LOG_DIR="$2"; shift 2 ;;
+    --log-dir=*) LOG_DIR="${1#--log-dir=}"; shift ;;
+    -h|--help) usage; exit 0 ;;
+    -*) echo "container_smoke.sh: unknown option $1" >&2; usage; exit 2 ;;
+    *) POSITIONAL+=("$1"); shift ;;
+  esac
+done
+[ ${#POSITIONAL[@]} -ge 1 ] && [ ${#POSITIONAL[@]} -le 2 ] || { usage; exit 2; }
+STAGED="${POSITIONAL[0]}"
+PORT="${POSITIONAL[1]:-20000}"
+MANIFEST="$STAGED/tt_kernel_manifest.json"
+[ -f "$MANIFEST" ] || { echo "container_smoke.sh: $MANIFEST not found (run tt-model package first)" >&2; exit 2; }
+HERE="$(cd "$(dirname "$0")" && pwd)"
+PROFILE_ARGS=()
+[ -n "$PROFILE" ] && PROFILE_ARGS=(--profile "$PROFILE")
+
+# The package name and the served profile (tt-model's rule: --profile, else default_profile, else the first serve
+# profile), hence the container name tt-model gives it: tt-model-<name>-<profile> (tt_kernel/container.py).
+read -r NAME PROFILE_NAME < <(python3 - "$MANIFEST" "$PROFILE" <<'EOF'
+import json, sys
+m = json.load(open(sys.argv[1]))
+c = m.get("container") or {}
+profiles = c.get("serve_profiles") or [{}]
+print(m.get("name") or "model", sys.argv[2] or c.get("default_profile") or profiles[0].get("name") or "default")
+EOF
+)
+[ -n "${NAME:-}" ] || { echo "container_smoke.sh: cannot read the package name from $MANIFEST" >&2; exit 2; }
+CONTAINER="tt-model-$NAME-$PROFILE_NAME"
+
+[ -n "$LOG_DIR" ] || LOG_DIR="$(cd "$HERE/../.." && pwd)/logs/smoke"
+if ! mkdir -p "$LOG_DIR" 2>/dev/null || [ ! -w "$LOG_DIR" ]; then
+  echo "container_smoke.sh: cannot write $LOG_DIR; keeping the evidence in ${TMPDIR:-/tmp}" >&2
+  LOG_DIR="${TMPDIR:-/tmp}"
+fi
+LOG_DIR="$(cd "$LOG_DIR" && pwd)"
+STARTED="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
+STEM="$LOG_DIR/$NAME${PROFILE:+-$PROFILE}-$(date -u +%Y%m%dT%H%M%SZ)"
+SMOKE_JSON="${SMOKE_OUT:-$STEM.smoke.json}"
+
+fetch() {  # fetch URL FILE: one GET with a 30 s timeout; the body goes to FILE
+  python3 - "$1" "$2" <<'EOF'
+import sys, urllib.request
+try:
+    with urllib.request.urlopen(sys.argv[1], timeout=30) as r:
+        body = r.read()
+except Exception as e:  # noqa: BLE001 -- best effort: the smoke test reports the server's state
+    sys.exit(f"container_smoke.sh: GET {sys.argv[1]} failed: {e}")
+open(sys.argv[2], "wb").write(body)
+EOF
+}
+
+CHILD=""
+STOPPED=0
+cleanup() {
+  local rc=$?
+  [ "$STOPPED" = 1 ] && return
+  STOPPED=1
+  if [ -n "$CHILD" ]; then kill -TERM "$CHILD" 2>/dev/null; wait "$CHILD" 2>/dev/null; fi
+  # The log is lost with the container: follow it (whole history, then the shutdown lines) while it stops.
+  local logger=""
+  if docker inspect "$CONTAINER" >/dev/null 2>&1; then
+    docker logs --follow "$CONTAINER" > "$STEM.container.log" 2>&1 &
+    logger=$!
+  else
+    tt-model logs "${PROFILE_ARGS[@]}" "$MANIFEST" > "$STEM.container.log" 2>&1 || true
+  fi
+  tt-model stop "${PROFILE_ARGS[@]}" "$MANIFEST" || true
+  if [ -n "$logger" ]; then
+    for _ in $(seq 1 30); do kill -0 "$logger" 2>/dev/null || break; sleep 1; done
+    kill "$logger" 2>/dev/null
+    wait "$logger" 2>/dev/null
+  fi
+  python3 - "$STEM.result.json" "$NAME" "$PROFILE_NAME" "$PORT" "$rc" "$STARTED" "$CONTAINER" "$MANIFEST" <<'EOF'
+import datetime, json, sys
+out, name, profile, port, rc, started, container, manifest = sys.argv[1:]
+ended = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+json.dump({"bundle": name, "profile": profile, "port": int(port) if port.isdigit() else port, "rc": int(rc),
+           "result": "PASS" if rc == "0" else "FAIL",
+           "started": started, "ended": ended, "container": container, "manifest": manifest}, open(out, "w"), indent=1)
+EOF
+  echo "container_smoke.sh: rc=$rc; evidence in $STEM.*"
+}
+trap cleanup EXIT
+trap 'exit 143' TERM
+trap 'exit 130' INT
+
+# Each step runs in the background and is waited for: bash defers a trap while a FOREGROUND command runs, so a TERM
+# sent to this script alone (devrun's timeout signals the whole process group) would otherwise wait for the step.
+step() {
+  "$@" &
+  CHILD=$!
+  wait "$CHILD"
+  local rc=$?
+  CHILD=""
+  return "$rc"
+}
+
+# --port and --profile BEFORE the target: options after it are passed through to the container (tt-model cli rule)
+step tt-model serve --port "$PORT" "${PROFILE_ARGS[@]}" "$MANIFEST" || exit 1
+step fetch "http://127.0.0.1:$PORT/info" "$STEM.info.json"
+step python3 "$HERE/../tt_frnet/server/smoke_test.py" --url "http://127.0.0.1:$PORT" --wait 600 \
+    --manifest "$MANIFEST" "${PROFILE_ARGS[@]}" --out "$SMOKE_JSON"

code/scripts/diag_conv_precision.py

@@ -0,0 +1,224 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Device diagnostic (M1b follow-up): where does the device lose accuracy outside the point island?
+
+    cd bundles/frnet-p150 && bin/devrun -t 1200 -- env PYTHONPATH=code python code/scripts/diag_conv_precision.py \\
+        --out /home/ubuntu/experiments/tt-models/logs/frnet/m1b/diag_conv_precision.json
+
+Teacher-forced units on the OT128 sample's golden taps (torch fp32 reference): each unit's device output is compared
+with the golden output (``dev``) and with a CPU emulation of the device's rounding points with exact fp32 weights
+(``emu``: bf16 inputs, conv outputs rounded to bf16 before / after each activation, bf16 adds); ``dev_vs_emu``
+rel-L2 measures the device's arithmetic error beyond the emulated bf16 roundings (0 for a perfect fp32 conv engine).
+Units: the stem (3 convs + HardSwish), every stage's BasicBlocks (stage input -> ``layer<l>``), every attention
+(``pixel_fusion<l>`` -> ``attention<l>``), the ``addcmul`` (-> ``stage<l>``), ``fuse2``, the softmax, the first conv
+of each stage alone (no activation), and the element-wise ops (HardSwish, sigmoid) on bf16 inputs. Configurations:
+``FRNET_CONV_TERMS`` 1 (float32 conv weights: the device's TF32-like path) and 2 (C17 two-term bf16 weights).
+Eager runs (no trace): a diagnostic, not a gate.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+sys.path.insert(0, str(HERE.parents[1]))
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--out", type=Path, required=True)
+    ap.add_argument("--terms", nargs="*", type=int, default=[1, 2])
+    args = ap.parse_args()
+
+    import torch
+    import torch.nn.functional as F
+
+    torch.set_num_threads(4)
+    from tt_frnet.device import device_session
+    from tt_frnet.reference.config import FRNetConfig
+    from tt_frnet.reference.weights import load_weights
+    from tt_frnet.tests._data import golden_dir, weights_dir
+    from tt_frnet.tt.layers import free
+    from tt_frnet.tt.model import KNOBS, LOGIT_PAD, TtFRNet
+    from tt_frnet.ttaw.metrics import error_stats, pcc
+    from tt_frnet.ttaw.ops.conv import FeatureMap, fmap_from_numpy, fmap_to_numpy
+    from tt_frnet.ttaw.tensors import round_to_bf16
+
+    variant = "ot128"
+    wdir = weights_dir()
+    cfg = FRNetConfig.from_model_dir(wdir, variant)
+    weights = load_weights(wdir, variant)
+    taps = np.load(golden_dir() / variant / "pandaset_019_40.taps.npz", mmap_mode="r")
+
+    def g(name):
+        return np.asarray(taps[name], np.float32)
+
+    def bf(a):
+        return round_to_bf16(np.asarray(a, np.float32))
+
+    def t(a):
+        return torch.from_numpy(np.ascontiguousarray(a, np.float32))
+
+    # ---- CPU emulation of the device rounding points (exact fp32 weights)
+    def emu_conv(x, layer, act=True):
+        y = F.conv2d(t(x), t(layer.weight), t(layer.bias), stride=layer.stride, padding=layer.padding).numpy()
+        y = bf(y)
+        if act and layer.activation == "hardswish":
+            y = bf(F.hardswish(t(y)).numpy())
+        elif act and layer.activation == "sigmoid":
+            y = bf(torch.sigmoid(t(y)).numpy())
+        return y
+
+    def emu_blocks(x, st):
+        for c1, c2, ds in st.blocks:
+            y = emu_conv(emu_conv(x, c1), c2)
+            idt = emu_conv(x, ds) if ds is not None else x
+            x = bf(F.hardswish(t(bf(y + idt))).numpy())
+        return x
+
+    def stats(dev, emu, gold):
+        d = error_stats(dev, gold)
+        e = error_stats(emu, gold)
+        de = error_stats(dev, emu)
+        return {"pcc_dev": d["pcc"], "pcc_emu": e["pcc"], "rel_l2_dev": d["rel_l2"], "rel_l2_emu": e["rel_l2"],
+                "rel_l2_dev_vs_emu": de["rel_l2"], "max_abs_dev_vs_emu": de["max_abs"]}
+
+    results = {"variant": variant, "frame": "pandaset_019_40", "configs": {}}
+    stage_inputs = ["backbone.fusion_stem", "backbone.stage1", "backbone.stage2", "backbone.stage3"]
+    emu_cache = {}
+    with device_session() as device:
+        for terms in args.terms:
+            t0 = time.time()
+            knobs = KNOBS.read(CONV_TERMS=terms)
+            net = TtFRNet(device, weights, cfg, knobs=knobs)
+            res = {}
+
+            def up(nchw):
+                return fmap_from_numpy(bf(nchw), device, dtype="bfloat16")
+
+            def report(name, dev, emu, gold):
+                if callable(emu):                   # the CPU emulation is the same for every configuration
+                    if name not in emu_cache:
+                        emu_cache[name] = emu()
+                    emu = emu_cache[name]
+                res[name] = stats(dev, emu, gold)
+                r = res[name]
+                print(f"[terms={terms}] {name:28s} pcc dev {r['pcc_dev']:.6f} emu {r['pcc_emu']:.6f} | rel_l2 dev "
+                      f"{r['rel_l2_dev']:.3e} emu {r['rel_l2_emu']:.3e} dev-vs-emu {r['rel_l2_dev_vs_emu']:.3e}",
+                      flush=True)
+
+            # stem
+            fr = g("backbone.frustum")
+            fr32 = np.concatenate([fr, np.zeros_like(fr)], 1)
+            x = up(fr32)
+            for conv in net.stem:
+                y = conv(x)
+                free(x)
+                x = y
+            dev = fmap_to_numpy(x)
+            free(x)
+            def emu_stem():
+                e = bf(fr)
+                for c in weights.stem:
+                    e = emu_conv(e, c)
+                return e
+            report("stem", dev, emu_stem, g("backbone.stem"))
+            # stages
+            for st_w, st, src in zip(weights.stages, net.stages, stage_inputs):
+                l = st["index"]
+                xin = g(src)
+                # first conv alone (pre-activation)
+                c1 = st["blocks"][0][0]
+                x = up(xin)
+                y = c1.conv(x)
+                dev = fmap_to_numpy(y)
+                free(y)
+                lay = st_w.blocks[0][0]
+                exact = F.conv2d(t(bf(xin)), t(lay.weight), t(lay.bias), stride=lay.stride,
+                                 padding=lay.padding).numpy()
+                report(f"layer{l}.block0.conv1(noact)", dev, bf(exact), exact)
+                # blocks
+                for c1_, c2_, ds_ in st["blocks"]:
+                    y1 = c1_(x)
+                    y = c2_(y1)
+                    free(y1)
+                    idt = ds_(x) if ds_ is not None else x
+                    import ttnn
+
+                    s = ttnn.add(y.tensor, idt.tensor, memory_config=ttnn.DRAM_MEMORY_CONFIG)
+                    out = y.with_tensor(ttnn.hardswish(s))
+                    free(s, y)
+                    if ds_ is not None:
+                        free(idt)
+                    free(x)
+                    x = out
+                dev = fmap_to_numpy(x)
+                free(x)
+                report(f"layer{l}", dev, lambda: emu_blocks(bf(xin), st_w), g(f"backbone.layer{l}"))
+                # attention
+                pfz = g(f"backbone.pixel_fusion{l}")
+                x = up(pfz)
+                a0 = st["attention"][0](x)
+                a1 = st["attention"][1](a0)
+                free(a0)
+                dev = fmap_to_numpy(a1)
+                free(a1)
+                report(f"attention{l}", dev,
+                       lambda: emu_conv(emu_conv(bf(pfz), st_w.attention[0]), st_w.attention[1]),
+                       g(f"backbone.attention{l}"))
+                # addcmul
+                lay_g, att_g = g(f"backbone.layer{l}"), g(f"backbone.attention{l}")
+                xl, att = up(lay_g), up(att_g)
+                out = ttnn.addcmul(xl.tensor, x.tensor, att.tensor, memory_config=ttnn.DRAM_MEMORY_CONFIG)
+                dev = fmap_to_numpy(x.with_tensor(out))
+                free(out, xl, att, x)
+                e = bf(bf(lay_g) + bf(pfz) * bf(att_g))
+                report(f"stage{l}(addcmul)", dev, e, g(f"backbone.stage{l}"))
+            # fuse2
+            f1 = g("decoder.fuse1")
+            x = up(f1)
+            y = net.fuse2(x)
+            dev = fmap_to_numpy(y)
+            free(x, y)
+            report("fuse2", dev, lambda: emu_conv(bf(f1), weights.fuse_layer2), g("decoder.fuse2"))
+            # element-wise ops on bf16 inputs (synthetic: the stage-1 map scaled to a wide range)
+            import torch as _t
+            import ttnn
+
+            src = g("backbone.layer1")[:, :, :32, :] * 3.0
+            xin = up(src)
+            for opname, fn, ref in (("hardswish", ttnn.hardswish, F.hardswish), ("sigmoid", ttnn.sigmoid,
+                                                                                 torch.sigmoid)):
+                yo = xin.with_tensor(fn(xin.tensor))
+                dev = fmap_to_numpy(yo)
+                free(yo)
+                exact = ref(t(bf(src))).numpy()
+                report(f"op.{opname}(bf16)", dev, bf(exact), exact)
+            free(xin)
+            # softmax (fp32 logits, padded with -1e30)
+            lg = g("head.logits")[:65536]
+            pad = np.full((lg.shape[0], LOGIT_PAD), -1e30, np.float32)
+            pad[:, : lg.shape[1]] = lg
+            lt = ttnn.from_torch(_t.from_numpy(pad.reshape(1, 1, *pad.shape)), dtype=ttnn.float32,
+                                 layout=ttnn.TILE_LAYOUT, device=device, memory_config=ttnn.DRAM_MEMORY_CONFIG)
+            pr = ttnn.softmax(lt, dim=-1, compute_kernel_config=net._softmax_cfg, numeric_stable=True)
+            dev = ttnn.to_torch(pr).float().numpy().reshape(-1, LOGIT_PAD)[:, : lg.shape[1]]
+            free(lt, pr)
+            exact = torch.softmax(t(lg), -1).numpy()
+            report("softmax(fp32)", dev, exact, g("pred_probs")[:65536])
+            net.release()
+            res["seconds"] = round(time.time() - t0, 1)
+            results["configs"][f"conv_terms={terms}"] = res
+    args.out.parent.mkdir(parents=True, exist_ok=True)
+    args.out.write_text(json.dumps(results, indent=1))
+    print("wrote", args.out)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/diag_frame_taps.py

@@ -0,0 +1,131 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Device diagnostic: per-module error of the device graph on one catalogued frame, next to the CPU emulation.
+
+    cd bundles/frnet-p150 && bin/devrun -t 1200 -- env PYTHONPATH=code python code/scripts/diag_frame_taps.py \\
+        --frame nuscenes_scene-0916_40.ot128 --out /home/ubuntu/experiments/tt-models/logs/frnet/m1b/diag_0916_40.json
+
+The exact fp32 TT dataflow (``reference/rewrites.py FRNetPlanTorch``) is the reference of every tap; the CPU
+emulation (``--emulation``, default the ``plan_exact_points`` row of ``code/scripts/precision_study.py``: bf16 conv
+operands, gathers and segment tables, exact point layers) and the device ``taps`` variant (one replay; knobs from the
+``FRNET_*`` environment) are compared with it tap by tap: where the device's ``1 - PCC`` grows faster than the
+emulation's is where the device loses accuracy that the emulation does not model. Also reported: ``pred_probs`` PCC
+and raw-point argmax agreement vs ONNX Runtime for the device and the emulation. The CPU work runs first (in the
+device lock: ~1-2 min at OT128).
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+sys.path.insert(0, str(HERE.parents[1]))
+sys.path.insert(0, str(HERE.parent))
+
+TAPS = ("encoder.pf3", "encoder.voxel_feats", "backbone.stem", "backbone.point_stem", "backbone.fusion_stem",
+        *(f"backbone.{k}{i}" for i in range(1, 5) for k in ("layer", "point_fusion", "pixel_fusion", "attention",
+                                                            "stage")),
+        "decoder.fuse1", "decoder.fuse2", "decoder.point_fuse1", "decoder.point_fuse2",
+        *(f"head.mlp{i}" for i in range(4)), "head.logits", "pred_probs")
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--frame", required=True)
+    ap.add_argument("--out", type=Path, required=True)
+    ap.add_argument("--emulation", default="plan_exact_points")
+    ap.add_argument("--threads", type=int, default=4)
+    ap.add_argument("--cpu-only", action="store_true", help="stop after the CPU part (a dry run without a device)")
+    args = ap.parse_args()
+
+    import torch
+
+    torch.set_num_threads(args.threads)
+    from precision_study import CONFIGS
+    from tt_frnet.host.preprocess import build_plan
+    from tt_frnet.reference.config import FRNetConfig
+    from tt_frnet.reference.rewrites import Emulation, FRNetPlanTorch
+    from tt_frnet.reference.weights import load_weights
+    from tt_frnet.tests import _data
+    from tt_frnet.ttaw.golden import TapRegistry
+    from tt_frnet.ttaw.metrics import argmax_agreement, pcc
+
+    variant = args.frame.rpartition(".")[2]
+    wdir = _data.weights_dir()
+    cfg = FRNetConfig.from_model_dir(wdir, variant)
+    w = load_weights(wdir, variant)
+    inputs, ort_probs, _ = _data.load_frame(args.frame)
+    plan = build_plan(inputs, cfg, pre_norm=(w.pre_norm_scale, w.pre_norm_shift))
+    n, n_raw = plan.n, inputs.num_points_raw
+    H, W = plan.frustum_hw
+    t0 = time.time()
+    exact = TapRegistry(include=list(TAPS))
+    FRNetPlanTorch(w)(plan, taps=exact)
+    emu = TapRegistry(include=list(TAPS))
+    emu_probs = plan.unsort(FRNetPlanTorch(w, emulation=Emulation.parse(CONFIGS[args.emulation]))(plan, taps=emu))
+    cpu_s = time.time() - t0
+
+    def rows(name, a):
+        """point taps: the first N sorted rows; voxel taps: the first M rows."""
+        a = np.asarray(a)
+        if a.shape[0] == plan.n_cap:
+            return a[:n]
+        return a[: plan.m] if a.shape[0] in (cfg.m_cap, cfg.m_cap + 1) else a
+
+    if args.cpu_only:
+        ort = np.asarray(ort_probs, np.float32)
+        print(f"cpu {cpu_s:.0f} s; emulation pred_probs pcc {pcc(emu_probs, ort):.6f} argmax "
+              f"{argmax_agreement(emu_probs[:n_raw], ort[:n_raw]):.5f}")
+        return 0
+    from tt_frnet.device import device_session
+    from tt_frnet.tt.model import TtFRNet, build_runner, frame_arrays
+
+    report = {"frame": args.frame, "N": n, "M": plan.m, "emulation": args.emulation, "cpu_s": round(cpu_s, 1),
+              "taps": {}}
+    with device_session() as device:
+        graph = TtFRNet(device, w, cfg)
+        arrays = frame_arrays(plan, cfg.m_cap)
+        runner = build_runner(graph, variants=("taps",), name="frnet_diag", init=arrays)
+        try:
+            runner.capture()
+            dev = runner("taps", inputs=arrays)
+            report["knobs"] = graph.knobs.as_dict()
+            for name in TAPS:
+                if name not in dev or name not in exact:
+                    continue
+                ref = np.asarray(exact[name], np.float32)
+                d = np.asarray(dev[name], np.float32)
+                if ref.ndim == 4:
+                    _, c, h, ww = ref.shape
+                    d = d.reshape(-1, d.shape[-1])[: h * ww, :c].reshape(1, h, ww, c).transpose(0, 3, 1, 2)
+                    e = np.asarray(emu[name], np.float32)
+                else:
+                    ref = rows(name, ref)[:, : ref.shape[-1]]
+                    d = rows(name, d.reshape(-1, d.shape[-1]))[:, : ref.shape[-1]]
+                    e = rows(name, np.asarray(emu[name], np.float32))[:, : ref.shape[-1]]
+                p_dev, p_emu = float(pcc(d, ref)), float(pcc(e, ref))
+                ratio = (1 - p_dev) / max(1 - p_emu, 1e-12)
+                report["taps"][name] = {"pcc_dev": p_dev, "pcc_emu": p_emu, "err_ratio": ratio}
+                print(f"{name:26s} dev {p_dev:.6f} emu {p_emu:.6f}  (1-pcc) ratio {ratio:6.2f}", flush=True)
+            probs = plan.unsort(np.asarray(dev["pred_probs"], np.float32).reshape(-1, 32)[:, :27])
+        finally:
+            runner.release()
+            graph.release()
+    ort = np.asarray(ort_probs, np.float32)
+    report["device"] = {"pred_probs_pcc": float(pcc(probs, ort)),
+                        "argmax_agreement_raw": float(argmax_agreement(probs[:n_raw], ort[:n_raw]))}
+    report["emulation_result"] = {"pred_probs_pcc": float(pcc(emu_probs, ort)),
+                                  "argmax_agreement_raw": float(argmax_agreement(emu_probs[:n_raw], ort[:n_raw]))}
+    print("device", report["device"], "emulation", report["emulation_result"])
+    args.out.parent.mkdir(parents=True, exist_ok=True)
+    args.out.write_text(json.dumps(report, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/fetch_samples.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: Apache-2.0
+# Download the public sample data that frnet-p150 may NOT redistribute (license unstated or non-commercial)
+# into ${FRNET_SAMPLES:-$HOME/.cache/tt_frnet/samples}, sha256-checked. Tests and benchmarks read it from there.
+# Nothing downloaded here is executed; parse it with numpy / sqlite3 only.
+set -euo pipefail
+DEST="${FRNET_SAMPLES:-$HOME/.cache/tt_frnet/samples}"
+mkdir -p "$DEST"
+
+fetch() {  # fetch <url> <sha256> <file name>
+  local url="$1" sum="$2" name="$3"
+  if [ -f "$DEST/$name" ] && echo "$sum  $DEST/$name" | sha256sum -c --status; then
+    echo "ok      $name"; return
+  fi
+  curl -fL --retry 3 -o "$DEST/$name.part" "$url"
+  echo "$sum  $DEST/$name.part" | sha256sum -c --status || { echo "sha256 mismatch: $name" >&2; exit 1; }
+  mv "$DEST/$name.part" "$DEST/$name"; echo "fetched $name"
+}
+
+# Example (Autoware demo rosbag; pinned in autoware/ansible/roles/demo_artifacts/tasks/main.yaml:48-53):
+# fetch https://autoware-files.s3.us-west-2.amazonaws.com/recordings/bags/demos/sample-rosbag.zip \
+#       5f9d36353393b3d249212153c19049822b1298db56512aa045b4f7f6fc37cf88 sample-rosbag.zip
+
+echo "samples in $DEST"

code/scripts/make_demo.py

@@ -0,0 +1,322 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Demo media of the model card and the wider TT-vs-CPU agreement check on public frames (workspace tool: it reads
+the public-dataset inputs and CPU runs of ``research/frnet/public_data`` and the raw datasets for the camera images;
+none of them ship).
+
+Two steps, so the device is held only for inference:
+
+    # 1. device (tt-env, under bin/devrun): the production API on the shipped samples, the demo frames and every
+    #    converted public frame (PandaSet 019 / 029 every 5th frame, all 81 nuScenes mini_val key-frames), both
+    #    profiles; per-frame p150 classes + agreement with the CPU reference (ONNX Runtime fp32 runs of the research
+    #    pipeline: research/frnet/public_data/runs)
+    bin/devrun -t 1800 -- python code/scripts/make_demo.py run --out logs/frnet/m5_docs/demo
+    # 2. CPU (research venv: OpenCV for the camera JPEGs): the renders into media/
+    tools/research-venv/bin/python code/scripts/make_demo.py render --out logs/frnet/m5_docs/demo --media media
+
+``run`` writes ``<out>/run/<frame>_<variant>.npz`` (``pred``: the p150 class of every input row, 255 = not scored:
+ego crop box or profile skip; ``cpu``: the reference's, same layout) and ``<out>/agreement.json`` (per frame and per
+set: raw-point argmax agreement of p150 vs CPU, N / M, timing). Agreement here is on the class decision of every raw
+point the node scores; the frozen device gates (``tests/test_e2e_device.py``) additionally compare the probabilities.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+CODE = HERE.parents[1]
+sys.path.insert(0, str(CODE))
+
+WS = Path("/home/ubuntu/experiments/tt-models")
+PUB = WS / "research" / "frnet" / "public_data"
+IGNORE = 255
+VARIANTS = ("ot128", "qt128")
+# camera + BEV renders (PandaSet front camera, nuScenes CAM_FRONT); TT vs CPU compares; the PandaSet BEV sequence
+CAMERA_FRAMES = ["pandaset_019_40", "pandaset_029_40", "nuscenes_scene-0103_20"]
+COMPARE_FRAMES = ["pandaset_019_40", "nuscenes_scene-0916_40"]
+GIF_FRAMES = [f"pandaset_019_{k:02d}" for k in range(0, 80, 5)]
+
+
+def public_frames():
+    return sorted(p.stem for p in (PUB / "inputs").glob("*.npz"))
+
+
+# ------------------------------------------------------------------------------------------------ run (device)
+def cmd_run(a) -> int:
+    from tt_frnet import FRNet
+    from tt_frnet.ttaw.server.smoke import decode_array
+
+    out = Path(a.out)
+    (out / "run").mkdir(parents=True, exist_ok=True)
+    frames = public_frames() if not a.frames else a.frames
+    report = {"frames": {}, "sets": {}, "shipped": {}}
+    for variant in a.variants:
+        t0 = time.perf_counter()
+        with FRNet.from_pretrained(variant=variant, weights_dir=a.weights_dir) as model:
+            load_s = time.perf_counter() - t0
+            dev = model.info.get("device")
+            print(f"[{variant}] loaded in {load_s:.1f} s, device {dev}", flush=True)
+            report.setdefault("device", {})[variant] = dev
+            # the shipped synthetic sample vs its stored reference body
+            sample = CODE / "tt_frnet" / "samples" / f"synthetic_{variant}.npz"
+            res = model(sample, calibration={"preset": f"synthetic_{variant}"})
+            ref = json.loads(sample.with_name(f"synthetic_{variant}.{variant}.reference.json").read_text())
+            ref_labels = np.asarray(decode_array(ref["labels"]).values, np.uint8)
+            np.savez_compressed(out / "run" / f"synthetic_{variant}.npz", points=res.points, pred=res.label_ids,
+                                cpu=ref_labels, num_points_raw=res.num_points_raw)
+            report["shipped"][variant] = {"agreement": float(np.mean(res.label_ids == ref_labels)),
+                                          "n": len(res), "n_raw": int(res.num_points_raw),
+                                          "timing_ms": res.timing_ms}
+            print(f"[{variant}] shipped: {report['shipped'][variant]}", flush=True)
+            agree_rows = {"pandaset": [0, 0], "nuscenes": [0, 0]}
+            per_set = {"pandaset": [], "nuscenes": []}
+            for name in frames:
+                z = np.load(PUB / "inputs" / f"{name}.npz", allow_pickle=False)
+                xyzi, T = z["xyzi"], z["T_base_link_sensor"]
+                run = np.load(PUB / "runs" / f"{name}_{variant}.npz", allow_pickle=False)
+                cpu = np.full(len(xyzi), IGNORE, np.uint8)
+                cpu[run["input_rows"]] = run["class_id"]
+                t1 = time.perf_counter()
+                res = model(xyzi, calibration={"T_base_link_from_sensor": T.tolist()})
+                ms = (time.perf_counter() - t1) * 1e3
+                pred = np.full(len(xyzi), IGNORE, np.uint8)
+                skipped = res.meta.get("skipped")
+                if not skipped:
+                    pred[res.input_index] = res.label_ids[: res.num_points_raw]
+                same_rows = bool(np.array_equal(pred != IGNORE, cpu != IGNORE))
+                scored = cpu != IGNORE
+                n_eq = int(np.sum(pred[scored] == cpu[scored]))
+                n_sc = int(scored.sum())
+                rec = {"variant": variant, "skipped": skipped, "same_rows": same_rows, "n_raw": n_sc,
+                       "agree": n_eq / n_sc if n_sc else None, "n": len(res), "ms": round(ms, 1)}
+                report["frames"][f"{name}.{variant}"] = rec
+                kind = name.split("_")[0]
+                if n_sc:
+                    agree_rows[kind][0] += n_eq
+                    agree_rows[kind][1] += n_sc
+                    per_set[kind].append(rec["agree"])
+                np.savez_compressed(out / "run" / f"{name}_{variant}.npz", pred=pred, cpu=cpu)
+                print(f"[{variant}] {name}: agree {rec['agree']} rows_equal {same_rows} skipped {skipped} "
+                      f"{ms:.0f} ms", flush=True)
+            for kind, (eq, n) in agree_rows.items():
+                if n:
+                    report["sets"][f"{kind}.{variant}"] = {
+                        "frames": len(per_set[kind]), "points": n, "pooled": eq / n,
+                        "min_frame": min(per_set[kind]), "median_frame": float(np.median(per_set[kind]))}
+            print(f"[{variant}] sets: { {k: v for k, v in report['sets'].items() if k.endswith(variant)} }",
+                  flush=True)
+    (out / "agreement.json").write_text(json.dumps(report, indent=1, default=float))
+    bad = [k for k, r in report["frames"].items() if not r["same_rows"]]
+    if bad:
+        print(f"scored rows differ from the CPU reference on {bad}", flush=True)
+        return 1
+    return 0
+
+
+# ------------------------------------------------------------------------------------------------ render (CPU)
+ATTR_PANDASET = ("Contains data from PandaSet (Scale AI and Hesai), https://pandaset.org, licensed under CC BY 4.0 and "
+                 "the PandaSet Dataset Terms. Changes: Pandar64 sweep re-expressed in an estimated sensor frame, "
+                 "front-camera image downscaled and dimmed, points and labels overlaid. Scale AI and Hesai do not "
+                 "endorse this work. Cite: P. Xiao et al., PandaSet: Advanced Sensor Suite Dataset for Autonomous "
+                 "Driving, ITSC 2021.")
+ATTR_NUSCENES = ("Rendered from the nuScenes dataset, (c) Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of "
+                 "Use (https://www.nuscenes.org/terms-of-use). Non-commercial use only; adaptations under the same "
+                 "license. Motional does not endorse this work. Changes: camera image downscaled and dimmed, LiDAR "
+                 "points and labels overlaid. Cite: H. Caesar et al., nuScenes: A Multimodal Dataset for Autonomous "
+                 "Driving, CVPR 2020; W. K. Fong et al., Panoptic nuScenes, RA-L 2022 (lidarseg labels).")
+NC_BADGE = "CC BY-NC-SA 4.0 · non-commercial"
+
+
+def _research_render():
+    sys.path.insert(0, str(PUB / "scripts"))
+    sys.path.insert(0, str(WS / "datasets" / "tools"))
+    import frnet_public_common as C  # noqa: E402  (workspace code)
+    import frnet_public_render as R  # noqa: E402
+
+    return C, R
+
+
+def _agree_text(rep, key):
+    r = rep["frames"].get(key)
+    if not r or r["agree"] is None:
+        return ""
+    return f"p150 vs fp32 CPU reference: {r['agree'] * 100:.2f} % of the {r['n_raw']:,} scored raw points agree"
+
+
+def render_camera(name: str, run_dir: Path, rep: dict, out_path: Path) -> None:
+    import cv2  # noqa: F401  (read_jpeg)
+
+    C, R = _research_render()
+    inp = np.load(PUB / "inputs" / f"{name}.npz", allow_pickle=False)
+    meta = json.loads(str(inp["meta_json"]))
+    pred = np.load(run_dir / f"{name}_ot128.npz", allow_pickle=False)["pred"]
+    gjson = json.loads((PUB / "runs" / f"{name}_ot128.json").read_text())
+    proc = pred != IGNORE
+    xyz = inp["xyzi"][:, :3]
+    agree = _agree_text(rep, f"{name}.ot128")
+    if name.startswith("pandaset"):
+        from pandaset_meta_loader import project
+        from pandaset_seq_loader import load_frame, load_sequence, read_jpeg
+
+        seq, k = meta["sequence"], meta["frame"]
+        m = load_sequence(C.PANDASET_ROOT, seq)
+        fr = load_frame(C.PANDASET_ROOT, seq, k, m)
+        world = fr["lidar"][inp["src_index"], :3]
+        cam = fr["cameras"]["front_camera"]
+        img = read_jpeg(cam["path"])
+        uv, idx = project(world[proc], cam["pose"], cam["K"], img.shape[1], img.shape[0])
+        sel = np.flatnonzero(proc)[idx]
+        depth = np.linalg.norm(inp["xyzi"][sel, :3], axis=1)
+        cam_im = R.camera_overlay(img, uv, depth, C.PALETTE_RGB[pred[sel]], R.W / img.shape[1])
+        bl, br = R.bev(xyz, R.colors_for(pred, ignore_rgb=R.UNPROC)), R.bev(xyz, R.colors_for(inp["gt27"]))
+        subtitle = (f"PandaSet sequence {seq}, frame {k:02d} · Hesai Pandar64, one sweep in the sensor frame · Autoware "
+                    f"lidar_frnet v2.0 weights, Autoware pre- and post-processing, network on one Tenstorrent "
+                    f"Blackhole p150 (N={gjson['N']:,}, M={gjson['M']:,}). {agree}.")
+        cam_caption = "front_camera (52° HFOV), points coloured by the p150 class"
+        cap_r = "PandaSet semseg mapped to FRNet classes (dark grey: no T4 counterpart, e.g. Ground, Pickup)"
+        footer, badge = ATTR_PANDASET, None
+    else:
+        import nuscenes_mini as nm
+        from pandaset_seq_loader import read_jpeg
+
+        ds = nm.NuScenesMini(C.NUSC_ROOT)
+        s = ds.get("sample", meta["sample_token"])
+        sd = ds.sensor_data(s, "LIDAR_TOP")
+        pts = ds.lidar(s)
+        T_lidar = ds.T_global_ego(sd) @ ds.T_ego_sensor(sd)
+        c = ds.cameras(s, ["CAM_FRONT"])["CAM_FRONT"]
+        T_cam_lidar = np.linalg.inv(c["T_global_ego"] @ c["T_ego_sensor"]) @ T_lidar
+        img = read_jpeg(c["path"])
+        xyz_c = nm.transform_points(T_cam_lidar, pts[:, :3].astype(np.float64))
+        ok = proc & (xyz_c[:, 2] > 1.0)
+        uvw = xyz_c[ok] @ c["K"].T
+        uv = uvw[:, :2] / uvw[:, 2:3]
+        ins = (uv[:, 0] >= 0) & (uv[:, 0] < img.shape[1]) & (uv[:, 1] >= 0) & (uv[:, 1] < img.shape[0])
+        sel = np.flatnonzero(ok)[ins]
+        cam_im = R.camera_overlay(img, uv[ins], xyz_c[sel, 2], C.PALETTE_RGB[pred[sel]], R.W / img.shape[1],
+                                  radius=2)
+        bl = R.bev(xyz, R.colors_for(pred, ignore_rgb=R.UNPROC), x_range=(-30.0, 50.0), y_half=40.0)
+        br = R.bev(xyz, R.colors_for(inp["gt27"]), x_range=(-30.0, 50.0), y_half=40.0)
+        subtitle = (f"nuScenes {meta['scene']}, key-frame {meta['keyframe_index']} (mini_val) · Velodyne HDL-32E "
+                    f"LIDAR_TOP rotated to x forward · Autoware lidar_frnet v2.0, network on one Tenstorrent Blackhole "
+                    f"p150 (N={gjson['N']:,}, M={gjson['M']:,}). Domain gap: a 32-beam sensor (+10/-30°) vs the "
+                    f"128-beam OT128 of the training data. {agree}.")
+        cam_caption = "CAM_FRONT, points coloured by the p150 class"
+        cap_r = "nuScenes-lidarseg mapped to FRNet classes (dark grey: no T4 counterpart, e.g. terrain, barrier)"
+        footer, badge = ATTR_NUSCENES, NC_BADGE
+    R.compose("FRNet OT128 · Tenstorrent Blackhole p150 (tt-nn)", subtitle, badge, cam_im, cam_caption, bl, br,
+              "p150 prediction (dark grey: not scored, ego crop box)", cap_r, pred[proc], footer, str(out_path))
+
+
+def render_compare_frame(name: str, run_dir: Path, rep: dict, out_path: Path, variant: str = "ot128") -> None:
+    from tt_frnet import viz
+    from tt_frnet.api import LABELS
+
+    inp = np.load(PUB / "inputs" / f"{name}.npz", allow_pickle=False)
+    r = np.load(run_dir / f"{name}_{variant}.npz", allow_pickle=False)
+    pred, cpu = r["pred"], r["cpu"]
+    scored = cpu != IGNORE
+    xyz = inp["xyzi"][scored, :3]
+    meta = json.loads(str(inp["meta_json"]))
+    nusc = name.startswith("nuscenes")
+    where = (f"nuScenes {meta['scene']}, key-frame {meta['keyframe_index']} (mini_val)" if nusc else
+             f"PandaSet sequence {meta['sequence']}, frame {meta['frame']:02d}")
+    header = (f"FRNet {variant.upper()} on {where}: every raw point the node scores, coloured by the p150 class (left) "
+              f"and by the fp32 CPU reference's (ONNX Runtime on the deployed ONNX, middle); right: the points whose "
+              f"class differs. {_agree_text(rep, f'{name}.{variant}')}.")
+    if nusc:
+        header += " Non-commercial render (CC BY-NC-SA 4.0)."
+    rng = ((-30.0, 50.0), 40.0) if nusc else ((-40.0, 60.0), 50.0)
+    img = viz.render_compare(xyz, pred[scored], cpu[scored], class_names=LABELS,
+                             titles=(f"p150 ({variant.upper()})", "fp32 CPU reference"), header=header,
+                             footer=(ATTR_NUSCENES.replace("camera image downscaled and dimmed, LiDAR points and "
+                                                           "labels overlaid", "LiDAR points rendered from above")
+                                     if nusc else ATTR_PANDASET.replace("front-camera image downscaled and dimmed, "
+                                                                        "points and labels overlaid",
+                                                                        "points rendered from above")),
+                             x_range=rng[0], y_half=rng[1], size=420)
+    img.save(out_path, optimize=True)
+    print("wrote", out_path, img.size)
+
+
+def render_shipped(run_dir: Path, rep: dict, media: Path) -> None:
+    from tt_frnet import viz
+    from tt_frnet.api import LABELS
+
+    for variant in VARIANTS:
+        r = np.load(run_dir / f"synthetic_{variant}.npz", allow_pickle=False)
+        s = rep["shipped"][variant]
+        header = (f"Shipped sample synthetic_{variant}.npz (generated by code/scripts/make_sample.py, no dataset): "
+                  f"{s['n']:,} output points ({s['n_raw']:,} raw + Autoware's interpolated points). Left: the p150 "
+                  f"classes ({variant.upper()} profile); middle: the stored fp32 CPU reference; right: the points "
+                  f"whose class differs ({s['agreement'] * 100:.2f} % agree).")
+        img = viz.render_compare(r["points"], r["pred"], r["cpu"], class_names=LABELS,
+                                 titles=(f"p150 ({variant.upper()})", "fp32 CPU reference (stored)"), header=header,
+                                 x_range=(-60.0, 60.0), y_half=60.0, size=420)
+        out = media / f"frnet_synthetic_{variant}_tt_vs_cpu.png"
+        img.save(out, optimize=True)
+        print("wrote", out, img.size)
+
+
+def render_gif(run_dir: Path, rep: dict, out_path: Path) -> None:
+    from PIL import Image, ImageDraw
+
+    C, R = _research_render()
+    frames = []
+    for name in GIF_FRAMES:
+        inp = np.load(PUB / "inputs" / f"{name}.npz", allow_pickle=False)
+        pred = np.load(run_dir / f"{name}_ot128.npz", allow_pickle=False)["pred"]
+        im = R.bev(inp["xyzi"][:, :3], R.colors_for(pred, ignore_rgb=R.UNPROC), size=480)
+        canvas = Image.new("RGB", (480, 480 + 44), R.BG)
+        canvas.paste(im, (0, 22))
+        d = ImageDraw.Draw(canvas)
+        k = int(name.split("_")[-1])
+        a = rep["frames"][f"{name}.ot128"]["agree"]
+        d.text((6, 3), f"FRNet OT128 on p150 · PandaSet 019, frame {k:02d} (every 5th: 2 Hz)", font=R.font(13),
+               fill=R.INK)
+        d.text((6, 504), f"agreement with the fp32 CPU reference {a * 100:.2f} % · CC BY 4.0, PandaSet (Scale AI, "
+                         f"Hesai)", font=R.font(11), fill=R.INK2)
+        frames.append(canvas.convert("P", palette=Image.ADAPTIVE, colors=128))
+    frames[0].save(out_path, save_all=True, append_images=frames[1:], duration=500, loop=0, optimize=True)
+    print("wrote", out_path, f"{out_path.stat().st_size / 1e6:.2f} MB")
+
+
+def cmd_render(a) -> int:
+    out, media = Path(a.out), Path(a.media)
+    media.mkdir(parents=True, exist_ok=True)
+    run_dir = out / "run"
+    rep = json.loads((out / "agreement.json").read_text())
+    render_shipped(run_dir, rep, media)
+    for name in CAMERA_FRAMES:
+        nc = "_NC" if name.startswith("nuscenes") else ""
+        render_camera(name, run_dir, rep, media / f"frnet_ot128_{name}_tt{nc}.png")
+    for name in COMPARE_FRAMES:
+        nc = "_NC" if name.startswith("nuscenes") else ""
+        render_compare_frame(name, run_dir, rep, media / f"frnet_ot128_{name}_tt_vs_cpu{nc}.png")
+    render_gif(run_dir, rep, media / "frnet_ot128_pandaset_019_bev_tt.gif")
+    return 0
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    r = sub.add_parser("run")
+    r.add_argument("--out", required=True)
+    r.add_argument("--variants", nargs="*", default=list(VARIANTS))
+    r.add_argument("--frames", nargs="*", help="input stems (default: every converted public frame)")
+    r.add_argument("--weights-dir", default=None)
+    d = sub.add_parser("render")
+    d.add_argument("--out", required=True)
+    d.add_argument("--media", required=True)
+    a = ap.parse_args()
+    return cmd_run(a) if a.cmd == "run" else cmd_render(a)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/make_goldens.py

@@ -0,0 +1,198 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Goldens of frnet-p150 from the independent fp32 CPU reference (research venv: torch, onnx, onnxruntime; CPU only).
+
+    cd bundles/frnet-p150 && OMP_NUM_THREADS=4 PYTHONPATH=code \\
+        /home/ubuntu/experiments/tt-models/tools/research-venv/bin/python code/scripts/make_goldens.py --set all
+
+Sets (output under ``research/frnet/goldens`` unless ``--out``; never shipped, except ``host``):
+
+- ``taps``: the PandaSet 019 f40 sample (staged) in both variants through the bundle's host pre-processing and the
+  torch fp32 reference with every module tap (``reference/weights.py`` ``MODULE_TAPS`` + the per-voxel max of every
+  point branch, ``backbone.segmax0-4``) -> ``<variant>/pandaset_019_40.taps.npz`` (+ the four network inputs as
+  ``input.*`` and ONNX Runtime's ``pred_probs`` as ``ort.pred_probs``) and a ``.json`` with each tap's shape,
+  sha256 and its PCC / max |diff| against the same tensor of the deployed graph under ONNX Runtime (intermediates);
+- ``frames``: every catalogued frame (``tests/_data.py`` ``FRAMES``: 16 OT128 + 4 QT128 public goldens, the VLS-128
+  x > 0 halves) through the torch reference vs the stored ORT ``pred_probs``: PCC, max |diff|, argmax agreement (raw
+  rows and all rows) and the ORT top-2 margin of every disagreeing point -> ``manifest.json``;
+- ``host``: ``code/tt_frnet/tests/goldens/preprocess.json`` -- sha256 of the four network inputs that the RESEARCH
+  pre-processing reference (``research/frnet/scripts/frnet_preproc_ref.py``) produces for the PandaSet 019 f40 sample (with
+  its crop transform) and for the research synthetic scans, so the host tests can check the bundle's
+  pre-processing where the research tree is absent.
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+CODE = HERE.parents[1]
+sys.path.insert(0, str(CODE))
+
+WS = Path("/home/ubuntu/experiments/tt-models")
+RES = WS / "research" / "frnet"
+STEM = "pandaset_019_40"
+
+
+def sha(a: np.ndarray) -> str:
+    return hashlib.sha256(np.ascontiguousarray(a).tobytes()).hexdigest()
+
+
+def pcc64(a: np.ndarray, b: np.ndarray) -> float:
+    from tt_frnet.ttaw.metrics import pcc
+
+    return float(pcc(a, b))
+
+
+def make_taps(out: Path, weights_dir: str, threads: int) -> None:
+    import torch
+
+    from tt_frnet.reference.ort import OrtFRNet
+    from tt_frnet.reference.pipeline import FRNetReference
+    from tt_frnet.reference.weights import MODULE_TAPS
+    from tt_frnet.ttaw.golden import TapRegistry
+
+    torch.set_num_threads(threads)
+    patterns = list(MODULE_TAPS) + ["backbone.segmax*"]
+    for variant in ("ot128", "qt128"):
+        t0 = time.time()
+        ref = FRNetReference(weights_dir, variant, backend="torch", threads=threads)
+        taps = TapRegistry(include=patterns)
+        staged = CODE.parent / "staging_samples_pandaset"              # git-ignored (tests/_data.py)
+        T = json.loads((staged / "calib" / f"{STEM}.json").read_text())["T_base_link_from_sensor"]
+        run = ref.run(staged / "samples" / f"{STEM}.npz", calibration={"T_base_link_from_sensor": T}, taps=taps)
+        t_torch = time.time() - t0
+        inp = run.prepared.inputs
+        ort_taps = TapRegistry(include=patterns)
+        ort = OrtFRNet(weights_dir, variant, threads=threads, intermediates=True, tap_patterns=patterns,
+                       weights=ref.weights)
+        probs_ort = ort(inp.points, inp.coors, inp.voxel_coors, inp.inverse_map, taps=ort_taps)
+        del ort
+        gold = np.load(RES / "public_data" / "golden" / f"{STEM}_{variant}.npz", allow_pickle=False)
+        if not np.array_equal(probs_ort, gold["autoware_pred_probs"]):
+            sys.exit(f"{variant}: ORT pred_probs differ from the research golden (onnxruntime version?)")
+        meta = {"frame": STEM, "variant": variant, "reference": "tt_frnet.reference.model (torch fp32)",
+                "onnx_sha256": ref.weights.onnx_sha256, "torch": torch.__version__, "threads": threads,
+                "N": inp.num_points, "N_raw": inp.num_points_raw, "M": inp.num_voxels,
+                "torch_s": round(t_torch, 1), "taps": {}}
+        arrays = {f"input.{k}": v for k, v in inp.onnx_feeds().items()}
+        arrays["ort.pred_probs"] = probs_ort
+        for name in taps.names():
+            v = np.ascontiguousarray(taps[name], np.float32)
+            arrays[name] = v
+            entry = {"shape": list(v.shape), "sha256": sha(v)}
+            if name in ort_taps:
+                o = np.asarray(ort_taps[name], np.float32)
+                entry.update(pcc_vs_ort=pcc64(v, o), max_abs_vs_ort=float(np.abs(v - o).max()))
+            meta["taps"][name] = entry
+        worst = min((e["pcc_vs_ort"], k) for k, e in meta["taps"].items() if "pcc_vs_ort" in e)
+        meta["worst_tap_vs_ort"] = {"tap": worst[1], "pcc": worst[0]}
+        d = out / variant
+        d.mkdir(parents=True, exist_ok=True)
+        path = d / f"{STEM}.taps.npz"
+        np.savez(path, **arrays)
+        meta["file_sha256"] = hashlib.sha256(path.read_bytes()).hexdigest()
+        meta["bytes"] = path.stat().st_size
+        (d / f"{STEM}.taps.json").write_text(json.dumps(meta, indent=1) + "\n")
+        print(f"taps {variant}: {len(taps.names())} taps, {path.stat().st_size / 1e9:.2f} GB, worst vs ORT "
+              f"{worst[1]} {worst[0]:.12f}, {time.time() - t0:.0f} s", flush=True)
+        del taps, ort_taps, arrays
+
+
+def make_frames(out: Path, weights_dir: str, threads: int) -> None:
+    import torch
+
+    from tt_frnet.reference.model import FRNetTorch
+    from tt_frnet.reference.weights import load_weights
+    from tt_frnet.tests import _data
+
+    torch.set_num_threads(threads)
+    nets = {}
+    manifest = {"what": "torch fp32 reference vs the ORT pred_probs stored in each research golden",
+                "torch": torch.__version__, "threads": threads, "frames": {}}
+    for name, (rel, prefix, variant) in _data.FRAMES.items():
+        path = RES / rel
+        if not path.is_file():
+            print(f"{name}: missing {rel}", flush=True)
+            continue
+        if variant not in nets:
+            nets[variant] = FRNetTorch(load_weights(weights_dir, variant))
+        inputs, ort_probs, g = _data.load_frame(name)
+        t0 = time.time()
+        probs = nets[variant](inputs.points, inputs.coors, inputs.voxel_coors, inputs.inverse_map)
+        dt = time.time() - t0
+        n_raw = inputs.num_points_raw
+        a, b = probs.argmax(1), ort_probs.argmax(1)
+        diff = np.flatnonzero(a != b)
+        top2 = np.sort(ort_probs[diff], axis=1)[:, -2:] if len(diff) else np.zeros((0, 2), np.float32)
+        counts = np.bincount(inputs.inverse_map)
+        manifest["frames"][name] = {
+            "golden": f"research/frnet/{rel}", "variant": variant, "N": inputs.num_points, "N_raw": n_raw,
+            "M": inputs.num_voxels, "max_points_per_cell": int(counts.max()),
+            "pcc": pcc64(probs, ort_probs), "max_abs": float(np.abs(probs - ort_probs).max()),
+            "argmax_agreement": float((a == b).mean()), "argmax_agreement_raw": float((a[:n_raw] == b[:n_raw]).mean()),
+            "disagreements": int(len(diff)),
+            "disagreement_ort_margin_max": float((top2[:, 1] - top2[:, 0]).max()) if len(diff) else 0.0,
+            "torch_s": round(dt, 1)}
+        print(f"{name}: N {inputs.num_points} M {inputs.num_voxels} pcc {manifest['frames'][name]['pcc']:.12f} "
+              f"max|d| {manifest['frames'][name]['max_abs']:.2e} argmax {manifest['frames'][name]['argmax_agreement']:.6f}"
+              f" ({len(diff)} flips) {dt:.1f} s", flush=True)
+    out.mkdir(parents=True, exist_ok=True)
+    (out / "manifest.json").write_text(json.dumps(manifest, indent=1) + "\n")
+
+
+def make_host(weights_dir: str) -> None:
+    from tt_frnet.reference.config import FRNetConfig
+
+    sys.path.insert(0, str(RES / "scripts"))
+    import frnet_preproc_ref as R  # noqa: E402  (workspace research code, numpy only)
+
+    out = {"what": "sha256 of the four network inputs produced by research/frnet/scripts/frnet_preproc_ref.py "
+                   "(variant 'autoware'); the bundle's host pre-processing must reproduce them",
+           "cases": {}}
+    pub = RES / "public_data"
+    for variant in ("ot128", "qt128"):
+        cfg = FRNetConfig.from_model_dir(weights_dir, variant)
+        z = np.load(pub / "inputs" / f"{STEM}.npz", allow_pickle=False)
+        T = z["T_base_link_sensor"]
+        crop12 = np.concatenate([T[:3, :3].reshape(-1), T[:3, 3]]).astype(np.float32)
+        cases = {f"{STEM}.{variant}": (z["xyzi"], crop12, cfg.crop_bounds)}
+        pts, _ = R.synthetic_scan(R.CFGS[variant], n_cols=600 if variant == "qt128" else 1200, seed=0)
+        cases[f"synthetic_seed0.{variant}"] = (pts, None, None)
+        for name, (xyzi, crop, bounds) in cases.items():
+            pre = R.autoware_preprocess(xyzi, R.CFGS[variant], sensor_to_ref_12=crop, crop_bounds=bounds,
+                                        variant="autoware")
+            out["cases"][name] = {"N0": int(len(xyzi)), "N_raw": int(pre["num_points_raw"]),
+                                  "N": int(pre["num_points"]), "M": int(pre["num_unique"]),
+                                  "input_sha256": sha(np.ascontiguousarray(xyzi, np.float32)),
+                                  **{k: sha(pre[k]) for k in ("points", "coors", "voxel_coors", "inverse_map")}}
+            print(f"host {name}: N {pre['num_points']} M {pre['num_unique']}", flush=True)
+    path = CODE / "tt_frnet" / "tests" / "goldens" / "preprocess.json"
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps(out, indent=1) + "\n")
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--set", choices=("taps", "frames", "host", "all"), default="all")
+    ap.add_argument("--out", type=Path, default=RES / "goldens")
+    ap.add_argument("--weights-dir", default=str(WS / "assets" / "frnet" / "hf_lidar_frnet"))
+    ap.add_argument("--threads", type=int, default=4)
+    a = ap.parse_args()
+    if a.set in ("host", "all"):
+        make_host(a.weights_dir)
+    if a.set in ("frames", "all"):
+        make_frames(a.out, a.weights_dir, a.threads)
+    if a.set in ("taps", "all"):
+        make_taps(a.out, a.weights_dir, a.threads)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/make_sample.py

@@ -0,0 +1,169 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Build the SHIPPED samples of frnet-p150: one synthetic LiDAR sweep per serve profile, generated here (no dataset
+involved, Apache-2.0), and their stored fp32 CPU-reference outputs (the container smoke test's reference). CPU only,
+no device; the reference needs ONNX Runtime (research venv).
+
+    cd bundles/frnet-p150 && OMP_NUM_THREADS=4 PYTHONPATH=code \\
+        /home/ubuntu/experiments/tt-models/tools/research-venv/bin/python code/scripts/make_sample.py \\
+        [--weights-dir <lidar_frnet v2.0 files>] [--check-against <dir with the same files>]
+
+The scan is a deterministic ray cast from a sensor 2 m above a flat road (seed 0) against 25 cars, 16 building
+facades, 30 poles / trees and 10 pedestrians (axis-aligned boxes), with 5 % beam dropout, 2 cm range noise and
+random 0-255 intensities, in the **sensor frame** (x forward, y left, z up). The beam layout follows the profile's
+own sensor: 128 rows evenly spread over the vertical field of view (OT128 +15 / -25 deg, QT128 +-52.6 deg) and a
+fixed number of columns over 360 deg, chosen so that the frame lies inside Autoware's profile (the node skips a frame
+with more than 60,000 unique cells): OT128 512 columns, QT128 600 columns. The generator is a copy of the research
+reference ``research/frnet/scripts/frnet_preproc_ref.py synthetic_scan`` (``--check-against`` compares the arrays
+with an earlier output, e.g. the M5 bench inputs ``logs/frnet/m5_baseline/inputs``).
+
+Writes, per profile ``<v>`` (``ot128``, ``qt128``):
+
+- ``code/tt_frnet/samples/synthetic_<v>.npz``: ``points``, a structured (x, y, z, intensity) float32 array;
+- ``code/tt_frnet/calib/synthetic_<v>.json``: the calibration preset of the same name (sensor -> base_link = a pure
+  2 m lift: base_link on the road under the sensor), for the ego crop box;
+- ``code/tt_frnet/samples/synthetic_<v>.<v>.reference.json``: the ``/predict`` body of the fp32 CPU reference
+  (``tt_frnet.reference.pipeline`` with ONNX Runtime on the deployed ONNX, the reference of record), reduced to what
+  the container smoke compares (``labels``, ``scores``, ``filter_keep``, npz_compressed) plus the counts.
+
+Regenerate the references whenever the reference, the weights or the post-processing changes.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+CODE = HERE.parents[1]
+sys.path.insert(0, str(CODE))
+
+WS = Path("/home/ubuntu/experiments/tt-models")
+KEEP = ("model", "frame_id", "num_points", "num_points_raw", "class_names", "class_counts")
+# per profile: the sensor's vertical field of view (deg) and rows (the ml_package frustum height) and the columns
+SENSORS = {"ot128": dict(fov_up_deg=15.0, fov_down_deg=-25.0, rows=128, cols=512),
+           "qt128": dict(fov_up_deg=52.6, fov_down_deg=-52.6, rows=128, cols=600)}
+SENSOR_HEIGHT = 2.0
+SEED = 0
+
+
+def synthetic_scan(fov_up_deg: float, fov_down_deg: float, rows: int, n_cols: int, seed: int = 0,
+                   sensor_height: float = 2.0, max_range: float = 200.0):
+    """Ray cast against a ground plane and boxes; returns ``(points (n, 4) float32 x, y, z, intensity, labels)`` in
+    a seeded random order (the order a driver would not guarantee either). Same arithmetic and random stream as
+    ``research/frnet/scripts/frnet_preproc_ref.py synthetic_scan`` (cfg = the profile's FOV, ``interp_h`` rows)."""
+    rng = np.random.default_rng(seed)
+    elev = np.deg2rad(np.linspace(fov_down_deg + 0.05, fov_up_deg - 0.05, rows))
+    azim = np.deg2rad(np.arange(n_cols) * (360.0 / n_cols) - 180.0)
+    E, A = np.meshgrid(elev, azim, indexing="ij")
+    d = np.stack([np.cos(E) * np.cos(A), np.cos(E) * np.sin(A), np.sin(E)], axis=-1).reshape(-1, 3)
+    t_best = np.full(len(d), np.inf)
+    label = np.full(len(d), -1)
+    with np.errstate(divide="ignore", invalid="ignore"):          # ground plane z = -sensor_height
+        tg = np.where(d[:, 2] < -1e-6, -sensor_height / d[:, 2], np.inf)
+    upd = tg < t_best
+    t_best[upd] = tg[upd]
+    label[upd] = 0
+    boxes = []                                                    # (cx, cy, cz_bottom, lx, ly, lz, class)
+    for _ in range(25):                                           # cars
+        cx, cy = rng.uniform(-60, 60), rng.choice([-1, 1]) * rng.uniform(3, 12)
+        boxes.append((cx, cy, -sensor_height, 4.5, 1.9, 1.6, 5))
+    for side in (-1, 1):                                          # building facades
+        for k in range(8):
+            boxes.append((-80 + 20 * k, side * rng.uniform(18, 25), -sensor_height, 15, 3, rng.uniform(6, 20), 3))
+    for _ in range(30):                                           # poles / trees
+        boxes.append((rng.uniform(-70, 70), rng.choice([-1, 1]) * rng.uniform(13, 16), -sensor_height, .4, .4, 6, 4))
+    for _ in range(10):                                           # pedestrians
+        boxes.append((rng.uniform(-30, 30), rng.choice([-1, 1]) * rng.uniform(12, 14), -sensor_height, .6, .6, 1.7,
+                      17))
+    for (cx, cy, cz, lx, ly, lz, cls) in boxes:
+        lo = np.array([cx - lx / 2, cy - ly / 2, cz])
+        hi = np.array([cx + lx / 2, cy + ly / 2, cz + lz])
+        with np.errstate(divide="ignore", invalid="ignore"):
+            inv = 1.0 / d
+            t1 = lo * inv
+            t2 = hi * inv
+        tmin = np.nanmax(np.minimum(t1, t2), axis=1)
+        tmax = np.nanmin(np.maximum(t1, t2), axis=1)
+        hit = (tmax >= np.maximum(tmin, 0)) & (tmin > 0.5)
+        upd = hit & (tmin < t_best)
+        t_best[upd] = tmin[upd]
+        label[upd] = cls
+    ok = np.isfinite(t_best) & (t_best < max_range)
+    ok &= rng.random(len(d)) > 0.05                               # beam dropout
+    xyz = d[ok] * (t_best[ok, None] + rng.normal(0, 0.02, size=(ok.sum(), 1)))   # range noise
+    inten = np.clip(rng.normal(40, 25, size=ok.sum()), 0, 255).round()          # 0..255 intensity
+    pts = np.concatenate([xyz, inten[:, None]], axis=1).astype(np.float32)
+    perm = rng.permutation(len(pts))
+    return pts[perm], label[ok][perm]
+
+
+def structured(xyzi: np.ndarray) -> np.ndarray:
+    pts = np.zeros(len(xyzi), dtype=[("x", "<f4"), ("y", "<f4"), ("z", "<f4"), ("intensity", "<f4")])
+    for i, name in enumerate(("x", "y", "z", "intensity")):
+        pts[name] = xyzi[:, i]
+    return pts
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--weights-dir", default=str(WS / "assets" / "frnet" / "hf_lidar_frnet"))
+    ap.add_argument("--threads", type=int, default=4)
+    ap.add_argument("--variants", nargs="*", default=list(SENSORS))
+    ap.add_argument("--check-against", type=Path, help="directory with synthetic_<v>.npz to compare the points with")
+    ap.add_argument("--no-reference", action="store_true", help="write the scans and presets only")
+    a = ap.parse_args()
+    samples, calib = CODE / "tt_frnet" / "samples", CODE / "tt_frnet" / "calib"
+    T = np.eye(4)
+    T[2, 3] = SENSOR_HEIGHT
+    for variant in a.variants:
+        s = SENSORS[variant]
+        xyzi, _ = synthetic_scan(s["fov_up_deg"], s["fov_down_deg"], s["rows"], s["cols"], seed=SEED,
+                                 sensor_height=SENSOR_HEIGHT)
+        pts = structured(xyzi)
+        stem = f"synthetic_{variant}"
+        if a.check_against is not None:
+            old = np.load(a.check_against / f"{stem}.npz", allow_pickle=False)["points"]
+            if not (old.dtype == pts.dtype and np.array_equal(old, pts)):
+                sys.exit(f"{stem}: the points differ from {a.check_against}")
+            print(f"{stem}: identical to {a.check_against / (stem + '.npz')}", flush=True)
+        np.savez_compressed(samples / f"{stem}.npz", points=pts)
+        preset = {"name": stem,
+                  "description": f"synthetic {variant.upper()}-like scan (code/scripts/make_sample.py): the sensor "
+                                 f"{SENSOR_HEIGHT:g} m above a flat road, base_link on the road under it (a pure lift)",
+                  "source": "code/scripts/make_sample.py (generated, no dataset; Apache-2.0)",
+                  "T_base_link_from_sensor": T.tolist()}
+        (calib / f"{stem}.json").write_text(json.dumps(preset, indent=1) + "\n")
+        print(f"{stem}: {len(pts)} points ({s['rows']} rows x {s['cols']} columns, seed {SEED})", flush=True)
+        if a.no_reference:
+            continue
+        from tt_frnet.io import encode_array
+        from tt_frnet.reference.pipeline import FRNetReference
+
+        ref = FRNetReference(a.weights_dir, variant, backend="ort", threads=a.threads)
+        run = ref.run(samples / f"{stem}.npz", calibration={"preset": stem})
+        if run.prepared.skipped is not None:
+            sys.exit(f"{stem}: outside the {variant} profile: {run.prepared.skipped}")
+        out = run.output
+        body = {k: v for k, v in out.to_dict().items() if k in KEEP}
+        body["labels"] = encode_array(out.label_ids, fmt="npz_compressed", key="labels")
+        body["scores"] = encode_array(out.scores, fmt="npz_compressed", key="scores")
+        body["filter_keep"] = encode_array(out.filter_keep, fmt="npz_compressed", key="filter_keep")
+        inp = run.prepared.inputs
+        body["meta"] = {"variant": variant, "reference": "onnxruntime fp32 (deployed ONNX)",
+                        "onnx_sha256": ref.weights.onnx_sha256, "num_unique_coors": inp.num_voxels,
+                        "cropped_by_ego_box": int((~inp.keep_mask).sum()),
+                        "generator": f"code/scripts/make_sample.py synthetic_scan({s['rows']} rows, {s['cols']} "
+                                     f"columns, seed {SEED})"}
+        path = samples / f"{stem}.{variant}.reference.json"
+        path.write_text(json.dumps(body) + "\n")
+        print(f"{stem}: N {inp.num_points} (raw {inp.num_points_raw}) M {inp.num_voxels} -> {path.name} "
+              f"({path.stat().st_size / 1e6:.2f} MB)", flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/make_samples.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Build the STAGED PandaSet sample of frnet-p150 and its stored CPU-reference outputs (research venv: numpy, onnx,
+onnxruntime; CPU only, no device). It does not ship until the user approves PandaSet-derived samples
+(research/DATASETS.md N5): it goes to the git-ignored ``staging_samples_pandaset/`` of the bundle checkout (laid out
+like the package: ``samples/``, ``calib/``), where the tests find it; the frozen ``sample`` gates and the PCC frame
+still use it. The shipped samples are the synthetic scans of ``code/scripts/make_sample.py``.
+
+    cd bundles/frnet-p150 && OMP_NUM_THREADS=4 PYTHONPATH=code \\
+        /home/ubuntu/experiments/tt-models/tools/research-venv/bin/python code/scripts/make_samples.py
+
+Writes (from research/frnet/public_data, PandaSet 019 frame 40, CC BY 4.0 + PandaSet Dataset Terms; D18), under
+``staging_samples_pandaset/``:
+
+- ``samples/pandaset_019_40.npz``: ``points``, a structured (x, y, z, intensity) float32 array of the
+  sensor-frame Pandar64 sweep, byte-identical values to ``shippable/pandaset_019_40_pandar64_xyzi.npy`` (sha256
+  checked against its meta);
+- ``samples/PANDASET_LICENSE.txt``: the PandaSet Dataset Terms of Use (verbatim, sha256 checked);
+- ``calib/pandaset_019_40.json``: the sensor -> base_link transform at full float64 precision (the
+  converted input's ``T_base_link_sensor``: its float32 cast is the research golden's crop transform bit for bit;
+  the 6-decimal copy in the sample meta is not);
+- ``samples/pandaset_019_40.<variant>.reference.json`` (ot128, qt128): the ``to_dict()`` of the
+  reference pipeline with ONNX Runtime on the deployed ONNX (the reference of record, S:frnet:361-365), reduced to
+  what the container smoke compares (``labels``, ``scores``, ``filter_keep``, npz_compressed) plus counts. The class
+  ids are checked against the research golden ``public_data/golden/pandaset_019_40_<variant>.npz`` (equal).
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import shutil
+import sys
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+CODE = HERE.parents[1]
+sys.path.insert(0, str(CODE))
+
+from tt_frnet.io import encode_array  # noqa: E402
+from tt_frnet.reference.config import VARIANTS  # noqa: E402
+
+WS = Path("/home/ubuntu/experiments/tt-models")
+PUB = WS / "research" / "frnet" / "public_data"
+STEM = "pandaset_019_40"
+KEEP = ("model", "frame_id", "num_points", "num_points_raw", "class_names", "class_counts")
+
+
+def sha256(path: Path) -> str:
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--weights-dir", default=str(WS / "assets" / "frnet" / "hf_lidar_frnet"))
+    ap.add_argument("--threads", type=int, default=4)
+    ap.add_argument("--variants", nargs="*", default=list(VARIANTS))
+    a = ap.parse_args()
+    staging = CODE.parent / "staging_samples_pandaset"
+    samples, calib = staging / "samples", staging / "calib"
+    samples.mkdir(parents=True, exist_ok=True)
+    calib.mkdir(parents=True, exist_ok=True)
+    meta = json.loads((PUB / "shippable" / f"{STEM}_meta.json").read_text())
+    src = PUB / "shippable" / f"{STEM}_pandar64_xyzi.npy"
+    lic = PUB / "shippable" / "LICENSE.txt"
+    for path in (src, lic):
+        if sha256(path) != meta["files"][path.name]:
+            sys.exit(f"{path}: sha256 differs from {STEM}_meta.json")
+    xyzi = np.load(src, allow_pickle=False)
+    pts = np.zeros(len(xyzi), dtype=[("x", "<f4"), ("y", "<f4"), ("z", "<f4"), ("intensity", "<f4")])
+    for i, name in enumerate(("x", "y", "z", "intensity")):
+        pts[name] = xyzi[:, i]
+    np.savez_compressed(samples / f"{STEM}.npz", points=pts)
+    shutil.copyfile(lic, samples / "PANDASET_LICENSE.txt")
+    T = np.load(PUB / "inputs" / f"{STEM}.npz", allow_pickle=False)["T_base_link_sensor"].astype(np.float64)
+    preset = {"name": STEM,
+              "description": "PandaSet 019 frame 40: static Pandar64 sensor -> base_link transform (fitted mount; "
+                             "base_link = the lidar-pose origin projected to the road, x forward) for the ego crop box",
+              "source": "research/frnet/public_data/inputs/pandaset_019_40.npz T_base_link_sensor "
+                        "(calib/pandaset_pandar64_mount.json fit)",
+              "T_base_link_from_sensor": [[float(v) for v in row] for row in T]}
+    (calib / f"{STEM}.json").write_text(json.dumps(preset, indent=1) + "\n")
+    print(f"sample {len(pts)} points, calibration preset {STEM}", flush=True)
+
+    from tt_frnet.reference.pipeline import FRNetReference
+
+    for variant in a.variants:
+        ref = FRNetReference(a.weights_dir, variant, backend="ort", threads=a.threads)
+        run = ref.run(samples / f"{STEM}.npz",
+                      calibration={"T_base_link_from_sensor": preset["T_base_link_from_sensor"]})
+        gold = np.load(PUB / "golden" / f"{STEM}_{variant}.npz", allow_pickle=False)
+        out = run.output
+        if not (np.array_equal(run.prepared.inputs.points, gold["autoware_points"]) and
+                np.array_equal(out.label_ids, gold["autoware_class_id"]) and
+                np.array_equal(out.filter_keep, gold["autoware_filter_keep"])):
+            sys.exit(f"{variant}: the reference output differs from the research golden")
+        body = {k: v for k, v in out.to_dict().items() if k in KEEP}
+        body["labels"] = encode_array(out.label_ids, fmt="npz_compressed", key="labels")
+        body["scores"] = encode_array(out.scores, fmt="npz_compressed", key="scores")
+        body["filter_keep"] = encode_array(out.filter_keep, fmt="npz_compressed", key="filter_keep")
+        body["meta"] = {"variant": variant, "reference": "onnxruntime fp32 (deployed ONNX)",
+                        "onnx_sha256": ref.weights.onnx_sha256, "num_unique_coors": run.prepared.inputs.num_voxels,
+                        "cropped_by_ego_box": int((~run.prepared.inputs.keep_mask).sum())}
+        path = samples / f"{STEM}.{variant}.reference.json"
+        path.write_text(json.dumps(body) + "\n")
+        print(f"{variant}: N {out.num_points if hasattr(out, 'num_points') else len(out)} "
+              f"M {run.prepared.inputs.num_voxels} -> {path.name} ({path.stat().st_size / 1e6:.2f} MB)", flush=True)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/precision_study.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""First-order CPU emulation of the TT numerics of FRNet, to size the fp32 islands of the port before the device runs
+(research venv: torch + onnx; CPU only, no device).
+
+    cd bundles/frnet-p150 && OMP_NUM_THREADS=4 PYTHONPATH=code \\
+        /home/ubuntu/experiments/tt-models/tools/research-venv/bin/python code/scripts/precision_study.py \\
+        --out /home/ubuntu/experiments/tt-models/logs/frnet/m0/precision_study.json
+
+The TT dataflow (``reference/rewrites.py`` ``FRNetPlanTorch``) runs with operand rounding per site
+(``Emulation``): ``bf16`` (RNE), ``tf32`` (truncation to 10 mantissa bits: probe P12's model of the device's fp32
+matmul, whose measured error is 1.4-1.75x that model's, so ``tf32`` rows are optimistic). Configurations:
+
+- ``bf16_all``: bf16 everywhere (weights and activations): the research simulation's worst case;
+- ``plan``: the planned functional numerics -- conv activations bf16 with fp32 weights (PLAN.md 9.3 precision policy),
+  the point / voxel layers as an fp32 island (device fp32 matmul ~ tf32), bf16 rows out of the ``ttnn.embedding``
+  gathers (bf16-only tables), fp32 interpolation matmuls (~ tf32);
+- ``plan_bf16_points``: ``plan`` with bf16 activations and fp32 weights in the point layers (the fast path);
+- ``plan_fp32_gather``: ``plan`` with exact fp32 gathers (what a fp32 row-gather kernel, K3, would buy);
+- ``plan_fp32_convs`` / ``plan_exact_points``: ``plan`` with fp32 conv activations, resp. exact fp32 point layers
+  (e.g. two-term bf16 operands): which island to deepen first when a frame misses a gate.
+
+Metrics per frame against the stored ONNX Runtime ``pred_probs`` (the reference of record): PCC over all points and
+argmax agreement on the raw points (the PLAN.md 2.8 gates: >= 0.998 and >= 98.5 %), and the module PCC of the device
+modules' outputs against the exact fp32 dataflow (gate >= 0.999). Indicative only: no math-fidelity phases, no tile
+layouts, no on-device accumulation order.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+
+HERE = Path(__file__).resolve()
+sys.path.insert(0, str(HERE.parents[1]))
+
+WS = Path("/home/ubuntu/experiments/tt-models")
+CONFIGS = {
+    "bf16_all": "conv_act=bf16,conv_weight=bf16,point_act=bf16,point_weight=bf16,gather=bf16,upsample=bf16",
+    "plan": "conv_act=bf16,point_act=tf32,point_weight=tf32,gather=bf16,upsample=tf32",
+    "plan_bf16_points": "conv_act=bf16,point_act=bf16,gather=bf16,upsample=tf32",
+    "plan_fp32_gather": "conv_act=bf16,point_act=tf32,point_weight=tf32,upsample=tf32",
+    "plan_fp32_convs": "point_act=tf32,point_weight=tf32,gather=bf16,upsample=tf32",
+    "plan_exact_points": "conv_act=bf16,gather=bf16,upsample=tf32",
+    # M1b device follow-up (PORT_LOG "M1 device results"): the device's float32 conv-weight path is not exact
+    # (C17 / YOLOX: TF32-like, close to bf16 weights); the point island exact (two-term operands)
+    "exact_points_tf32_conv_w": "conv_act=bf16,conv_weight=tf32,gather=bf16,upsample=tf32",
+    "exact_points_bf16_conv_w": "conv_act=bf16,conv_weight=bf16,gather=bf16,upsample=tf32",
+}
+FRAMES = ["pandaset_019_40.ot128", "pandaset_029_40.ot128", "nuscenes_scene-0103_20.ot128",
+          "nuscenes_scene-0916_20.ot128", "vls128_frame000_fronthalf.ot128", "pandaset_019_40.qt128",
+          "vls128_frame000_fronthalf.qt128"]
+MODULES = ("encoder.pf3", "encoder.voxel_feats", "backbone.stem", "backbone.point_stem", "backbone.fusion_stem",
+           "backbone.stage1", "backbone.stage2", "backbone.stage3", "backbone.stage4", "decoder.fuse2",
+           "decoder.point_fuse2", "head.logits")
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--out", type=Path, required=True)
+    ap.add_argument("--frames", nargs="*", default=FRAMES)
+    ap.add_argument("--configs", nargs="*", default=list(CONFIGS)[:4])
+    ap.add_argument("--threads", type=int, default=4)
+    ap.add_argument("--weights-dir", default=str(WS / "assets" / "frnet" / "hf_lidar_frnet"))
+    a = ap.parse_args()
+
+    import torch
+
+    from tt_frnet.host.preprocess import build_plan
+    from tt_frnet.reference.config import FRNetConfig
+    from tt_frnet.reference.rewrites import Emulation, FRNetPlanTorch
+    from tt_frnet.reference.weights import load_weights
+    from tt_frnet.tests import _data
+    from tt_frnet.ttaw.golden import TapRegistry
+    from tt_frnet.ttaw.metrics import pcc
+
+    torch.set_num_threads(a.threads)
+    weights, configs = {}, {}
+    result = {"configs": {k: CONFIGS[k] for k in a.configs}, "frames": {}, "note": __doc__.split("\n\n")[2]}
+    for frame in a.frames:
+        variant = frame.rpartition(".")[2]
+        if variant not in weights:
+            weights[variant] = load_weights(a.weights_dir, variant)
+            configs[variant] = FRNetConfig.from_model_dir(a.weights_dir, variant)
+        w = weights[variant]
+        inputs, ort_probs, _ = _data.load_frame(frame)
+        plan = build_plan(inputs, configs[variant], pre_norm=(w.pre_norm_scale, w.pre_norm_shift))
+        n_raw = inputs.num_points_raw
+        ref_label = ort_probs.argmax(1)
+        t0 = time.time()
+        exact = TapRegistry(include=list(MODULES))
+        FRNetPlanTorch(w)(plan, taps=exact)
+        rows = {}
+        for name in a.configs:
+            taps = TapRegistry(include=list(MODULES))
+            probs = plan.unsort(FRNetPlanTorch(w, emulation=Emulation.parse(CONFIGS[name]))(plan, taps=taps))
+            label = probs.argmax(1)
+            rows[name] = {"pred_probs_pcc": float(pcc(probs, ort_probs)),
+                          "argmax_agreement_raw": float((label[:n_raw] == ref_label[:n_raw]).mean()),
+                          "argmax_agreement_all": float((label == ref_label).mean()),
+                          "modules": {m: float(pcc(taps[m][: plan.n] if taps[m].shape[0] == plan.n_cap else taps[m],
+                                                   exact[m][: plan.n] if exact[m].shape[0] == plan.n_cap else exact[m]))
+                                      for m in MODULES}}
+            r = rows[name]
+            worst = min(r["modules"].items(), key=lambda kv: kv[1])
+            print(f"{frame} {name}: pcc {r['pred_probs_pcc']:.6f} argmax(raw) {r['argmax_agreement_raw']:.4%} "
+                  f"worst module {worst[0]} {worst[1]:.6f}", flush=True)
+        result["frames"][frame] = {"N": inputs.num_points, "N_raw": n_raw, "M": inputs.num_voxels,
+                                   "seconds": round(time.time() - t0, 1), "results": rows}
+        a.out.parent.mkdir(parents=True, exist_ok=True)
+        a.out.write_text(json.dumps(result, indent=1) + "\n")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

code/scripts/profile_ops.py

@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Device profile of frnet-p150: one eager forward and one traced replay of the whole graph between Tracy
+signposts, for one serve profile (``--variant ot128 | qt128``).
+
+    ROOT=/home/ubuntu/experiments/tt-models
+    $ROOT/bin/devrun -t 3600 -- python -m tracy -r -p -v --no-web-server --op-support-count 4000 \\
+        -o $ROOT/generated/profiler/frnet_baseline code/scripts/profile_ops.py --variant ot128
+    tt-perf-report <ops_perf_results_*.csv> --start-signpost trace --end-signpost trace_end --tracing-mode
+    python code/scripts/profile_summary.py <the same csv> profile_summary.json --md profile_summary.md
+
+Sections (signposts): ``eager`` .. ``eager_end``, one eager forward after the warm-up (``TraceRunner.run_eager``:
+the same graph and programs as the trace, nothing compiled), and ``trace`` .. ``trace_end``, ``--replays`` replays
+of the captured ``default`` trace (the served device path; the uploaded inputs are the plan of ``--input``). The
+device profiler buffer is flushed (``ttnn.ReadDeviceProfiler``) before and after each section. Inside the eager
+section every module of the graph opens a signpost (``m:<module>``: ``encoder``, ``segmax``, ``frustum2pixel``,
+``pixel2point``, ``stem``, ``point_stem``, ``fusion_stem``, ``layer<l>``, ``point_fusion<l>``, ``pixel_fusion<l>``,
+``attention<l>``, ``decoder.up``, ``decoder.fuse1``, ``decoder.fuse2``, ``point_fuse``, ``head``) and every layer a
+layer signpost (``c:<layer>``; ``--no-layer-signposts`` turns them off). Ops that the graph issues inline after a
+module (the residual ``add`` + ``hardswish`` of a BasicBlock, ``addcmul`` after the attention, the softmax after the
+logits) belong to the module signposted last. Through the identical op order the ops of the replay inherit the
+eager attribution (``profile_summary.py``). The output folder must be absolute and outside the bundle (PLAN.md 5.1).
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import time
+from pathlib import Path
+from typing import Callable, List, Sequence
+
+from tt_frnet import FRNet
+from tt_frnet.ttaw.profiling import read_device_profiler, signpost, signposted
+
+CODE = Path(__file__).resolve().parents[1]
+SAMPLES = CODE / "tt_frnet" / "samples"           # the shipped synthetic_<variant>.npz
+
+
+class _Signposted:
+    """A callable stand-in for a module that emits its signposts, then calls the module (attributes forward)."""
+
+    def __init__(self, inner, labels: Sequence[str]):
+        self.inner, self._labels = inner, list(labels)
+
+    def __call__(self, *args, **kwargs):
+        for label in self._labels:
+            signpost(label)
+        return self.inner(*args, **kwargs)
+
+    def __getattr__(self, name):
+        return getattr(self.inner, name)
+
+
+def module_signposts(graph, layers: bool = True) -> Callable[[], None]:
+    """Wrap the modules of a ``tt.model.TtFRNet`` (and the C19 gathers its forward calls) with signposting
+    stand-ins; returns the function that restores them."""
+    from tt_frnet.tt import model as tt_model
+
+    undo: List[Callable[[], None]] = []
+
+    def labels(module: str, layer: str) -> List[str]:
+        return [f"m:{module}"] + ([f"c:{layer}"] if layers else [])
+
+    def swap(obj, attr, lab):
+        inner = getattr(obj, attr)
+        setattr(obj, attr, _Signposted(inner, lab))
+        undo.append(lambda: setattr(obj, attr, inner))
+
+    def swap_item(seq, key, lab):
+        inner = seq[key]
+        seq[key] = _Signposted(inner, lab)
+        undo.append(lambda: seq.__setitem__(key, inner))
+
+    for fn_name in ("frustum2pixel", "pixel2point"):
+        orig = getattr(tt_model, fn_name)
+
+        def wrapped(*args, _orig=orig, _name=fn_name, **kwargs):
+            signpost(f"m:{_name}")
+            return _orig(*args, **kwargs)
+
+        setattr(tt_model, fn_name, wrapped)
+        undo.append(lambda _n=fn_name, _o=orig: setattr(tt_model, _n, _o))
+    for i in range(len(graph.ffe)):
+        swap_item(graph.ffe, i, labels("encoder", f"encoder.ffe{i}"))
+    swap(graph, "compression", labels("encoder", "encoder.compression"))
+    swap(graph, "segmax_encoder", labels("segmax", "segmax.encoder"))
+    swap(graph, "segmax_points", labels("segmax", "segmax.points"))
+    for i in range(len(graph.stem)):
+        swap_item(graph.stem, i, labels("stem", f"backbone.stem{i}"))
+    swap(graph, "point_stem", labels("point_stem", "backbone.point_stem"))
+    swap(graph, "fusion_stem", labels("fusion_stem", "backbone.fusion_stem"))
+    for st in graph.stages:
+        l = st["index"]
+        blocks = st["blocks"]
+        for b, (c1, c2, ds) in enumerate(blocks):
+            pre = f"backbone.layer{l}.block{b}"
+            new = (_Signposted(c1, labels(f"layer{l}", f"{pre}.conv1")),
+                   _Signposted(c2, labels(f"layer{l}", f"{pre}.conv2")),
+                   None if ds is None else _Signposted(ds, labels(f"layer{l}", f"{pre}.downsample")))
+            blocks[b] = new
+            undo.append(lambda _bl=blocks, _b=b, _o=(c1, c2, ds): _bl.__setitem__(_b, _o))
+        swap_item(st, "point_fusion", labels(f"point_fusion{l}", f"backbone.point_fusion{l}"))
+        swap_item(st, "pixel_fusion", labels(f"pixel_fusion{l}", f"backbone.pixel_fusion{l}"))
+        for j in range(len(st["attention"])):
+            swap_item(st["attention"], j, labels(f"attention{l}", f"backbone.attention{l}.conv{j}"))
+    for s in list(graph.ups):
+        swap_item(graph.ups, s, labels("decoder.up", f"decoder.up{s}"))
+    swap(graph, "fuse1", labels("decoder.fuse1", "decoder.fuse1"))
+    swap(graph, "fuse2", labels("decoder.fuse2", "decoder.fuse2"))
+    swap(graph, "point_fuse1", labels("point_fuse", "decoder.point_fuse1"))
+    swap(graph, "point_fuse2", labels("point_fuse", "decoder.point_fuse2"))
+    for i in range(len(graph.mlps)):
+        swap_item(graph.mlps, i, labels("head", f"head.mlp{i}"))
+    swap(graph, "logits", labels("head", "head.logits"))
+
+    def restore() -> None:
+        for fn in reversed(undo):
+            fn()
+
+    return restore
+
+
+def calibration_for(path: Path, spec=None) -> dict:
+    """The ``calibration`` of ``path`` (the rules of ``bench.py calibration_for``)."""
+    if spec:
+        if spec == "none":
+            return {"ego_crop_box": None}
+        if spec.startswith("preset:"):
+            return {"preset": spec[len("preset:"):]}
+        return json.loads(Path(spec).read_text())
+    stem = path.name.split(".")[0]
+    if (CODE / "tt_frnet" / "calib" / f"{stem}.json").is_file():
+        return {"preset": stem}
+    side = path.parent / f"{stem}.calib.json"
+    return json.loads(side.read_text()) if side.is_file() else {"ego_crop_box": None}
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    ap.add_argument("--input", default=None, help="default: the shipped synthetic sample of the variant")
+    ap.add_argument("--calib", default=None, help="preset:<name> | <calibration .json> | none (bench.py rules)")
+    ap.add_argument("--variant", default=None, help="ot128 | qt128 (default: FRNET_VARIANT or ot128)")
+    ap.add_argument("--dispatch", default=None, choices=["eth", "worker"])
+    ap.add_argument("--cqs", type=int, default=None, choices=[1, 2])
+    ap.add_argument("--replays", type=int, default=1, help="replays inside the trace section")
+    ap.add_argument("--no-eager", dest="eager", action="store_false")
+    ap.add_argument("--no-layer-signposts", dest="layers", action="store_false")
+    a = ap.parse_args()
+    import ttnn
+
+    src = Path(a.input or SAMPLES / f"synthetic_{a.variant or os.environ.get('FRNET_VARIANT', 'ot128')}.npz")
+    calib = calibration_for(src, a.calib)
+    t0 = time.perf_counter()
+    with FRNet.from_pretrained(variant=a.variant, dispatch=a.dispatch, num_command_queues=a.cqs) as model:
+        tr, dev = model.runner, model.device
+        print("loaded in %.1f s:" % (time.perf_counter() - t0), json.dumps(model.device_info), flush=True)
+        prepared = model._prepare(points=str(src), calibration=calib)
+        out = model(str(src), calibration=calib)       # one served frame: upload + replay + readback
+        print(f"variant {model.variant}: N {prepared.plan.n} M {prepared.plan.m} (N_cap {prepared.plan.n_cap}, "
+              f"M_cap {prepared.plan.m_cap}); {len(out.label_ids)} labelled points", flush=True)
+        ttnn.synchronize_device(dev)
+        read_device_profiler(dev)                      # warm-up / capture / first frame out of the buffer
+        inputs = model.staging.write(prepared.plan)
+        if a.eager:
+            restore = module_signposts(model.graph, layers=a.layers)
+            try:
+                with signposted("eager"):
+                    tr.run_eager("default", inputs=inputs)
+                    ttnn.synchronize_device(dev)
+            finally:
+                restore()
+            read_device_profiler(dev)
+        tr.upload(inputs)
+        ttnn.synchronize_device(dev)
+        read_device_profiler(dev)
+        with signposted("trace"):
+            tr.replay("default", n=a.replays)
+            ttnn.synchronize_device(dev)
+        read_device_profiler(dev)
+        print("trace:", json.dumps(tr.describe()), flush=True)
+
+
+if __name__ == "__main__":
+    main()

code/scripts/profile_summary.py

@@ -0,0 +1,251 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""Summary of a device profile of ``profile_ops.py`` (CPU only, standard library): the OPT_BASELINE.md tables.
+
+    python code/scripts/profile_summary.py <ops_perf_results_*.csv> <out.json> [--md out.md]
+
+Reads the Tracy ops CSV (``python -m tracy -r ...``) and its sections ``eager`` .. ``eager_end`` and ``trace`` ..
+``trace_end``. The eager section carries per-module (``m:<module>``) and per-conv (``c:<layer>``) signposts; the
+replay has the same op sequence (checked op code by op code), so every replayed op inherits the module / layer of
+the eager op at the same position. Per section: program count, device kernel sum, FW sum, op-to-op gap sum and
+median, span (first FW start -> last FW end at the CSV's chip frequency), cores per op, math fidelities; tables per
+category, module, op code and layer; the top single ops; the largest gaps. If the trace section holds several
+replays (``profile_ops.py --replays N``), the last replay session is summarised. Input shapes of the CSV are those of
+the first op that built a cached program (``--no-op-info-cache`` in tracy shows every op's own), so read shapes of
+later ops of the same program with care; durations are per op.
+"""
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import re
+import statistics
+from collections import OrderedDict, defaultdict
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+KERNEL = "DEVICE KERNEL DURATION [ns]"
+FW = "DEVICE FW DURATION [ns]"
+GAP = "OP TO OP LATENCY [ns]"
+START = "DEVICE FW START CYCLE"
+END = "DEVICE FW END CYCLE"
+
+CATEGORIES = [  # (category, regex on the op code), first match wins
+    ("segment_reduce (K1 generic_op)", r"GenericOp|generic_op|Generic"),
+    ("conv2d", r"Conv2d|conv2d|Conv2D"),
+    ("halo", r"Halo|halo"),
+    ("matmul (point linears, 1x1 convs, resize matmuls)", r"Matmul|matmul|Linear"),
+    ("slice / DRAM-slicing copies", r"Slice|slice"),
+    ("ternary (addcmul)", r"Ternary|ternary|Addcmul|addcmul"),
+    ("binary (adds, two-term / K-split sums)", r"Binary|binary"),
+    ("softmax", r"Softmax|softmax"),
+    ("reduction", r"Reduce|reduce|Max\b|MaxDevice"),
+    ("embedding (gathers)", r"Embedding|embedding"),
+    ("tilize / untilize / layout", r"Tilize|tilize|Untilize|untilize|ToLayout|Layout"),
+    ("reshape / transpose / permute / concat", r"Reshape|reshape|Transpose|transpose|Permute|permute|View|Concat|concat"),
+    ("move / copy / reshard", r"Move|move|Copy|copy|Shard|shard|Interleaved|interleaved|Reallocate"),
+    ("pad / fill", r"Pad|pad|Fill|fill"),
+    ("unary / typecast (hardswish, relu, sigmoid)", r"Unary|unary|Typecast|typecast"),
+]
+COMPUTE = ("Conv2dDeviceOperation", "MatmulDeviceOperation")
+
+
+def category(op: str) -> str:
+    for name, pat in CATEGORIES:
+        if re.search(pat, op):
+            return name
+    return "other"
+
+
+def num(row: Dict[str, str], key: str) -> float:
+    v = (row.get(key) or "").strip()
+    try:
+        return float(v) if v else 0.0
+    except ValueError:
+        return 0.0
+
+
+def logical(v: str) -> str:
+    """'32[16]' (padded[logical]) -> '16'; '7' -> '7'."""
+    m = re.match(r"\s*([0-9]+)(?:\[([0-9]+)\])?", v or "")
+    return (m.group(2) or m.group(1)) if m else "?"
+
+
+def chip_freq_mhz(csv_path: Path) -> Optional[float]:
+    for cand in [csv_path.parent / "profile_log_device.csv", csv_path.parent.parent / "profile_log_device.csv",
+                 csv_path.parent / ".logs" / "profile_log_device.csv",
+                 csv_path.parent.parent / ".logs" / "profile_log_device.csv"]:
+        try:
+            head = cand.read_text(errors="ignore").splitlines()[0]
+        except (OSError, IndexError):
+            continue
+        m = re.search(r"CHIP_FREQ\[MHz\]:\s*([0-9.]+)", head)
+        if m:
+            return float(m.group(1))
+    return None
+
+
+def sections(rows: List[Dict[str, str]], name_key: str) -> Dict[str, List[Tuple[Dict[str, str], str, str]]]:
+    """{"eager": [(row, module, layer)], "trace": [...]} from the signposts."""
+    out: Dict[str, List[Tuple[Dict[str, str], str, str]]] = {"eager": [], "trace": []}
+    cur: Optional[str] = None
+    module, layer = "", ""
+    for r in rows:
+        if (r.get("OP TYPE") or "") == "signpost":
+            code = r.get(name_key, "")
+            if code in ("eager", "trace"):
+                cur, module, layer = code, "", ""
+            elif code in ("eager_end", "trace_end"):
+                cur = None
+            elif code.startswith("m:"):
+                module, layer = code[2:], ""
+            elif code.startswith("c:"):
+                layer = code[2:]
+            continue
+        if cur is not None:
+            out[cur].append((r, module, layer))
+    return out
+
+
+def last_replay(sec: List[Tuple[Dict[str, str], str, str]]) -> List[Tuple[Dict[str, str], str, str]]:
+    """The ops of the last trace replay session (all of them when the CSV has no session ids)."""
+    ids = [(r.get("METAL TRACE REPLAY SESSION ID") or "").strip() for r, _, _ in sec]
+    if not any(ids):
+        return sec
+    last = [i for i in ids if i][-1]
+    return [x for x, i in zip(sec, ids) if i == last]
+
+
+def summarize(sec: List[Tuple[Dict[str, str], str, str]], name_key: str, freq: float, top: int = 10) -> Dict[str, Any]:
+    rows = [r for r, _, _ in sec]
+    kern = [num(r, KERNEL) / 1e3 for r in rows]
+    fw = [num(r, FW) / 1e3 for r in rows]
+    gaps = [num(r, GAP) / 1e3 for r in rows]
+    if gaps:
+        gaps[0] = 0.0            # the first op's gap reaches back across the section boundary
+    starts = [num(r, START) for r in rows if num(r, START) > 0]
+    ends = [num(r, END) for r in rows if num(r, END) > 0]
+    span = (max(ends) - min(starts)) / freq if starts and ends else None
+    total = sum(kern) or 1.0
+
+    def group(keyfn) -> List[Dict[str, Any]]:
+        agg: Dict[str, List[float]] = OrderedDict()
+        for (r, m, l), k, g in zip(sec, kern, gaps):
+            key = keyfn(r, m, l)
+            a = agg.setdefault(key, [0, 0.0, 0.0, 0.0])
+            a[0] += 1
+            a[1] += k
+            a[2] += g
+            if r.get(name_key, "") in COMPUTE:
+                a[3] += k
+        return [{"key": key, "ops": int(c), "kernel_us": round(t, 1), "share": round(t / total, 4),
+                 "gap_us": round(gs, 1), "compute_us": round(cu, 1)}
+                for key, (c, t, gs, cu) in sorted(agg.items(), key=lambda kv: -kv[1][1])]
+
+    fid: Dict[str, int] = defaultdict(int)
+    cores = []
+    for r in rows:
+        f = (r.get("MATH FIDELITY") or "").strip()
+        if f:
+            fid[f] += 1
+        cores.append(num(r, "CORE COUNT"))
+    order = sorted(range(len(rows)), key=lambda i: -kern[i])[:top]
+    top_ops = []
+    for i in order:
+        r, m, l = sec[i]
+        shape = "x".join(logical(r.get(f"INPUT_0_{d}_PAD[LOGICAL]") or r.get(f"INPUT_0_{d}") or "") for d in "WZYX")
+        oshape = "x".join(logical(r.get(f"OUTPUT_0_{d}_PAD[LOGICAL]") or r.get(f"OUTPUT_0_{d}") or "") for d in "WZYX")
+        top_ops.append({"index": i, "op": r.get(name_key), "module": m, "layer": l, "kernel_us": round(kern[i], 1),
+                        "fw_us": round(fw[i], 1), "cores": int(num(r, "CORE COUNT")),
+                        "fidelity": (r.get("MATH FIDELITY") or "").strip(), "input0": shape, "output0": oshape,
+                        "in0_dtype": (r.get("INPUT_0_DATATYPE") or "").strip(),
+                        "in0_mem": (r.get("INPUT_0_MEMORY") or "").strip(),
+                        "out0_mem": (r.get("OUTPUT_0_MEMORY") or "").strip()})
+    gap_order = sorted(range(1, len(rows)), key=lambda i: -gaps[i])[:top]
+    top_gaps = [{"index": i, "op": rows[i].get(name_key), "after": rows[i - 1].get(name_key),
+                 "gap_us": round(gaps[i], 1), "module": sec[i][1], "layer": sec[i][2]} for i in gap_order]
+    nz_gaps = [g for g in gaps[1:] if g > 0]
+    ops_list = [{"index": i, "op": r.get(name_key), "module": m, "layer": l, "kernel_us": round(k, 2),
+                 "gap_us": round(g, 2), "cores": int(num(r, "CORE COUNT")),
+                 "fidelity": (r.get("MATH FIDELITY") or "").strip(),
+                 "input0": "x".join(logical(r.get(f"INPUT_0_{d}_PAD[LOGICAL]") or r.get(f"INPUT_0_{d}") or "")
+                                    for d in "WZYX"),
+                 "in0_mem": (r.get("INPUT_0_MEMORY") or "").strip(),
+                 "out0_mem": (r.get("OUTPUT_0_MEMORY") or "").strip()}
+                for i, ((r, m, l), k, g) in enumerate(zip(sec, kern, gaps))]
+    return {
+        "ops": len(rows), "kernel_sum_us": round(sum(kern), 1), "fw_sum_us": round(sum(fw), 1),
+        "op2op_sum_us": round(sum(gaps), 1),
+        "op2op_median_us": round(statistics.median(nz_gaps), 2) if nz_gaps else None,
+        "span_us": round(span, 1) if span else None, "freq_mhz": freq,
+        "cores_median": statistics.median(cores) if cores else None,
+        "ops_below_120_cores": sum(1 for c in cores if 0 < c < 120),
+        "compute_us": round(sum(k for r, k in zip(rows, kern) if r.get(name_key, "") in COMPUTE), 1),
+        "fidelity": dict(fid),
+        "by_op": group(lambda r, m, l: r.get(name_key, "?")),
+        "by_category": group(lambda r, m, l: category(r.get(name_key, "?"))),
+        "by_module": group(lambda r, m, l: m or "?"),
+        "by_layer": group(lambda r, m, l: l or f"[{m or '?'}]"),
+        "top_ops": top_ops, "top_gaps": top_gaps, "ops_list": ops_list,
+    }
+
+
+def md_table(rows: List[Dict[str, Any]], n: int, label: str) -> str:
+    out = [f"| {label} | ops | kernel us | share | compute us | op-to-op us |", "|---|---:|---:|---:|---:|---:|"]
+    for r in rows[:n]:
+        out.append(f"| {r['key']} | {r['ops']} | {r['kernel_us']:.1f} | {100 * r['share']:.1f} % | "
+                   f"{r['compute_us']:.1f} | {r['gap_us']:.1f} |")
+    return "\n".join(out)
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("csv")
+    ap.add_argument("out_json")
+    ap.add_argument("--md")
+    ap.add_argument("--freq-mhz", type=float, default=None)
+    a = ap.parse_args()
+    path = Path(a.csv)
+    with open(path, newline="") as f:
+        rows = list(csv.DictReader(f))
+    name_key = "OP CODE" if rows and "OP CODE" in rows[0] else "OP NAME"
+    freq = a.freq_mhz or chip_freq_mhz(path) or 1350.0
+    secs = sections(rows, name_key)
+    eager, trace_all = secs["eager"], secs["trace"]
+    trace = last_replay(trace_all)
+    # the replay inherits the module / layer of the eager op at the same position (identical op sequence)
+    same = len(eager) == len(trace) and all(e[0].get(name_key) == t[0].get(name_key) for e, t in zip(eager, trace))
+    if same:
+        trace = [(r, e[1], e[2]) for (r, _, _), e in zip(trace, eager)]
+    res = {"csv": str(path), "columns": list(rows[0].keys()) if rows else [], "freq_mhz": freq,
+           "trace_matches_eager_op_sequence": same, "trace_section_ops": len(trace_all),
+           "trace_ids": sorted({(r.get("METAL TRACE ID") or "").strip() for r, _, _ in trace_all}),
+           "replay_sessions": sorted({(r.get("METAL TRACE REPLAY SESSION ID") or "").strip() for r, _, _ in trace_all}),
+           "eager": summarize(eager, name_key, freq), "trace": summarize(trace, name_key, freq)}
+    Path(a.out_json).write_text(json.dumps(res, indent=1) + "\n")
+    t = res["trace"]
+    lines = [f"csv: {path}", f"freq {freq} MHz; trace op sequence == eager: {same}; trace ids {res['trace_ids']}; "
+             f"replay sessions {res['replay_sessions']} ({len(trace_all)} ops in the trace section)"]
+    for s in ("eager", "trace"):
+        x = res[s]
+        lines.append(f"{s}: ops {x['ops']}  kernel {x['kernel_sum_us']:.1f} us (conv/matmul {x['compute_us']:.1f})  "
+                     f"fw {x['fw_sum_us']:.1f} us  op2op {x['op2op_sum_us']:.1f} us (median {x['op2op_median_us']})  "
+                     f"span {x['span_us']} us  cores median {x['cores_median']}  <120 cores: "
+                     f"{x['ops_below_120_cores']}  fidelity {x['fidelity']}")
+    lines += ["", md_table(t["by_category"], 20, "category (trace)"), "", md_table(t["by_module"], 20, "module (trace)"),
+              "", md_table(t["by_op"], 25, "op code (trace)"), "", md_table(t["by_layer"], 40, "layer (trace)"), "",
+              "| # | op | module | layer | kernel us | cores | fidelity | input0 | output0 | in mem | out mem |",
+              "|---:|---|---|---|---:|---:|---|---|---|---|---|"]
+    for k, o in enumerate(t["top_ops"], 1):
+        lines.append(f"| {k} | {o['op']} | {o['module']} | {o['layer']} | {o['kernel_us']:.1f} | {o['cores']} | "
+                     f"{o['fidelity']} | {o['input0']} | {o['output0']} | {o['in0_mem']} | {o['out0_mem']} |")
+    lines += ["", "largest op-to-op gaps (trace): " + json.dumps(t["top_gaps"])]
+    text = "\n".join(lines)
+    print(text)
+    if a.md:
+        Path(a.md).write_text(text + "\n")
+
+
+if __name__ == "__main__":
+    main()

code/scripts/visualizer_report.py

@@ -0,0 +1,116 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: Apache-2.0
+"""ttnn-visualizer memory + graph report of ONE eager forward of the served graph (default precision policy and
+knobs, ETH dispatch, the device grid). Never use it for timing: graph capture and detailed buffer tracing
+synchronise after every op.
+
+    ROOT=/home/ubuntu/experiments/tt-models; cd $ROOT/bundles/frnet-p150
+    $ROOT/bin/devrun -t 2400 -- env PYTHONPATH=$PWD/code:$PYTHONPATH HF_HUB_OFFLINE=1 \\
+        python code/scripts/visualizer_report.py $ROOT/generated/ttnn_visualizer/frnet_baseline [--variant ot128]
+    ttnn-visualizer --profiler-path $ROOT/generated/ttnn_visualizer/frnet_baseline \\
+        --performance-path $ROOT/generated/profiler/frnet_baseline/reports/<ts>
+
+What the pytest fixture of ttnn does (ttnn/graph_report.py ``_run_pytest_graph_report_fixture``), in a script:
+warm-up forward (JIT, weight upload), then the second forward inside ``ttnn.graph.full_graph_capture`` (Python I/O,
+stack traces, detailed buffer tracing), ``ttnn.graph_report.import_report`` -> ``db.sqlite``, and ``config.json``
+(the settings the capture ran with). The intermediate graph JSON is deleted after the import. The inputs are the
+device plan of ``--input`` (default: the shipped synthetic sample of the variant with its calibration preset).
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import time
+from pathlib import Path
+
+CODE = Path(__file__).resolve().parents[1]
+SAMPLES = CODE / "tt_frnet" / "samples"           # the shipped synthetic_<variant>.npz
+
+
+def main() -> None:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    ap.add_argument("out_dir")
+    ap.add_argument("--variant", default="ot128")
+    ap.add_argument("--input", default=None, help="default: the shipped synthetic sample of the variant")
+    ap.add_argument("--calib", default=None, help="preset:<name> | <calibration .json> | none (default: the preset "
+                                                    "named like the input, else crop disabled)")
+    ap.add_argument("--json-dir", default=None, help="where the intermediate graph JSON goes (default: out_dir; "
+                                                      "it can be several GB, e.g. /dev/shm/<name>); deleted after the import")
+    a = ap.parse_args()
+    out_dir = Path(a.out_dir).resolve()
+    out_dir.mkdir(parents=True, exist_ok=True)
+    import ttnn
+
+    from tt_frnet.api import FRNet
+    from tt_frnet.device import close_device, describe_device, open_device
+    from tt_frnet.host.frame import FRNetPreprocessor
+    from tt_frnet.reference.config import FRNetConfig
+    from tt_frnet.reference.weights import load_weights
+    from tt_frnet.tt.layers import free
+    from tt_frnet.tt.model import TtFRNet, default_policy, frame_arrays
+    from tt_frnet.ttaw.tensors import to_host_tensor
+
+    src = Path(a.input or SAMPLES / f"synthetic_{a.variant}.npz")
+    if a.calib == "none":
+        calib = {"ego_crop_box": None}
+    elif a.calib and a.calib.startswith("preset:"):
+        calib = {"preset": a.calib[len("preset:"):]}
+    elif a.calib:
+        calib = json.loads(Path(a.calib).read_text())
+    else:
+        stem = src.name.split(".")[0]
+        calib = {"preset": stem} if (CODE / "tt_frnet" / "calib" / f"{stem}.json").is_file() else {"ego_crop_box": None}
+    weights_path = FRNet.resolve_weights()
+    config = FRNetConfig.from_model_dir(weights_path, a.variant)
+    weights = load_weights(weights_path, a.variant)
+    pre = FRNetPreprocessor(config, pre_norm=(weights.pre_norm_scale, weights.pre_norm_shift))
+    prepared = pre(str(src), calib)
+    if prepared.plan is None:
+        raise SystemExit(f"the frame is skipped: {prepared.skipped}")
+    dev = open_device(allow_fallback=False)
+    try:
+        print("device:", json.dumps(describe_device(dev, 0)), flush=True)
+        graph = TtFRNet(dev, weights, config, policy=default_policy())
+        arrays = frame_arrays(prepared.plan, graph.m_cap)
+        inp = {}
+        for key, (shape, dtype) in graph.input_specs().items():
+            tdt = ttnn.float32 if dtype == "float32" else ttnn.uint32
+            t = ttnn.allocate_tensor_on_device(ttnn.Shape(list(shape)), tdt, ttnn.ROW_MAJOR_LAYOUT, dev,
+                                               ttnn.DRAM_MEMORY_CONFIG)
+            ttnn.copy_host_to_device_tensor(to_host_tensor(arrays[key].reshape(shape), dtype, ttnn.ROW_MAJOR_LAYOUT), t)
+            inp[key] = t
+
+        def forward():
+            out = graph.forward(inp)
+            ttnn.synchronize_device(dev)
+            free(out)
+
+        t0 = time.perf_counter()
+        forward()                                       # warm-up: JIT + weight upload (host writes)
+        print(f"warm-up forward {time.perf_counter() - t0:.1f} s", flush=True)
+        json_dir = Path(a.json_dir).resolve() if a.json_dir else out_dir
+        json_dir.mkdir(parents=True, exist_ok=True)
+        json_path = json_dir / "graph_capture.json"
+        t0 = time.perf_counter()
+        with ttnn.graph.full_graph_capture(str(json_path)):
+            forward()
+        print(f"captured forward {time.perf_counter() - t0:.1f} s -> {json_path} "
+              f"({json_path.stat().st_size / 1e6:.1f} MB)", flush=True)
+        free(*inp.values())
+        graph.release()
+    finally:
+        close_device(dev)
+    t0 = time.perf_counter()
+    db = ttnn.graph_report.import_report(json_path, out_dir)
+    ttnn.save_config_to_json_file(out_dir / "config.json")
+    cfg = json.loads((out_dir / "config.json").read_text())     # record what the capture ran with
+    cfg.update({"enable_fast_runtime_mode": False, "enable_graph_report": True, "enable_graph_python_stack_traces": True,
+                "enable_detailed_buffer_report": True, "root_report_path": str(out_dir.parent),
+                "report_name": out_dir.name})
+    (out_dir / "config.json").write_text(json.dumps(cfg, indent=4) + "\n")
+    print(f"imported in {time.perf_counter() - t0:.1f} s -> {db} ({Path(db).stat().st_size / 1e6:.1f} MB)", flush=True)
+    json_path.unlink(missing_ok=True)
+
+
+if __name__ == "__main__":
+    main()

code/tt_frnet/__init__.py

@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: Apache-2.0
+"""FRNet (Autoware lidar_frnet) on a Tenstorrent Blackhole p150 (tt-nn), packaged as changh95/frnet-p150.
+
+Python API (see PYTHON.md)::
+
+    from tt_frnet import FRNet
+
+    with FRNet.from_pretrained(device_id=0) as model:
+        out = model("code/tt_frnet/samples/synthetic_ot128.npz", calibration={"preset": "synthetic_ot128"})
+
+Importing the package has no side effects (no device, no ttnn / torch import, no network); the names below load
+on first use. The HTTP server is ``tt_frnet.server.app:app`` (SERVING.md). ``tt_frnet.ttaw`` is the
+vendored shared package of the Autoware ports (``ttaw/VENDORED.json`` records its version and file hashes; never
+edit it here).
+"""
+
+__version__ = "0.1.0"
+__all__ = ["FRNet", "Output", "open_device", "load_points", "load_image", "__version__"]
+
+_LAZY = {
+    "FRNet": (".api", "FRNet"),
+    "Output": (".api", "Output"),
+    "open_device": (".device", "open_device"),
+    "load_points": (".io", "load_points"),
+    "load_image": (".io", "load_image"),
+}
+
+
+def __getattr__(name):
+    if name in _LAZY:
+        import importlib
+
+        module, attr = _LAZY[name]
+        return getattr(importlib.import_module(module, __name__), attr)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+    return sorted(list(globals()) + __all__)

code/tt_frnet/api.py

@@ -0,0 +1,189 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Python API: FRNet (Autoware lidar_frnet) on one Tenstorrent Blackhole p150.
+
+    from tt_frnet import FRNet
+
+    with FRNet.from_pretrained(device_id=0) as model:  # weights -> HF cache, device open, traces captured
+        out = model("code/tt_frnet/samples/synthetic_ot128.npz", calibration={"preset": "synthetic_ot128"})
+    print(out.to_dict())  # the same JSON as POST /predict
+
+The contract shared by every bundle of the Autoware collection is the vendored ``ttaw.api_base.ModelBase``
+(BUNDLE_CONVENTIONS.md section 8): ``from_pretrained`` resolves the pinned weights before it claims the chip, opens it
+(ETH dispatch, 12x10), builds the graph and captures every trace variant in ``warmup_variants``, so the first call is
+as fast as the later ones; calls are serialised by a lock (one chip, batch 1) and fill ``timing_ms``; ``close()`` is
+idempotent, also runs at interpreter exit, and closes the chip only if the model opened it. The HTTP server
+(``tt_frnet.server.app``) calls this class, so ``/predict`` and ``model(...)`` agree bit for bit.
+
+This module holds the identity of the port and its hooks: host pre- and post-processing ported from Autoware live in
+``tt_frnet.host`` (shared with the CPU reference ``tt_frnet.reference``, so device and reference see identical
+network inputs), the ttnn graph in ``tt_frnet.tt`` (run by a ``ttaw.trace.TraceRunner``; milestones M1-M3 of
+PORT_LOG.md). Importing it has no side effects: ttnn / torch are imported inside the hooks.
+
+Inputs (``host/frame.py`` has the full contract): ``points`` -- ONE sweep in the **sensor frame**, fields x, y, z,
+intensity (raw 0-255); ``calibration`` -- ``{"T_base_link_from_sensor": ...}`` (or ``{"preset": name}`` of
+``calib/``) for the node's ego crop box, ``ego_crop_box`` to override or disable the box (``None``).
+``variant``: ``ot128`` (Autoware's default sensor model) or ``qt128``, a load-time choice (weights + grid).
+
+Output: :class:`FRNetSegmentation` (a ``ttaw.outputs.Segmentation3D``) in the node's output order: the kept raw
+points (input order) then Autoware's interpolated points; ``label_ids`` = class_id, ``scores`` = probability,
+``points``, ``filter_keep`` (the filtered cloud), ``input_index``. A frame outside the node's profiles gives an empty
+output whose ``meta["skipped"]`` is the node's message (the node publishes nothing).
+"""
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+from . import io as tio
+from .device import DEVICE_DEFAULTS
+from .host.postprocess import RUNTIME_PARAMS, FRNetSegmentation
+from .reference.config import DEFAULT_VARIANT, VARIANTS
+from .ttaw.api_base import ModelBase
+
+__all__ = ["FRNet", "Output", "FRNetSegmentation", "LABELS"]
+
+# The result class of this model (ttaw.outputs: Detections3D, Detections2D, Segmentation3D, Mask2D, Trajectory).
+Output = FRNetSegmentation
+
+# ml_package_frnet_{ot128,qt128}.param.yaml class_names (identical in both variants; _build_host checks the yaml).
+# Indices 22 / 23 read "debris" / "stroller" here, but the weights were trained with stroller = 22, debris = 23
+# (AWML frnet_1xb8_t4dataset-ot128-seg.py:60-61; S:frnet:314-317): Autoware's names are kept for parity.
+LABELS = ("drivable_surface", "other_flat_surface", "sidewalk", "manmade", "vegetation", "car", "bus",
+          "emergency_vehicle", "train", "truck", "tractor_unit", "semi_trailer", "construction_vehicle", "forklift",
+          "kart", "motorcycle", "bicycle", "pedestrian", "personal_mobility", "animal", "pushable_pullable",
+          "traffic_cone", "debris", "stroller", "other_stuff", "noise+ghost_point", "unknown")
+
+
+def _given(value: Any) -> bool:
+    """A real input (``None`` and empty containers are 'not given'); never the truth value of an array."""
+    if value is None:
+        return False
+    if isinstance(value, (list, tuple, dict, str)):
+        return len(value) > 0
+    return True
+
+
+class FRNet(ModelBase):
+    """FRNet (Autoware lidar_frnet) on one Blackhole p150. Create it with :meth:`from_pretrained`."""
+
+    MODEL_NAME = "frnet-p150"
+    ENV_PREFIX = "FRNET"  # prefix of the environment knobs (SERVING.md section 3.4)
+    DEFAULT_REPO = "AutowareFoundation/lidar_frnet"
+    DEFAULT_TAG = "v2.0"  # the Autoware ansible artifacts pin
+    # the commit DEFAULT_TAG points to (pinned: tags can move)
+    DEFAULT_REVISION = "a53a1c11b0b28d31fd50a03abef5db0e830a5b3c"
+    ALLOW_PATTERNS = ["frnet_*.onnx", "ml_package_frnet_*.param.yaml", "deploy_metadata.yaml"]
+    VARIANTS = list(VARIANTS)  # load-time: selects the ONNX, the frustum grid and the trace shapes
+    DEFAULT_VARIANT = DEFAULT_VARIANT
+    INPUT_KIND = "lidar"  # lidar | camera | multicam | lidar+multicam | planner
+    CAMERA_ORDER = ()  # multicam models: the fixed camera order of the network
+    POINT_FIELDS = tio.DEFAULT_POINT_FIELDS  # columns the network consumes, in order (x, y, z, intensity)
+    LABELS = list(LABELS)
+    # Per-request knobs (host post-processing only; defaults are Autoware's): host/postprocess.py
+    RUNTIME_PARAMS = RUNTIME_PARAMS
+    EXTRA_INPUTS = ()
+    DEVICE_DEFAULTS = DEVICE_DEFAULTS  # validated open parameters (device.py)
+
+    # ---- port-specific hooks (called by ModelBase; keep host work out of _forward) ---------------------------
+    def _build(self) -> None:
+        """Host side (config, weights, pre- / post-processors), then the ttnn graph of ``tt_frnet.tt``
+        (``TtFRNet``: point encoder, K1 segmented max, gathers, convs, decoder, head) as the ``"default"`` variant of
+        a ``ttaw.trace.TraceRunner`` with the per-frame tables as persistent inputs (no capture here; ``_warm_one``
+        captures). Warm-up runs an empty frame (every index valid)."""
+        from .tt.model import FrameStaging, TtFRNet, build_runner, default_policy
+
+        self._build_host()
+        self.graph = TtFRNet(self.device, self.frnet_weights, self.config, policy=default_policy())
+        self.runner = build_runner(self.graph, variants=("default",), name=f"{self.MODEL_NAME}/{self.variant}")
+        self.staging = FrameStaging(self.graph)   # persistent host buffers of the inputs (written in place)
+
+    def _build_host(self) -> None:
+        """The host half of ``_build`` (no device): usable alone by host tests and tools."""
+        from .host.frame import FRNetPreprocessor
+        from .host.postprocess import FRNetPostprocessor
+        from .reference.config import FRNetConfig
+        from .reference.weights import check_variant_grid, load_weights
+
+        self.config = FRNetConfig.from_model_dir(self.weights_path, self.variant)
+        if tuple(self.config.class_names) != tuple(self.LABELS):
+            raise ValueError(f"ml_package class_names differ from FRNet.LABELS: {self.config.class_names}")
+        self.frnet_weights = load_weights(self.weights_path, self.variant)
+        check_variant_grid(self.frnet_weights, self.config.frustum_h, self.config.frustum_w)
+        self.pre = FRNetPreprocessor(self.config, pre_norm=(self.frnet_weights.pre_norm_scale,
+                                                            self.frnet_weights.pre_norm_shift))
+        self.post = FRNetPostprocessor(self.config)
+
+    def _warm_one(self, variant: Dict[str, Any]) -> None:
+        """Warm up and capture the traces of one entry of ``default_warmup_variants()``. ``TraceRunner.capture``
+        runs every pending variant eagerly (kernel JIT, program cache) before any capture, then captures with
+        program-cache misses forbidden; it is idempotent."""
+        self.runner.capture()
+
+    def _prepare(self, points: Any = None, *, sweeps: Optional[Any] = None, images: Any = None,
+                 calibration: Any = None, stream: Any = None, inputs: Any = None, **_: Any):
+        """Host pre-processing (``host/frame.py``): decoding, the ego crop, float32 projection, fp16 interpolation,
+        unique cells, the node's profile checks and the device plan. Client mistakes raise ``tio.InputError``
+        (HTTP 400); an out-of-profile frame is ``prepared.skipped`` (no device run, empty output)."""
+        if points is None:
+            raise tio.InputError("this model needs 'points'")
+        extra = [name for name, value in (("sweeps", sweeps), ("images", images), ("inputs", inputs),
+                                          ("stream", stream)) if _given(value)]
+        if extra:
+            raise tio.InputError(f"FRNet takes one sensor-frame sweep ('points') and its 'calibration' only; got "
+                                 f"{extra}")
+        return self.pre(points, calibration)
+
+    def _forward(self, prepared: Any) -> Any:
+        """Upload the frame plan (features, CSR offsets, gather tables) into the persistent device inputs, replay
+        the trace, read ``pred_probs`` back (one D2H) and return it as ``(N, 27)`` float32 in the network-input order
+        (``plan.unsort``). A skipped frame runs nothing. Runs under the model lock."""
+        if prepared.skipped is not None:
+            return None
+        return self.device_probs(prepared.plan)
+
+    def device_probs(self, plan: Any) -> Any:
+        """``pred_probs`` ``(N, 27)`` float32 (network-input order) of one ``host.preprocess.FramePlan`` (the device
+        half of ``_forward``; tests and tools feed stored network inputs through it). Call it under the model
+        lock (``model(...)`` does)."""
+        import numpy as np
+
+        raw = self.runner("default", inputs=self.staging.write(plan))
+        probs = np.asarray(raw, np.float32).reshape(-1, raw.shape[-1])
+        return plan.unsort(probs[:, : self.config.num_classes])
+
+    def _postprocess(self, raw: Any, prepared: Any, params: Dict[str, Any]) -> Output:
+        """Host post-processing (``host/postprocess.py``, ``fill_cloud_kernel``): first-max argmax, probability,
+        the class-probability filter -> ``FRNetSegmentation``; ``raw`` = ``pred_probs`` or None (skipped)."""
+        meta = {"variant": self.variant, **prepared.stats()}
+        self.post.filter_indices(params["filter_classes"])             # an unknown class name is a 400 either way
+        if raw is None:
+            return self.post.empty(model=self.MODEL_NAME, frame_id=prepared.frame_id, meta=meta)
+        inp = prepared.inputs
+        return self.post(raw, points=inp.points, num_points_raw=inp.num_points_raw,
+                         input_index=prepared.input_index, params=params, model=self.MODEL_NAME,
+                         frame_id=prepared.frame_id, meta=meta)
+
+    def _release(self) -> None:
+        """Release the traces, persistent tensors and device weights; also called when ``from_pretrained`` fails
+        half-way (tolerates a partial build)."""
+        runner = getattr(self, "runner", None)
+        if runner is not None:
+            runner.release()
+            self.runner = None
+        graph = getattr(self, "graph", None)
+        if graph is not None:
+            graph.release()
+            self.graph = None
+
+    def extra_info(self) -> Dict[str, Any]:
+        """Additions to ``model.info`` and ``/info``: the node parameters and the runner's traces."""
+        out: Dict[str, Any] = {}
+        config = getattr(self, "config", None)
+        if config is not None:
+            out["frnet"] = config.describe()
+        runner = getattr(self, "runner", None)
+        if runner is not None:
+            out["trace"] = runner.describe()
+        graph = getattr(self, "graph", None)
+        if graph is not None:
+            out["graph"] = graph.describe()
+        return out

code/tt_frnet/calib/README.md

@@ -0,0 +1,18 @@
+# Calibration presets
+
+FRNet takes one LiDAR sweep in the **sensor frame**; the only calibration it uses is the static
+`sensor -> base_link` transform of the node's ego crop box (Autoware looks it up once from TF,
+`lidar_frnet_node.cpp:154-182`). A preset file holds `{"name", "description", "source", "T_base_link_from_sensor":
+4x4 row-major}` (any `ttaw.io.parse_transform` spelling works, also `{translation, rotation_wxyz}` or
+`{x, y, z, roll, pitch, yaw}`), optionally `"ego_crop_box": [min_x, min_y, min_z, max_x, max_y, max_z]` to replace
+Autoware's default box (`frnet.param.yaml`: -1.53, -1.15, -0.1, 6.1, 1.15, 3.1 in base_link) or `null` to disable it.
+The server lists the presets in `/info` (`calibration_presets`); a request may say `"calibration": {"preset":
+"<name>"}` (the Python API too) instead of sending the matrix.
+
+| preset | content | source |
+|---|---|---|
+| `synthetic_ot128.json`, `synthetic_qt128.json` | the sensor -> base_link transform of the shipped synthetic samples: a pure 2 m lift (base_link on the road under the sensor, x forward) | `code/scripts/make_sample.py` |
+
+The PandaSet 019 frame 40 preset (`pandaset_019_40.json`, the fitted Pandar64 mount) belongs to the staged PandaSet
+sample and moves here with it once PandaSet-derived samples are approved (`staging_samples_pandaset/` of the bundle
+checkout).

code/tt_frnet/calib/synthetic_ot128.json

@@ -0,0 +1,31 @@
+{
+ "name": "synthetic_ot128",
+ "description": "synthetic OT128-like scan (code/scripts/make_sample.py): the sensor 2 m above a flat road, base_link on the road under it (a pure lift)",
+ "source": "code/scripts/make_sample.py (generated, no dataset; Apache-2.0)",
+ "T_base_link_from_sensor": [
+  [
+   1.0,
+   0.0,
+   0.0,
+   0.0
+  ],
+  [
+   0.0,
+   1.0,
+   0.0,
+   0.0
+  ],
+  [
+   0.0,
+   0.0,
+   1.0,
+   2.0
+  ],
+  [
+   0.0,
+   0.0,
+   0.0,
+   1.0
+  ]
+ ]
+}

code/tt_frnet/calib/synthetic_qt128.json

@@ -0,0 +1,31 @@
+{
+ "name": "synthetic_qt128",
+ "description": "synthetic QT128-like scan (code/scripts/make_sample.py): the sensor 2 m above a flat road, base_link on the road under it (a pure lift)",
+ "source": "code/scripts/make_sample.py (generated, no dataset; Apache-2.0)",
+ "T_base_link_from_sensor": [
+  [
+   1.0,
+   0.0,
+   0.0,
+   0.0
+  ],
+  [
+   0.0,
+   1.0,
+   0.0,
+   0.0
+  ],
+  [
+   0.0,
+   0.0,
+   1.0,
+   2.0
+  ],
+  [
+   0.0,
+   0.0,
+   0.0,
+   1.0
+  ]
+ ]
+}

code/tt_frnet/device.py

@@ -0,0 +1,59 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Opening the Blackhole p150 the way every published number of frnet-p150 was measured.
+
+The implementation is the vendored ``ttaw.device`` (C01): dispatch on the idle ETH cores
+(``ttnn.DispatchCoreConfig(ttnn.DispatchCoreType.ETH)``), which gives a 12x10 = 120-core compute grid on a p150
+(11x10 with WORKER dispatch, the A/B switch). ETH dispatch needs ``patches/tt-metal-eth-dispatch.patch`` on tt-metal
+44d6650 (the container image is built from a patched tree); if the ETH open fails, a ``RuntimeWarning`` is issued and
+WORKER dispatch is used, unless ``allow_fallback=False``. Never hard-code the grid: use :func:`compute_grid` or
+``device.compute_with_storage_grid_size()``.
+
+This module binds it to the validated open parameters of this port, :data:`DEVICE_DEFAULTS` (the model class uses
+the same dict), which the ``FRNET_DISPATCH``, ``_NUM_CQS``, ``_L1_SMALL``, ``_TRACE_REGION`` and
+``_WORKER_L1_SIZE`` variables and ``TT_DEVICE_ID`` override per process (SERVING.md section 3.4). Importing it has
+no side effects (``ttnn`` is imported when a device is opened).
+"""
+from __future__ import annotations
+
+import contextlib
+import dataclasses
+from typing import Any, Dict, Iterator, Optional
+
+from .ttaw.device import DeviceConfig, close_device, compute_grid, core_grid, describe_device, full_core_range_set
+
+__all__ = ["ENV_PREFIX", "DEVICE_DEFAULTS", "DeviceConfig", "device_config", "open_device", "device_session",
+           "close_device", "describe_device", "compute_grid", "core_grid", "full_core_range_set"]
+
+ENV_PREFIX = "FRNET"
+
+# Validated device-open parameters of this port (fill per model; the card's numbers are measured with them).
+DEVICE_DEFAULTS: Dict[str, Any] = {
+    "num_command_queues": 1,  # 1, or 2 when the input upload (CQ1) overlaps the trace (CQ0)
+    "l1_small_size": 65536,  # L1_SMALL bytes per core (conv config tensors; P5 / P14 conv values, measured at M2)
+    "trace_region_size": 256 << 20,  # DRAM bytes for the traces of all warm-up variants (measured at M2)
+}
+
+
+def device_config(**overrides: Any) -> DeviceConfig:
+    """:data:`DEVICE_DEFAULTS` < the ``FRNET_*`` / ``TT_DEVICE_ID`` environment < explicit non-None ``overrides``
+    (``device_id``, ``dispatch``, ``num_command_queues``, ``l1_small_size``, ``trace_region_size``,
+    ``worker_l1_size``, ``allow_fallback``): the same resolution as ``FRNet.from_pretrained``."""
+    config = DeviceConfig.from_env(ENV_PREFIX, **DEVICE_DEFAULTS)
+    return dataclasses.replace(config, **{k: v for k, v in overrides.items() if v is not None})
+
+
+def open_device(device_id: Optional[int] = None, *, dispatch: Optional[str] = None, **overrides: Any):
+    """Open one chip like the published numbers: ETH dispatch (``dispatch="worker"`` is the A/B opt-in) and this
+    port's sizes. Close it with :func:`close_device`, or use :func:`device_session`."""
+    return device_config(device_id=device_id, dispatch=dispatch, **overrides).open()
+
+
+@contextlib.contextmanager
+def device_session(device_id: Optional[int] = None, **overrides: Any) -> Iterator[Any]:
+    """``with device_session() as dev:`` opens with :func:`open_device` and always closes, also when the body
+    raises."""
+    device = open_device(device_id, **overrides)
+    try:
+        yield device
+    finally:
+        close_device(device)

code/tt_frnet/host/__init__.py

@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host pre- and post-processing of FRNet with the exact ``autoware_lidar_frnet`` semantics (numpy only), shared by
+the device path (``api.py``) and the CPU reference (``reference/pipeline.py``), so TT and CPU always see identical
+network inputs:
+
+- ``preprocess.py``   the node's pre-processing (ego crop, float32 projection, fp16 farthest-point interpolation,
+                      counting-sort unique cells, profile checks) -> the four ONNX inputs; and the TT device plan
+                      (points sorted by cell and padded to N_cap, per-cell segments, the fp32 8-d features with
+                      ``pre_norm`` folded, the gather tables that replace every scatter of the graph);
+- ``frame.py``        one request -> a prepared frame (decoding, calibration / crop transform, profile verdict);
+- ``postprocess.py``  ``fill_cloud_kernel``: first-max argmax, probability, palette, class-probability filter, and
+                      the ``FRNetSegmentation`` output.
+
+Every function is tested against the research reference scripts (research/frnet/scripts/frnet_preproc_ref.py and
+the independent verify_preproc_ort_independent.py) and the stored research goldens (``tests/test_host_vs_research.py``).
+"""

code/tt_frnet/host/frame.py

@@ -0,0 +1,138 @@
+# SPDX-License-Identifier: Apache-2.0
+"""One request -> one prepared frame (the host half of ``FRNet._prepare``), numpy only.
+
+Input contract (research/frnet/SPEC.md section 2; the node's ``~/input/pointcloud``):
+
+- ``points``: ONE LiDAR sweep in the **sensor frame** (x forward, y left, z up), fields ``x, y, z, intensity``
+  (intensity raw, as the node casts it: 0-255 for uint8 drivers, floats passed through); any ``ttaw.io.load_points``
+  form. Rows with a non-finite x / y / z are dropped by the decoder (PCL ``is_dense=false`` clouds); the output's
+  ``input_index`` refers to the decoded rows;
+- ``calibration`` (optional mapping, or ``{"preset": name}`` of ``calib/<name>.json``):
+  ``T_base_link_from_sensor`` (any ``ttaw.io.parse_transform`` spelling; alias ``T_ref_from_sensor``) -- the static
+  transform the node looks up once for its ego crop box (``lidar_frnet_node.cpp:154-182``); ``ego_crop_box`` --
+  ``[min_x, min_y, min_z, max_x, max_y, max_z]`` in base_link or a mapping of those keys (default: Autoware's
+  ``frnet.param.yaml`` box), ``null`` / all zeros = crop disabled (the node's own switch).
+
+With the crop enabled and no transform, the node skips every frame (its TF lookup fails); here that is a client error
+(``InputError``, HTTP 400) naming both fixes. A frame outside the node's profiles is not an error: the node logs and
+publishes nothing, so :class:`Prepared` carries ``skipped`` (the node's message), no device run happens and the
+output is empty with that reason.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, Dict, Mapping, Optional, Tuple
+
+import numpy as np
+
+from .. import io as tio
+from ..reference.config import FRNetConfig, parse_crop_box
+from .preprocess import FramePlan, NetworkInputs, autoware_preprocess, build_plan, crop_transform_12, profile_errors
+
+__all__ = ["Prepared", "FRNetPreprocessor", "CALIB_DIR", "TRANSFORM_KEYS"]
+
+CALIB_DIR = Path(__file__).resolve().parents[1] / "calib"
+TRANSFORM_KEYS = ("T_base_link_from_sensor", "T_ref_from_sensor")
+POINT_FIELDS = ("x", "y", "z", "intensity")
+
+
+@dataclass
+class Prepared:
+    """A prepared frame. ``skipped`` set: the node would publish nothing (no device run)."""
+
+    num_input_points: int
+    inputs: Optional[NetworkInputs]
+    plan: Optional[FramePlan]
+    skipped: Optional[str]
+    crop: Dict[str, Any] = field(default_factory=dict)
+    frame_id: str = "sensor"
+
+    @property
+    def input_index(self) -> np.ndarray:
+        """(num_points_raw,) int64: decoded input row of every raw output point."""
+        if self.inputs is None:
+            return np.zeros(0, np.int64)
+        return np.flatnonzero(self.inputs.keep_mask).astype(np.int64)
+
+    def stats(self) -> Dict[str, Any]:
+        out: Dict[str, Any] = {"num_points_input": self.num_input_points, "ego_crop": dict(self.crop)}
+        if self.inputs is not None:
+            out.update(self.inputs.stats())
+        if self.plan is not None:
+            out["plan"] = self.plan.stats()
+        if self.skipped is not None:
+            out["skipped"] = self.skipped
+        return out
+
+
+class FRNetPreprocessor:
+    """``FRNetPreprocessor(config, pre_norm=(scale, shift))(points, calibration)`` -> :class:`Prepared`.
+    ``plan=False`` stops after the four network inputs (the CPU reference needs nothing else)."""
+
+    def __init__(self, config: FRNetConfig, *, pre_norm: Optional[Tuple[np.ndarray, np.ndarray]] = None,
+                 calib_dir: Optional[Path] = CALIB_DIR, plan: bool = True):
+        self.config = config
+        self.pre_norm = pre_norm
+        self.calib_dir = calib_dir
+        self.plan = plan
+
+    # ---- calibration -------------------------------------------------------------------------------------------
+    def crop_setup(self, calibration: Optional[Mapping[str, Any]]) -> Tuple[Optional[np.ndarray], Tuple[float, ...],
+                                                                            Dict[str, Any]]:
+        """``calibration`` -> ``(transform 12 float32 or None, bounds (6,), description)``; client mistakes raise
+        ``InputError``."""
+        if calibration is not None and not isinstance(calibration, Mapping):
+            raise tio.InputError("calibration must be an object")
+        calib = tio.resolve_calibration(calibration, self.calib_dir) if calibration is not None else {}
+        unknown = sorted(set(calib) - set(TRANSFORM_KEYS) - {"ego_crop_box", "name", "description", "source"})
+        if unknown:
+            raise tio.InputError(f"calibration: unknown keys {unknown}; FRNet takes {list(TRANSFORM_KEYS)[:1]} "
+                                 "and ego_crop_box")
+        bounds = tuple(self.config.crop_bounds)
+        if "ego_crop_box" in calib:
+            try:
+                bounds = parse_crop_box(calib["ego_crop_box"])
+            except ValueError as e:
+                raise tio.InputError(f"calibration.{e}") from None
+        given = [k for k in TRANSFORM_KEYS if k in calib and calib[k] is not None]
+        if len(given) > 1:
+            raise tio.InputError(f"calibration: give one of {list(TRANSFORM_KEYS)}, not both")
+        T12 = None
+        if given:
+            T = tio.parse_transform(calib[given[0]], field=f"calibration.{given[0]}")
+            R = T[:3, :3]
+            if not (np.allclose(R @ R.T, np.eye(3), atol=1e-4) and abs(np.linalg.det(R) - 1.0) < 1e-4):
+                raise tio.InputError(f"calibration.{given[0]}: the rotation is not a proper rotation matrix")
+            T12 = crop_transform_12(T)
+        enabled = bool(np.any(np.asarray(bounds, np.float64).astype(np.float32) != 0.0))
+        if enabled and T12 is None:
+            raise tio.InputError(
+                "the ego crop box (Autoware frnet.param.yaml filter.ego_crop_box) needs the static sensor -> base_link "
+                "transform: pass calibration={'T_base_link_from_sensor': <4x4 | {translation, rotation_*} | "
+                "{x, y, z, roll, pitch, yaw}>} (or a calib preset), or disable the crop with "
+                "calibration={'ego_crop_box': None} (the node skips every frame when its TF lookup fails)")
+        desc = {"enabled": enabled, "bounds": [float(b) for b in bounds],
+                "reference_frame": self.config.crop_reference_frame}
+        if T12 is not None:
+            desc["sensor_to_ref_12"] = [float(v) for v in T12]
+        return T12, bounds, desc
+
+    # ---- one frame ---------------------------------------------------------------------------------------------
+    def __call__(self, points: Any, calibration: Optional[Mapping[str, Any]] = None) -> Prepared:
+        if points is None:
+            raise tio.InputError("this model needs 'points'")
+        T12, bounds, crop_desc = self.crop_setup(calibration)
+        cloud = tio.load_points(points)
+        xyzi = cloud.select(POINT_FIELDS)
+        frame_id = cloud.frame_id if cloud.frame_id not in ("", "base_link") else "sensor"
+        n0 = int(len(xyzi))
+        errs = profile_errors(n0, None, self.config)
+        if errs:
+            return Prepared(n0, None, None, errs[0], crop_desc, frame_id)
+        inputs = autoware_preprocess(xyzi, self.config, sensor_to_ref_12=T12, crop_bounds=bounds)
+        errs = profile_errors(n0, inputs, self.config)
+        if errs:
+            return Prepared(n0, inputs, None, errs[0], crop_desc, frame_id)
+        plan = build_plan(inputs, self.config, pre_norm=self.pre_norm) if self.plan else None
+        return Prepared(n0, inputs, plan, None, crop_desc, frame_id)

code/tt_frnet/host/postprocess.py

@@ -0,0 +1,159 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host post-processing of ``autoware_lidar_frnet`` (research/frnet/SPEC.md section 5), numpy only.
+
+``fill_cloud_kernel`` (``PKG/lib/postprocess/postprocess_kernel.cu:200-264`` @ universe ``9ceaccf``), one thread per
+point over all ``N`` points (interpolated ones included):
+
+- ``class_id``: ``best = -1e9f``, ``class_id = num_classes - 1``, then ``if (prob > best)`` over the classes in
+  order -- the **first** maximum wins and a NaN never wins (:func:`autoware_argmax`); ``probability = best``;
+- segmentation cloud: (x, y, z of the network input points, class_id uint8, probability float32);
+- visualization cloud: (x, y, z, ``palette[class_id]``: ``(r << 16) | (g << 8) | b`` as a float32);
+- filtered cloud: a point is dropped iff **any** filter class has ``prob >= threshold`` (float32 compare against
+  ``static_cast<float>(0.05)``); kept raw points are the original records, kept interpolated points are synthesized
+  from x, y, z, intensity with the other fields 0. The CUDA kernel appends them with ``atomicAdd`` (any order);
+  here the filtered cloud is the mask :attr:`FRNetSegmentation.filter_keep` over the output points.
+
+AWML's offline max-probability threshold (``AWML/deploy/postprocessing.py``) is not used by Autoware and not here.
+The output class adds the point arrays the node publishes to ``ttaw.outputs.Segmentation3D``.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, Mapping, Optional, Sequence, Tuple
+
+import numpy as np
+
+from ..reference.config import NODE_PARAMS, FRNetConfig, parse_filter_classes
+from ..ttaw.io import InputError, encode_array
+from ..ttaw.outputs import Segmentation3D
+
+__all__ = ["RUNTIME_PARAMS", "autoware_argmax", "filter_keep_mask", "visualization_rgb", "FRNetSegmentation",
+           "FRNetPostprocessor"]
+
+# Per-request knobs (RT-host), Autoware's defaults (frnet.param.yaml:6-7). ``filter_classes``: comma-separated
+# class names, "" = none; the node allows at most 16.
+RUNTIME_PARAMS: Dict[str, Tuple[Any, Any, Any, Any]] = {
+    "class_probability_threshold": (float, 0.0, 1.0, NODE_PARAMS["class_probability_threshold"]),
+    "filter_classes": (str, None, None, ",".join(NODE_PARAMS["filter_classes"])),
+}
+
+ARGMAX_INIT = np.float32(-1e9)
+
+
+def autoware_argmax(pred_probs: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
+    """``fill_cloud_kernel`` L218-228 -> ``(class_id uint8 (N,), probability float32 (N,))``."""
+    p = np.asarray(pred_probs, np.float32)
+    if p.ndim != 2:
+        raise ValueError(f"pred_probs must be (N, C), got {p.shape}")
+    n, c = p.shape
+    if n == 0:
+        return np.zeros(0, np.uint8), np.zeros(0, np.float32)
+    q = np.where(np.isnan(p), np.float32(-np.inf), p)
+    idx = np.argmax(q, axis=1)                           # first maximum (NaN treated as never greater)
+    best = q[np.arange(n), idx]
+    none = ~(best > ARGMAX_INIT)                         # no class beat the -1e9 start: class C - 1, prob -1e9
+    if none.any():
+        idx = idx.copy()
+        best = best.copy()
+        idx[none] = c - 1
+        best[none] = ARGMAX_INIT
+    return idx.astype(np.uint8), best.astype(np.float32)
+
+
+def filter_keep_mask(pred_probs: np.ndarray, class_indices: Sequence[int], threshold: float) -> np.ndarray:
+    """``fill_cloud_kernel`` L245-254: True = the point stays in the filtered cloud (no filter class reaches the
+    float32 threshold)."""
+    p = np.asarray(pred_probs, np.float32)
+    thr = np.float32(threshold)
+    hit = np.zeros(p.shape[0], bool)
+    for c in class_indices:
+        hit |= p[:, int(c)] >= thr
+    return ~hit
+
+
+def visualization_rgb(class_id: np.ndarray, config: FRNetConfig) -> np.ndarray:
+    """The visualization cloud's packed ``rgb`` float32 per point (``const_palette[class_id]``)."""
+    return config.palette_packed[np.asarray(class_id, np.int64)]
+
+
+@dataclass
+class FRNetSegmentation(Segmentation3D):
+    """Per-point classes of one frame in the node's output order: the input points that survive the ego crop box
+    (input order, after non-finite rows are dropped), then the interpolated points (``num_points_raw`` raw rows
+    first). ``label_ids`` = Autoware ``class_id``, ``scores`` = its ``probability``; ``points`` (N, 4) the network
+    input x, y, z, intensity (the segmentation cloud's coordinates); ``filter_keep`` (N,) the filtered cloud's
+    membership; ``input_index`` (num_points_raw,) int32 the input row of every raw output point."""
+
+    points: Optional[np.ndarray] = None
+    filter_keep: Optional[np.ndarray] = None
+    input_index: Optional[np.ndarray] = None
+    num_points_raw: int = 0
+    extras: Dict[str, Any] = field(default_factory=dict)
+
+    def __post_init__(self) -> None:
+        super().__post_init__()
+        n = len(self)
+        if self.points is not None:
+            self.points = np.asarray(self.points, np.float32)
+            if self.points.ndim != 2 or self.points.shape[0] != n:
+                raise ValueError(f"points must be ({n}, C), got {self.points.shape}")
+        if self.filter_keep is not None:
+            self.filter_keep = np.asarray(self.filter_keep, bool).reshape(n)
+        if self.input_index is not None:
+            self.input_index = np.asarray(self.input_index, np.int32).reshape(-1)   # decoded rows < 2**31
+            if self.input_index.shape[0] != self.num_points_raw:
+                raise ValueError("input_index must have num_points_raw entries")
+        if self.num_points_raw > n:
+            raise ValueError("num_points_raw exceeds the number of points")
+
+    def to_dict(self, output_format: str = "json") -> Dict[str, Any]:
+        body = super().to_dict(output_format)
+        body["num_points_raw"] = int(self.num_points_raw)
+        if self.points is not None:
+            body["points"] = encode_array(self.points, key="points")
+        if self.filter_keep is not None:
+            body["filter_keep"] = encode_array(self.filter_keep, key="filter_keep")
+        if self.input_index is not None:
+            body["input_index"] = encode_array(self.input_index, key="input_index")
+        for k, v in self.extras.items():
+            body[k] = encode_array(np.asarray(v), key=k)
+        return body
+
+
+class FRNetPostprocessor:
+    """``pred_probs`` (N, 27) of the network input points -> :class:`FRNetSegmentation`."""
+
+    def __init__(self, config: FRNetConfig):
+        self.config = config
+
+    def filter_indices(self, filter_classes: Any) -> Tuple[int, ...]:
+        """Request ``filter_classes`` (comma-separated names) -> class indices; unknown names are a client error
+        (the node throws at start-up)."""
+        try:
+            return self.config.filter_class_indices(parse_filter_classes(filter_classes))
+        except ValueError as e:
+            raise InputError(f"filter_classes: {e}") from None
+
+    def __call__(self, pred_probs: np.ndarray, *, points: np.ndarray, num_points_raw: int,
+                 input_index: Optional[np.ndarray], params: Mapping[str, Any], model: str = "",
+                 frame_id: str = "sensor", meta: Optional[Dict[str, Any]] = None) -> FRNetSegmentation:
+        p = {k: v[3] for k, v in RUNTIME_PARAMS.items()}
+        p.update(params or {})
+        indices = self.filter_indices(p["filter_classes"])
+        class_id, prob = autoware_argmax(pred_probs)
+        keep = filter_keep_mask(pred_probs, indices, p["class_probability_threshold"])
+        meta = dict(meta or {})
+        meta.update(filter={"classes": [self.config.class_names[i] for i in indices],
+                            "class_probability_threshold": float(p["class_probability_threshold"]),
+                            "kept": int(keep.sum())})
+        return FRNetSegmentation(class_id, class_names=self.config.class_names, scores=prob, model=model,
+                                 frame_id=frame_id, meta=meta, points=points, filter_keep=keep,
+                                 input_index=input_index, num_points_raw=int(num_points_raw))
+
+    def empty(self, *, model: str = "", frame_id: str = "sensor",
+              meta: Optional[Dict[str, Any]] = None) -> FRNetSegmentation:
+        """No output (a frame the node refuses: profile checks), with the reason in ``meta``."""
+        return FRNetSegmentation(np.zeros(0, np.uint8), class_names=self.config.class_names,
+                                 scores=np.zeros(0, np.float32), model=model, frame_id=frame_id,
+                                 meta=dict(meta or {}), points=np.zeros((0, 4), np.float32),
+                                 filter_keep=np.zeros(0, bool), input_index=np.zeros(0, np.int32), num_points_raw=0)

code/tt_frnet/host/preprocess.py

@@ -0,0 +1,367 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Host pre-processing of ``autoware_lidar_frnet`` (research/frnet/SPEC.md section 3; PLAN.md 2.8), numpy only.
+
+Two halves, both deterministic:
+
+1. :func:`autoware_preprocess` -- the node's pre-processing, ``PKG/lib/preprocess/preprocess_kernel.cu`` +
+   ``PKG/lib/lidar_frnet.cpp:196-262`` @ universe ``9ceaccf``, producing the four network inputs exactly as
+   the deployed ONNX takes them (unpadded: the graph sizes its voxel tensors as ``max(inverse_map) + 1``,
+   S:frnet:255-262, so the ONNX Runtime / torch goldens must never see padding):
+
+   - **ego crop** (``projectPoints_kernel`` L211-228): with the cached static ``sensor -> base_link`` transform as 12
+     float32 (row-major R, then t), a point is dropped iff it lies *strictly* inside the box on all three axes; the
+     reference-frame coordinates are evaluated in float32 as the kernel writes them
+     (``((r0 * x + r1 * y) + r2 * z) + t``; FMA contraction by nvcc could move a point within one ulp of a face);
+   - **projection** (``project2d`` L151-163) in float32: ``depth = sqrtf(x^2+y^2+z^2)``, ``yaw = -atan2f(y, x)``,
+     ``pitch = asinf(z / depth)``, ``u = 0.5 (yaw / pi + 1) W``, ``v = (1 - (pitch + |down|) / total) H``, NaN -> 0
+     (``fmaxf``), clamp to ``[0, W - 1]`` / ``[0, H - 1]``, truncate. float64 math moves 0.33 % of the points to
+     another cell (S:frnet:190-193); numpy's float32 ``arctan2`` / ``arcsin`` can still differ from CUDA's
+     ``atan2f`` / ``asinf`` by an ulp (unavoidable, documented);
+   - **interpolation grid** (L232-258): one point per pixel stored as 4 x fp16 (x, y, z, intensity); the
+     **farthest** point wins, by ``depth^2`` computed in float32 from the fp16-rounded x, y, z (exact: products of
+     fp16 values are exact in float32, so nvcc's FMA contraction cannot change it). The CUDA race between the
+     first plain store and the CAS loop, and the order among equal depths, are fixed here: the farthest point
+     wins, ties go to the later input point (the research reference's policy);
+   - **interpolation** (``interpolatePoints_kernel`` L332-379): an empty pixel with ``1 <= x <= W - 2`` whose left
+     and right neighbours are filled gets ``float(left + right) * 0.5`` (fp16 sum, no azimuth wrap-around), appended
+     after the raw points in row-major pixel order (CUDA appends with ``atomicAdd``: non-deterministic there);
+   - **frustum cells and unique** (``generateUniqueCoors`` L403-468): ``coors = (0, row, col)``, key
+     ``row * W + col``; ``voxel_coors`` = the occupied cells in ascending key order, ``inverse_map`` = the rank of
+     each point's cell -- here a counting sort over the ``H * W`` cells (S:frnet:460-466);
+   - **profile checks** (``lidar_frnet.cpp:138-149, 227-254``): the input count, ``N`` after interpolation and ``M``
+     must lie in the ``num_points`` / ``num_unique_coors`` profiles. The CUDA appends are not bounds-checked (an
+     over-profile frame overflows the device buffers before the ``N`` check, S:frnet:201-205); here every count is
+     checked after it is known and an out-of-profile frame is refused (:func:`profile_errors`).
+
+2. :func:`build_plan` -- the tables of the TT device graph (PLAN.md 2.8; PORT_LOG.md "trace layout"): the points
+   **sorted by cell** (stable, so a cell's points keep their input order) and padded to the capacity ``N_cap`` =
+   ``num_points.max``, the per-cell segments (CSR offsets, last row) of the sorted points, the fp32 8-d point
+   features with ``pre_norm`` folded in, and the gather indices that replace every scatter of the graph:
+   ``point2pix[s]`` (pixel of each sorted point at stride s: pixel2point) and ``pix2vox[s]`` (cell -> voxel row
+   or the zero sentinel row ``M_cap``: frustum2pixel; at s > 1 several cells fall on one pixel and the **last
+   voxel in voxel order** wins, which is what ORT's sequential ScatterND does, S:frnet:285-287). The network is
+   permutation-equivariant in the points, so the host un-permutes the device output with ``plan.order``.
+
+Nothing here imports torch, onnx or ttnn.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Sequence, Tuple
+
+import numpy as np
+
+from ..reference.config import STRIDES, FRNetConfig
+
+__all__ = ["NetworkInputs", "FramePlan", "crop_transform_12", "crop_keep_mask", "project2d", "interpolation_grid",
+           "interpolate_points", "unique_coors", "autoware_preprocess", "profile_errors", "decorate_points",
+           "build_plan", "segmax_rounds", "segmax_shift_tables"]
+
+PI_F32 = np.float32(np.pi)   # M_PIf
+
+
+# ------------------------------------------------------------------------------------------------- the node
+
+@dataclass
+class NetworkInputs:
+    """The four ONNX inputs of one frame, unpadded, plus the bookkeeping of how they were made."""
+
+    points: np.ndarray          # (N, 4) float32: kept raw points (input order), then interpolated points
+    coors: np.ndarray           # (N, 3) int64 (0, row, col) frustum cell of every point
+    voxel_coors: np.ndarray     # (M, 3) int64 occupied cells in ascending key order
+    inverse_map: np.ndarray     # (N,) int64 voxel row of every point
+    num_points_raw: int         # rows [0, num_points_raw) of `points` are raw input points
+    keep_mask: np.ndarray       # (N0,) bool: input rows that survived the ego crop box
+    crop_applied: bool = True
+
+    @property
+    def num_points(self) -> int:
+        return int(self.points.shape[0])
+
+    @property
+    def num_voxels(self) -> int:
+        return int(self.voxel_coors.shape[0])
+
+    @property
+    def num_interpolated(self) -> int:
+        return self.num_points - self.num_points_raw
+
+    def onnx_feeds(self) -> Dict[str, np.ndarray]:
+        """``{points, coors, voxel_coors, inverse_map}`` as the deployed ONNX binds them (float32 / int64)."""
+        return {"points": np.ascontiguousarray(self.points, np.float32),
+                "coors": np.ascontiguousarray(self.coors, np.int64),
+                "voxel_coors": np.ascontiguousarray(self.voxel_coors, np.int64),
+                "inverse_map": np.ascontiguousarray(self.inverse_map, np.int64)}
+
+    def stats(self) -> Dict[str, Any]:
+        counts = np.bincount(self.inverse_map) if self.num_points else np.zeros(1, np.int64)
+        return {"num_points_input": int(self.keep_mask.shape[0]), "num_points_raw": self.num_points_raw,
+                "num_points_interpolated": self.num_interpolated, "num_points": self.num_points,
+                "num_unique_coors": self.num_voxels, "cropped_by_ego_box": int((~self.keep_mask).sum()),
+                "crop_applied": self.crop_applied, "max_points_per_cell": int(counts.max()) if counts.size else 0}
+
+
+def crop_transform_12(T_ref_from_sensor: Any) -> np.ndarray:
+    """The node's cached crop transform (``lidar_frnet_node.cpp:160-180``): ``static_cast<float>`` of the 3x3
+    rotation (row-major) then the translation of the double ``base_link <- sensor`` transform -> (12,) float32."""
+    T = np.asarray(T_ref_from_sensor, np.float64)
+    if T.shape not in ((4, 4), (3, 4)):
+        raise ValueError(f"crop transform must be 4x4 or 3x4, got {T.shape}")
+    return np.concatenate([T[:3, :3].reshape(-1), T[:3, 3]]).astype(np.float32)
+
+
+def crop_keep_mask(xyz: np.ndarray, sensor_to_ref_12: np.ndarray, bounds_f32: np.ndarray) -> np.ndarray:
+    """``projectPoints_kernel`` L211-228: True = keep (outside the open box ``min < p < max`` on some axis)."""
+    T = np.asarray(sensor_to_ref_12, np.float32).reshape(12)
+    b = np.asarray(bounds_f32, np.float32).reshape(6)
+    x, y, z = (np.asarray(xyz[:, i], np.float32) for i in range(3))
+    px = ((T[0] * x + T[1] * y) + T[2] * z) + T[9]
+    py = ((T[3] * x + T[4] * y) + T[5] * z) + T[10]
+    pz = ((T[6] * x + T[7] * y) + T[8] * z) + T[11]
+    inside = (px > b[0]) & (px < b[3]) & (py > b[1]) & (py < b[4]) & (pz > b[2]) & (pz < b[5])
+    return ~inside
+
+
+def project2d(xyz: np.ndarray, scale_x: int, scale_y: int, fov: Tuple[np.float32, np.float32, np.float32]
+              ) -> Tuple[np.ndarray, np.ndarray]:
+    """``project2d`` (preprocess_kernel.cu L151-163) in float32 -> (col, row) int64, clamped then truncated."""
+    _, down, total = fov
+    x = np.asarray(xyz[:, 0], np.float32)
+    y = np.asarray(xyz[:, 1], np.float32)
+    z = np.asarray(xyz[:, 2], np.float32)
+    depth = np.sqrt(x * x + y * y + z * z, dtype=np.float32)
+    yaw = -np.arctan2(y, x).astype(np.float32)
+    with np.errstate(invalid="ignore", divide="ignore"):
+        pitch = np.arcsin(z / depth).astype(np.float32)
+    proj_x = np.float32(0.5) * (yaw / PI_F32 + np.float32(1.0))
+    proj_y = np.float32(1.0) - (pitch + np.float32(abs(down))) / np.float32(total)
+    proj_x = proj_x * np.float32(scale_x)
+    proj_y = proj_y * np.float32(scale_y)
+    proj_x = np.where(np.isnan(proj_x), np.float32(0.0), proj_x)      # fmaxf(NaN, 0) == 0
+    proj_y = np.where(np.isnan(proj_y), np.float32(0.0), proj_y)
+    px = np.minimum(np.maximum(proj_x, np.float32(0.0)), np.float32(scale_x) - np.float32(1.0))
+    py = np.minimum(np.maximum(proj_y, np.float32(0.0)), np.float32(scale_y) - np.float32(1.0))
+    return px.astype(np.int64), py.astype(np.int64)
+
+
+def interpolation_grid(raw: np.ndarray, config: FRNetConfig) -> Tuple[np.ndarray, np.ndarray]:
+    """The interpolation grid of ``projectPoints_kernel`` (L232-258) -> ``(grid (IH * IW, 4) float16, filled
+    (IH * IW,) bool)``: per pixel the farthest point by fp16 ``depth^2``, ties to the later input point."""
+    iw, ih = config.interp_w, config.interp_h
+    grid = np.zeros((iw * ih, 4), np.float16)
+    filled = np.zeros(iw * ih, bool)
+    if not len(raw):
+        return grid, filled
+    u, v = project2d(raw[:, :3], iw, ih, config.fov)
+    pix = v * iw + u
+    h = raw.astype(np.float16)                            # half4(x, y, z, intensity): __float2half (RNE)
+    hf = h[:, :3].astype(np.float32)
+    depth2 = hf[:, 0] * hf[:, 0] + hf[:, 1] * hf[:, 1] + hf[:, 2] * hf[:, 2]   # exact products (fp16 operands)
+    order = np.lexsort((np.arange(len(raw)), depth2, pix))   # by pixel, then depth, then input index
+    ps = pix[order]
+    last = np.r_[ps[1:] != ps[:-1], True]                 # last of each pixel group = farthest, latest on ties
+    grid[ps[last]] = h[order[last]]
+    filled[ps[last]] = True
+    return grid, filled
+
+
+def interpolate_points(grid: np.ndarray, filled: np.ndarray, config: FRNetConfig) -> np.ndarray:
+    """``interpolatePoints_kernel`` (L332-379) -> (K, 4) float32 points in row-major pixel order."""
+    iw, ih = config.interp_w, config.interp_h
+    f2 = filled.reshape(ih, iw)
+    cand = np.zeros_like(f2)
+    cand[:, 1:-1] = ~f2[:, 1:-1] & f2[:, :-2] & f2[:, 2:]
+    ys, xs = np.nonzero(cand)
+    g2 = grid.reshape(ih, iw, 4)
+    with np.errstate(over="ignore"):                                 # an fp16 overflow is inf, as __hadd gives
+        s16 = (g2[ys, xs - 1] + g2[ys, xs + 1]).astype(np.float16)  # __half operator+ (RNE)
+    return s16.astype(np.float32) * np.float32(0.5)
+
+
+def unique_coors(coors: np.ndarray, frustum_w: int, frustum_h: int) -> Tuple[np.ndarray, np.ndarray]:
+    """``generateUniqueCoors`` (L403-468) as a counting sort over the frustum cells -> ``(voxel_coors (M, 3) int64,
+    inverse_map (N,) int64)``: the occupied cells in ascending key order and each point's cell rank."""
+    keys = coors[:, 1] * frustum_w + coors[:, 2]
+    counts = np.bincount(keys, minlength=frustum_w * frustum_h)
+    occupied = counts > 0
+    cell_rank = np.cumsum(occupied) - 1
+    cells = np.flatnonzero(occupied)
+    voxel_coors = np.stack([np.zeros_like(cells), cells // frustum_w, cells % frustum_w], axis=1).astype(np.int64)
+    return voxel_coors, cell_rank[keys].astype(np.int64)
+
+
+def autoware_preprocess(points_xyzi: np.ndarray, config: FRNetConfig, *,
+                        sensor_to_ref_12: Optional[np.ndarray] = None,
+                        crop_bounds: Optional[Sequence[float]] = None) -> NetworkInputs:
+    """The node's pre-processing of one sensor-frame cloud ``(N0, 4)`` x, y, z, intensity (raw; float32) ->
+    :class:`NetworkInputs`. ``crop_bounds`` (6 numbers) overrides the config's box; the crop runs iff a bound is
+    non-zero, and then needs ``sensor_to_ref_12`` (:func:`crop_transform_12`): the node skips the frame when its TF
+    lookup fails, so a missing transform raises ``ValueError`` here. Profile checks are separate
+    (:func:`profile_errors`)."""
+    pts = np.ascontiguousarray(points_xyzi, np.float32)
+    if pts.ndim != 2 or pts.shape[1] != 4:
+        raise ValueError(f"points must be (N, 4) x, y, z, intensity, got {pts.shape}")
+    bounds = config.crop_bounds_f32 if crop_bounds is None else np.asarray(crop_bounds, np.float64).astype(np.float32)
+    crop = bool(np.any(bounds != 0.0))
+    if crop:
+        if sensor_to_ref_12 is None:
+            raise ValueError("the ego crop box is enabled but no sensor -> base_link transform was given")
+        keep = crop_keep_mask(pts[:, :3], sensor_to_ref_12, bounds)
+    else:
+        keep = np.ones(len(pts), bool)
+    raw = pts[keep]
+    grid, filled = interpolation_grid(raw, config)
+    interp = interpolate_points(grid, filled, config)
+    allpts = np.concatenate([raw, interp], axis=0) if len(interp) else raw.copy()
+    fx, fy = project2d(allpts[:, :3], config.frustum_w, config.frustum_h, config.fov)
+    coors = np.stack([np.zeros_like(fy), fy, fx], axis=1).astype(np.int64)
+    voxel_coors, inverse_map = unique_coors(coors, config.frustum_w, config.frustum_h)
+    return NetworkInputs(points=np.ascontiguousarray(allpts, np.float32), coors=coors, voxel_coors=voxel_coors,
+                         inverse_map=inverse_map, num_points_raw=int(len(raw)), keep_mask=keep, crop_applied=crop)
+
+
+def profile_errors(num_input_points: int, inputs: Optional[NetworkInputs], config: FRNetConfig) -> List[str]:
+    """The node's three profile checks, in its order (``lidar_frnet.cpp:138-149`` input count before the crop,
+    L227-237 ``N`` after interpolation, L243-254 ``M``); empty when the frame is accepted."""
+    lo, opt, hi = config.num_points
+    if not lo <= num_input_points <= hi:
+        return [f"Number of points ({num_input_points}) in input pointcloud does not match the profile "
+                f"[min: {lo}, opt: {opt}, max: {hi}]"]
+    if inputs is None:
+        return []
+    if not lo <= inputs.num_points <= hi:
+        return [f"Number of points ({inputs.num_points}) after interpolation does not match the profile "
+                f"[min: {lo}, opt: {opt}, max: {hi}]"]
+    ulo, uopt, uhi = config.num_unique
+    if not ulo <= inputs.num_voxels <= uhi:
+        return [f"Number of unique coors ({inputs.num_voxels}) does not match the profile "
+                f"[min: {ulo}, opt: {uopt}, max: {uhi}]"]
+    return []
+
+
+# ------------------------------------------------------------------------------------------- the TT device plan
+
+def decorate_points(points: np.ndarray, inverse_map: np.ndarray, num_voxels: int, *,
+                    pre_norm: Optional[Tuple[np.ndarray, np.ndarray]] = None) -> np.ndarray:
+    """The frustum encoder's 8-d point features ``[x, y, z, i, |xyz|, x - mx, y - my, z - mz]`` (``m`` = the
+    cell mean of x, y, z; AWML ``frustum_encoder.py:79-100``), computed in float64. With ``pre_norm = (scale,
+    shift)`` the ``pre_norm`` BatchNorm1d is applied (``x * scale + shift``, float64) -- the port's host fold
+    (PLAN.md 2.8: xyz up to +-200 m minus a cell mean needs fp32 or better, never bf16). One rounding to float32."""
+    p = np.asarray(points, np.float64)
+    n = len(p)
+    out = np.empty((n, 8), np.float64)
+    out[:, :4] = p
+    out[:, 4] = np.sqrt((p[:, :3] ** 2).sum(axis=1))
+    if n:
+        counts = np.bincount(inverse_map, minlength=num_voxels).astype(np.float64)
+        for c in range(3):
+            mean = np.bincount(inverse_map, weights=p[:, c], minlength=num_voxels) / np.maximum(counts, 1.0)
+            out[:, 5 + c] = p[:, c] - mean[inverse_map]
+    if pre_norm is not None:
+        scale, shift = (np.asarray(a, np.float64).reshape(8) for a in pre_norm)
+        out = out * scale + shift
+    return out.astype(np.float32)
+
+
+def segmax_rounds(max_segment_length: int) -> int:
+    """Rounds of the log-step segmented max (Hillis-Steele) that cover segments up to that length:
+    ``ceil(log2(L))`` (0 for L <= 1). PLAN.md 1.3: 5 for OT128 (<= 17 points per cell seen), 7 for QT128 (<= 111)."""
+    return int(np.ceil(np.log2(max_segment_length))) if max_segment_length > 1 else 0
+
+
+@dataclass
+class FramePlan:
+    """The per-frame host tables of the device graph (module docstring, part 2). Row ``i`` of every per-point table
+    is the ``i``-th point in **sorted** order; rows ``N .. N_cap - 1`` are padding (one trailing sentinel segment:
+    their voxel id is ``M_cap``, their pixel 0, their features 0, and nothing the output keeps reads them)."""
+
+    n: int                                   # real points
+    m: int                                   # real voxels (occupied cells)
+    n_raw: int
+    n_cap: int
+    m_cap: int
+    frustum_hw: Tuple[int, int]
+    order: np.ndarray                        # (N,) int64: sorted row -> network-input point index
+    seg_id: np.ndarray                       # (N_cap,) int64 voxel id of each sorted row (padding: M_cap)
+    seg_offsets: np.ndarray                  # (M + 1,) int64 CSR offsets of the voxels' sorted rows
+    seg_last: np.ndarray                     # (M_cap,) uint32 last sorted row of each voxel (unused voxels: N_cap - 1)
+    point2pix: Dict[int, np.ndarray]         # stride -> (N_cap,) uint32 pixel of each sorted row (padding: 0)
+    pix2vox: Dict[int, np.ndarray]           # stride -> ((H/s) * (W/s),) uint32 voxel row or M_cap (zero row)
+    features: np.ndarray                     # (N_cap, 8) float32 decorated (+ pre_norm) features
+    max_segment_length: int = 0
+    strides: Tuple[int, ...] = field(default=STRIDES)
+
+    @property
+    def rank(self) -> np.ndarray:
+        """(N,) int64: network-input point index -> sorted row (the inverse of ``order``)."""
+        r = np.empty(self.n, np.int64)
+        r[self.order] = np.arange(self.n)
+        return r
+
+    def unsort(self, sorted_rows: np.ndarray) -> np.ndarray:
+        """Device rows (sorted, padded) -> the network-input point order (the first ``N`` sorted rows only)."""
+        return np.asarray(sorted_rows)[: self.n][self.rank]
+
+    def stats(self) -> Dict[str, Any]:
+        return {"n": self.n, "m": self.m, "n_cap": self.n_cap, "m_cap": self.m_cap,
+                "max_points_per_cell": self.max_segment_length,
+                "segmax_rounds": segmax_rounds(self.max_segment_length)}
+
+
+def build_plan(inputs: NetworkInputs, config: FRNetConfig, *,
+               pre_norm: Optional[Tuple[np.ndarray, np.ndarray]] = None,
+               n_cap: Optional[int] = None, m_cap: Optional[int] = None,
+               strides: Sequence[int] = STRIDES) -> FramePlan:
+    """The device tables of one accepted frame (:class:`FramePlan`). ``n_cap`` / ``m_cap`` default to the
+    config's capacities (COMPILE: the profile maxima); a frame above them raises ``ValueError`` (the profile check
+    refuses it first)."""
+    n_cap = config.n_cap if n_cap is None else int(n_cap)
+    m_cap = config.m_cap if m_cap is None else int(m_cap)
+    n, m = inputs.num_points, inputs.num_voxels
+    if n > n_cap or m > m_cap:
+        raise ValueError(f"frame N={n} / M={m} exceeds the capacity N_cap={n_cap} / M_cap={m_cap}")
+    H, W = config.frustum_h, config.frustum_w
+    inv = inputs.inverse_map
+    order = np.argsort(inv, kind="stable")              # voxel ids are key ranks: == the stable sort by cell key
+    counts = np.bincount(inv, minlength=m).astype(np.int64)
+    offsets = np.zeros(m + 1, np.int64)
+    np.cumsum(counts, out=offsets[1:])
+    seg_id = np.full(n_cap, m_cap, np.int64)
+    seg_id[:n] = inv[order]
+    seg_last = np.full(m_cap, n_cap - 1, np.uint32)
+    seg_last[:m] = (offsets[1:] - 1).astype(np.uint32)
+    coors_sorted = inputs.coors[order]
+    vox = inputs.voxel_coors
+    point2pix: Dict[int, np.ndarray] = {}
+    pix2vox: Dict[int, np.ndarray] = {}
+    for s in strides:
+        hs, ws = H // s, W // s
+        p2p = np.zeros(n_cap, np.uint32)
+        p2p[:n] = ((coors_sorted[:, 1] // s) * ws + coors_sorted[:, 2] // s).astype(np.uint32)
+        point2pix[s] = p2p
+        last = np.full(hs * ws, -1, np.int64)
+        np.maximum.at(last, (vox[:, 1] // s) * ws + vox[:, 2] // s, np.arange(m, dtype=np.int64))
+        pix2vox[s] = np.where(last < 0, m_cap, last).astype(np.uint32)
+    feats = np.zeros((n_cap, 8), np.float32)
+    feats[:n] = decorate_points(inputs.points[order], inv[order], m, pre_norm=pre_norm)
+    return FramePlan(n=n, m=m, n_raw=inputs.num_points_raw, n_cap=n_cap, m_cap=m_cap, frustum_hw=(H, W),
+                     order=order, seg_id=seg_id, seg_offsets=offsets, seg_last=seg_last, point2pix=point2pix,
+                     pix2vox=pix2vox, features=feats, max_segment_length=int(counts.max()) if m else 0,
+                     strides=tuple(strides))
+
+
+def segmax_shift_tables(seg_id: np.ndarray, rounds: int) -> List[np.ndarray]:
+    """The gather tables of the exact log-step segmented max (PLAN.md 1.3, the K1 stock-op fallback): round
+    ``r`` uses ``idx[i] = i - 2^r`` if row ``i - 2^r`` is in the same segment, else ``i`` (max is idempotent);
+    after ``rounds`` rounds each segment's last row holds the segment max when its length is <= ``2^rounds``.
+    -> ``rounds`` x (N_cap,) uint32."""
+    seg = np.asarray(seg_id)
+    i = np.arange(len(seg), dtype=np.int64)
+    out = []
+    for r in range(rounds):
+        d = 1 << r
+        j = i - d
+        same = np.zeros(len(seg), bool)
+        same[d:] = seg[d:] == seg[:-d]
+        out.append(np.where(same, j, i).astype(np.uint32))
+    return out

code/tt_frnet/io.py

@@ -0,0 +1,34 @@
+# SPDX-License-Identifier: Apache-2.0
+"""Input decoding and output encoding shared by the Python API and the HTTP server of frnet-p150.
+
+The implementation is the vendored ``ttaw.io`` (C08): point clouds (``bin`` / ``npy`` / ``npz`` / ``pcd`` /
+``list``), images, cameras + calibration, transforms, named arrays and the output encoders; numpy only, data
+parsing only, and every client mistake raises :class:`InputError`, which the server maps to HTTP 400. This module
+re-exports it with the Autoware input layout of this model bound as the default point fields, so ``model(points)``
+and ``POST /predict`` decode the same bytes into the same array.
+"""
+from __future__ import annotations
+
+from typing import Any, Mapping, Optional, Sequence
+
+from .ttaw import io as _io
+from .ttaw.io import *  # noqa: F401,F403  (the decoders and encoders: ttaw/API.md section 9)
+from .ttaw.io import MAX_POINTS_DEFAULT, PointCloud
+
+__all__ = list(_io.__all__)  # the same public names; the three below are rebound to this model's layout
+
+DEFAULT_POINT_FIELDS = tuple(["x", "y", "z", "intensity"])  # the Autoware input layout of this model
+
+
+def decode_points(spec: Mapping[str, Any], *, max_points: int = MAX_POINTS_DEFAULT, max_bytes: Optional[int] = None,
+                  default_fields: Sequence[str] = DEFAULT_POINT_FIELDS) -> PointCloud:
+    """The ``points`` envelope of ``/predict`` -> :class:`PointCloud` (``ttaw.io.decode_points`` with this model's
+    default fields)."""
+    return _io.decode_points(spec, max_points=max_points, max_bytes=max_bytes, default_fields=default_fields)
+
+
+def load_points(source: Any, *, fields: Optional[Sequence[str]] = None, fmt: Optional[str] = None,
+                frame_id: str = "base_link", default_fields: Sequence[str] = DEFAULT_POINT_FIELDS) -> PointCloud:
+    """Python-API point input (path, bytes + ``fmt``, (N, C) array or tensor, structured array, envelope,
+    :class:`PointCloud`) -> :class:`PointCloud` (``ttaw.io.load_points`` with this model's default fields)."""
+    return _io.load_points(source, fields=fields, fmt=fmt, frame_id=frame_id, default_fields=default_fields)

code/tt_frnet/reference/__init__.py

@@ -0,0 +1,26 @@
+# SPDX-License-Identifier: Apache-2.0
+"""CPU reference of FRNet (Autoware lidar_frnet) -- the ground truth every PCC / output-agreement gate compares
+against (PLAN.md 0.3 item 1). It runs where ttnn is absent (the research venv) and never imports ttnn.
+
+- ``config.py``    the Autoware parameters this port reproduces (``ml_package_frnet_<variant>.param.yaml`` of the
+                   weights, ``frnet.param.yaml`` node parameters) and the COMPILE constants, each with its source line;
+- ``weights.py``   the deployed ONNX read as data with the vendored ``ttaw.weights.OnnxWeights`` (initializers by
+                   consuming node; structure and every wiring assumption checked on the graph; BN folded in float64
+                   into the point layers, ``pre_norm`` kept as the host's scale / shift) -- the loader the TT port
+                   uses too;
+- ``model.py``     fp32 torch rebuild of the deployed graph with per-module taps (``ttaw.golden.TapRegistry``; names
+                   in ``weights.MODULE_TAPS`` / ``OPT_IN_TAPS``), ported from research/frnet/scripts/frnet_torch_ref.py;
+- ``rewrites.py``  the TT device dataflow (sorted + padded points, gather-form scatters with a zero sentinel row, the
+                   exact log-step segmented max, K-split convs and point layers, interpolation-matrix up-sampling)
+                   proven equal to ``model.py`` on the CPU, and a first-order precision emulation;
+- ``ort.py``       ONNX Runtime golden of the deployed ONNX with matching intermediates (research / tests only:
+                   onnxruntime is not in the image);
+- ``pipeline.py``  the reference end to end: ``host/`` pre-processing -> torch / ORT / plan network -> ``host/``
+                   post-processing -> ``FRNetSegmentation``.
+
+Pre- and post-processing are not duplicated here: they live in ``tt_frnet.host`` (shared with the device path, so
+TT and CPU see identical network inputs) and are tested against the research scripts.
+
+The reduced ``to_dict()`` of the ORT reference on a sample is stored per variant next to it as
+``samples/synthetic_<variant>.<variant>.reference.json``: ``server/smoke_test.py`` compares the served output with it.
+"""

code/tt_frnet/reference/config.py

@@ -0,0 +1,284 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The Autoware ``lidar_frnet`` parameters this port reproduces, and the port's compile-time constants.
+
+Sources (autoware_universe main ``9ceaccf``, ``perception/autoware_lidar_frnet`` = ``PKG/``; weights
+``AutowareFoundation/lidar_frnet@v2.0`` = commit ``a53a1c11``; research/frnet/SPEC.md sections 1-3):
+
+- per variant, from the weights repo: ``ml_package_frnet_<variant>.param.yaml`` (``fov_up_deg`` / ``fov_down_deg``,
+  ``frustum_width`` / ``frustum_height`` (the network grid), ``interpolation_width`` / ``interpolation_height`` (the
+  pre-processing grid), the 27 ``class_names``, the 81-int ``palette``, the ``num_points`` / ``num_unique_coors``
+  [min, opt, max] profiles) and ``deploy_metadata.yaml`` (``version: v2.0``);
+- node-wide, :data:`NODE_PARAMS` = ``PKG/config/frnet.param.yaml:3-16`` (filter threshold and classes, the filtered
+  cloud's output format, the ego crop box).
+
+Every validation the node does on these values is repeated here: palette size and 0-255 range
+(``PKG/include/.../utils.hpp`` ``make_palette``), filter classes must be class names (``make_filter_class_indices``),
+the crop box is enabled iff any float32 bound is non-zero (``PKG/src/lidar_frnet_node.cpp:51-66``). ``yaml`` is
+imported when a file is read, never at module import; nothing here imports torch / onnx / ttnn.
+
+The two variants are an inseparable tuple (weights, frustum H x W baked into the ONNX, FOV, interpolation grid:
+S:frnet:535-538): a user selects ``ot128`` / ``qt128`` only (LOAD), every number below follows from it.
+"""
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass, field, replace
+from pathlib import Path
+from typing import Any, Dict, Mapping, Optional, Sequence, Tuple
+
+import numpy as np
+
+__all__ = [
+    "VARIANTS", "DEFAULT_VARIANT", "DEPLOY_VERSION", "ONNX_SHA256", "NODE_PARAMS", "NUM_CLASSES", "STRIDES",
+    "BLOCKS", "CROP_BOUND_KEYS", "FRNetConfig", "variant_files", "parse_crop_box", "parse_filter_classes",
+]
+
+VARIANTS = ("ot128", "qt128")       # PKG/launch/lidar_frnet.launch.xml:9-12 (sensor_model choices)
+DEFAULT_VARIANT = "ot128"           # the launch default (Hesai OT128)
+DEPLOY_VERSION = "v2.0"             # deploy_metadata.yaml of the pinned weights; Autoware's ansible pin (S:frnet:75-80)
+
+# sha256 of the deployed graphs at the pinned revision (S:frnet:84-93): a different file is refused.
+ONNX_SHA256 = {
+    "ot128": "d29bf5950432360d800c22ff86a590d4d9d006acc501563ac611cfa33747c11f",
+    "qt128": "647af92790e3d591a19d065fe94d3ded797916ac9ffe5bf96b5e8b8324b31ad2",
+}
+
+# PKG/config/frnet.param.yaml:3-16 (node parameters; RT-host in this port)
+NODE_PARAMS: Dict[str, Any] = {
+    "class_probability_threshold": 0.05,
+    "filter_classes": ("drivable_surface",),
+    "output_format": "",
+    "ego_crop_box": {"reference_frame": "base_link", "min_x": -1.53, "min_y": -1.15, "min_z": -0.1,
+                     "max_x": 6.1, "max_y": 1.15, "max_z": 3.1},
+}
+CROP_BOUND_KEYS = ("min_x", "min_y", "min_z", "max_x", "max_y", "max_z")
+
+NUM_CLASSES = 27                    # conv_seg Gemm 64 -> 27; ml_package class_names (COMPILE)
+STRIDES = (1, 2, 4, 8)              # AWML configs/model/frnet.py: backbone strides of layer1..4 (checked on the graph)
+BLOCKS = (3, 4, 6, 3)               # BasicBlocks per stage (checked on the graph)
+POSTPROCESS_MAX_CLASSES = 64        # PKG/lib/postprocess/postprocess_kernel.cu:32 (const_palette[64])
+POSTPROCESS_MAX_FILTER_CLASSES = 16  # postprocess_kernel.cu:30 (const_filter_class_indices[16])
+
+
+def variant_files(weights_dir: Any, variant: str) -> Dict[str, Path]:
+    """The files of one variant in a weights directory (the flat HF repo layout)."""
+    if variant not in VARIANTS:
+        raise ValueError(f"unknown variant {variant!r}; choose one of {VARIANTS}")
+    root = Path(weights_dir)
+    return {"onnx": root / f"frnet_{variant}.onnx", "ml_package": root / f"ml_package_frnet_{variant}.param.yaml",
+            "deploy_metadata": root / "deploy_metadata.yaml"}
+
+
+def _ros_params(path: Path) -> Dict[str, Any]:
+    import yaml
+
+    with open(path) as f:
+        doc = yaml.safe_load(f)
+    try:
+        return doc["/**"]["ros__parameters"]
+    except (KeyError, TypeError):
+        raise ValueError(f"{path}: not a ROS 2 parameter file (no /**: ros__parameters)") from None
+
+
+def _profile(values: Sequence[Any], name: str) -> Tuple[int, int, int]:
+    v = tuple(int(x) for x in values)
+    if len(v) != 3 or not (0 < v[0] <= v[1] <= v[2]):
+        raise ValueError(f"{name} must be [min, opt, max] with 0 < min <= opt <= max, got {list(values)}")
+    return v  # type: ignore[return-value]
+
+
+@dataclass(frozen=True)
+class FRNetConfig:
+    """One variant's parameters. COMPILE values (device shapes): ``frustum_h`` x ``frustum_w``, ``n_cap``,
+    ``m_cap``, ``num_classes``; LOAD: the variant itself; RT-host: the node parameters (filter, crop box)."""
+
+    variant: str
+    version: str
+    fov_up_deg: float
+    fov_down_deg: float
+    frustum_w: int
+    frustum_h: int
+    interp_w: int
+    interp_h: int
+    class_names: Tuple[str, ...]
+    palette: Tuple[int, ...]                       # 3 x num_classes ints, RGB per class
+    num_points: Tuple[int, int, int]               # [min, opt, max] after interpolation (and of the raw input)
+    num_unique: Tuple[int, int, int]               # [min, opt, max] unique frustum cells
+    class_probability_threshold: float = NODE_PARAMS["class_probability_threshold"]
+    filter_classes: Tuple[str, ...] = NODE_PARAMS["filter_classes"]
+    crop_bounds: Tuple[float, ...] = tuple(NODE_PARAMS["ego_crop_box"][k] for k in CROP_BOUND_KEYS)
+    crop_reference_frame: str = NODE_PARAMS["ego_crop_box"]["reference_frame"]
+    model_dir: Optional[str] = field(default=None, compare=False)
+
+    # ---- construction --------------------------------------------------------------------------------------
+    @classmethod
+    def from_model_dir(cls, model_dir: Any, variant: str = DEFAULT_VARIANT, *,
+                       check_version: bool = True) -> "FRNetConfig":
+        """Read ``ml_package_frnet_<variant>.param.yaml`` (and check ``deploy_metadata.yaml``) of a weights
+        directory laid out as the HF repo."""
+        files = variant_files(model_dir, variant)
+        p = _ros_params(files["ml_package"])
+        version = DEPLOY_VERSION
+        if files["deploy_metadata"].is_file():
+            import yaml
+
+            meta = yaml.safe_load(files["deploy_metadata"].read_text()) or {}
+            version = str(meta.get("version", ""))
+            if check_version and version != DEPLOY_VERSION:
+                raise ValueError(f"deploy_metadata.yaml version {version!r} != the pinned {DEPLOY_VERSION!r}")
+        cfg = cls(variant=variant, version=version, fov_up_deg=float(p["fov_up_deg"]),
+                  fov_down_deg=float(p["fov_down_deg"]), frustum_w=int(p["frustum_width"]),
+                  frustum_h=int(p["frustum_height"]), interp_w=int(p["interpolation_width"]),
+                  interp_h=int(p["interpolation_height"]), class_names=tuple(str(c) for c in p["class_names"]),
+                  palette=tuple(int(v) for v in p["palette"]), num_points=_profile(p["num_points"], "num_points"),
+                  num_unique=_profile(p["num_unique_coors"], "num_unique_coors"), model_dir=str(model_dir))
+        cfg.validate()
+        return cfg
+
+    def validate(self) -> None:
+        """The node's own checks (utils.hpp ``Dims2d``, ``make_palette``, ``make_filter_class_indices``)."""
+        for name in ("frustum_w", "frustum_h", "interp_w", "interp_h"):
+            if getattr(self, name) <= 0:
+                raise ValueError(f"{name} must be positive")
+        n = len(self.class_names)
+        if n == 0 or n > POSTPROCESS_MAX_CLASSES:
+            raise ValueError(f"{n} classes; the node supports 1..{POSTPROCESS_MAX_CLASSES}")
+        if len(self.palette) % 3:
+            raise ValueError("Palette size must be a multiple of 3.")
+        if len(self.palette) != 3 * n:
+            raise ValueError("Palette size does not match class names size.")
+        if any(v < 0 or v > 255 for v in self.palette):
+            raise ValueError("Color values must be within 0-255 range.")
+        self.filter_class_indices()
+        if len(self.crop_bounds) != 6:
+            raise ValueError("ego crop box needs 6 bounds (min_x, min_y, min_z, max_x, max_y, max_z)")
+
+    def with_node_params(self, **overrides: Any) -> "FRNetConfig":
+        """A copy with other node parameters (``class_probability_threshold``, ``filter_classes``,
+        ``crop_bounds``); re-validated."""
+        cfg = replace(self, **overrides)
+        cfg.validate()
+        return cfg
+
+    # ---- derived values (each computed as the node computes it) ---------------------------------------------
+    @property
+    def num_classes(self) -> int:
+        return len(self.class_names)
+
+    @property
+    def fov(self) -> Tuple[np.float32, np.float32, np.float32]:
+        """utils.hpp ``FieldOfView``: ``(up, down, total)`` radians, computed in double, stored as float32."""
+        up = np.float32(np.pi * self.fov_up_deg / 180.0)
+        down = np.float32(np.pi * self.fov_down_deg / 180.0)
+        total = np.float32(np.pi * abs(self.fov_up_deg - self.fov_down_deg) / 180.0)
+        return up, down, total
+
+    @property
+    def crop_bounds_f32(self) -> np.ndarray:
+        """``static_cast<float>`` of the six bounds (lidar_frnet_node.cpp:53-58)."""
+        return np.asarray(self.crop_bounds, np.float64).astype(np.float32)
+
+    @property
+    def crop_enabled(self) -> bool:
+        """Enabled iff any float32 bound is non-zero (lidar_frnet_node.cpp:59-66)."""
+        return bool(np.any(self.crop_bounds_f32 != 0.0))
+
+    def filter_class_indices(self, names: Optional[Sequence[str]] = None) -> Tuple[int, ...]:
+        """Indices of ``names`` (default: the configured filter classes) in ``class_names``; an unknown name raises
+        ``ValueError`` (the node throws ``Filter class '<name>' not found in class names.``)."""
+        names = self.filter_classes if names is None else tuple(names)
+        if len(names) > POSTPROCESS_MAX_FILTER_CLASSES:
+            raise ValueError(f"at most {POSTPROCESS_MAX_FILTER_CLASSES} filter classes")
+        out = []
+        for name in names:
+            if name not in self.class_names:
+                raise ValueError(f"Filter class '{name}' not found in class names.")
+            out.append(self.class_names.index(name))
+        return tuple(out)
+
+    @property
+    def palette_rgb(self) -> np.ndarray:
+        """``(num_classes, 3)`` uint8."""
+        return np.asarray(self.palette, np.int64).reshape(-1, 3).astype(np.uint8)
+
+    @property
+    def palette_packed(self) -> np.ndarray:
+        """utils.hpp ``make_palette``: ``(r << 16) | (g << 8) | b`` reinterpreted as float32 (the visualization
+        cloud's ``rgb`` field), ``(num_classes,)`` float32."""
+        rgb = self.palette_rgb.astype(np.uint32)
+        return ((rgb[:, 0] << 16) | (rgb[:, 1] << 8) | rgb[:, 2]).astype(np.uint32).view(np.float32)
+
+    @property
+    def n_cap(self) -> int:
+        """Point capacity of the device graph (COMPILE): the profile maximum, so every frame Autoware accepts fits
+        (functional port; smaller buckets are optimization work)."""
+        return self.num_points[2]
+
+    @property
+    def m_cap(self) -> int:
+        """Frustum-cell (voxel) capacity of the device graph (COMPILE); the gather tables add one zero row."""
+        return self.num_unique[2]
+
+    @property
+    def frustum_cells(self) -> int:
+        return self.frustum_h * self.frustum_w
+
+    def research_cfg(self) -> Dict[str, Any]:
+        """The same values in the layout of research/frnet/scripts/frnet_preproc_ref.py ``CFGS[variant]``."""
+        return dict(name=self.variant, fov_up_deg=self.fov_up_deg, fov_down_deg=self.fov_down_deg,
+                    frustum_w=self.frustum_w, frustum_h=self.frustum_h, interp_w=self.interp_w,
+                    interp_h=self.interp_h, num_points=self.num_points, num_unique=self.num_unique)
+
+    def describe(self) -> Dict[str, Any]:
+        return {"variant": self.variant, "version": self.version, "fov_deg": [self.fov_up_deg, self.fov_down_deg],
+                "frustum_hw": [self.frustum_h, self.frustum_w], "interpolation_hw": [self.interp_h, self.interp_w],
+                "num_points": list(self.num_points), "num_unique_coors": list(self.num_unique),
+                "num_classes": self.num_classes, "class_probability_threshold": self.class_probability_threshold,
+                "filter_classes": list(self.filter_classes), "ego_crop_box": dict(zip(CROP_BOUND_KEYS,
+                                                                                      self.crop_bounds)),
+                "ego_crop_reference_frame": self.crop_reference_frame}
+
+
+def parse_crop_box(value: Any) -> Optional[Tuple[float, ...]]:
+    """An ego crop box override: ``None`` / ``False`` (disabled = all-zero bounds, the node's own switch), six
+    numbers ``[min_x, min_y, min_z, max_x, max_y, max_z]``, or a mapping with those keys (missing keys keep
+    Autoware's defaults). Raises ``ValueError`` on anything else."""
+    if value is None or value is False:
+        return (0.0,) * 6
+    if isinstance(value, Mapping):
+        unknown = sorted(set(value) - set(CROP_BOUND_KEYS) - {"reference_frame"})
+        if unknown:
+            raise ValueError(f"ego_crop_box: unknown keys {unknown}")
+        if value.get("reference_frame", "base_link") != "base_link":
+            raise ValueError("ego_crop_box.reference_frame must be base_link (the transform's target frame)")
+        base = NODE_PARAMS["ego_crop_box"]
+        vals = [value.get(k, base[k]) for k in CROP_BOUND_KEYS]
+    else:
+        vals = list(value)
+        if len(vals) != 6:
+            raise ValueError("ego_crop_box needs 6 numbers [min_x, min_y, min_z, max_x, max_y, max_z]")
+    try:
+        out = tuple(float(v) for v in vals)
+    except (TypeError, ValueError):
+        raise ValueError("ego_crop_box bounds must be numbers") from None
+    if not all(np.isfinite(out)):
+        raise ValueError("ego_crop_box bounds must be finite")
+    return out
+
+
+_IDENT = re.compile(r"^[A-Za-z0-9_+\-]+$")
+
+
+def parse_filter_classes(value: Any) -> Tuple[str, ...]:
+    """``filter_classes`` as a request gives it: a comma-separated string (``""`` = no filter class) or a list of
+    names. Names are checked against the class list by :meth:`FRNetConfig.filter_class_indices`."""
+    if value is None:
+        return ()
+    if isinstance(value, str):
+        names = [v.strip() for v in value.split(",") if v.strip()]
+    else:
+        names = [str(v).strip() for v in value]
+    bad = [n for n in names if not _IDENT.match(n)]
+    if bad:
+        raise ValueError(f"invalid filter class names {bad}")
+    return tuple(names)

code/tt_frnet/reference/model.py

@@ -0,0 +1,179 @@
+# SPDX-License-Identifier: Apache-2.0
+"""fp32 torch rebuild of the deployed FRNet graph (AWML ``frustum_encoder.py:76-137``, ``frnet_backbone.py:252-356``,
+``frnet_head.py:52-70`` in deploy mode), with per-module taps. Ported from the verified research reference
+``research/frnet/scripts/frnet_torch_ref.py`` (max |diff| vs ORT 1.6e-5 .. 3.0e-5 on the public goldens), on the
+weights as the TT port holds them (``reference/weights.py``: point-layer BatchNorms folded, ``pre_norm`` as x * scale
++ shift).
+
+Input: the four ONNX inputs of one frame, **unpadded** (``host/preprocess.py`` ``NetworkInputs``); the graph sizes
+its voxel tensors as ``max(inverse_map) + 1`` (S:frnet:255-262). Every sparse op is deterministic and follows ORT:
+
+- cell mean: index-add of the points / of ones, divided (``ScatterElements(add)`` x2 + ``Div``);
+- per-cell max (``ScatterElements(max)``, init ``-FLT_MAX``): ``scatter_reduce(amax, include_self=True)``;
+- frustum2pixel (``ScatterND`` into zeros, NHWC -> NCHW): at strides 2 / 4 / 8 several cells fall on one pixel
+  and ORT applies the updates sequentially, so the **last voxel row** wins (S:frnet:285-287);
+- pixel2point (``Gather``): row ``(row // s) * (W // s) + col // s`` of the NHWC map;
+- decoder: bilinear ``align_corners=True`` resize of the five maps to H x W, channel concat.
+
+Taps (``ttaw.golden.TapRegistry``; names in ``weights.MODULE_TAPS`` / ``OPT_IN_TAPS``): point tensors ``(N, C)``,
+voxel tensors ``(M, C)``, maps ``(1, C, h, w)`` NCHW, all float32 numpy. ``forward`` returns ``pred_probs``
+``(N, 27)`` float32. torch is imported here (research venv / tt-env), never by ``api.py`` at module level.
+"""
+from __future__ import annotations
+
+from typing import Any, Optional
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+
+from ..ttaw.golden import NULL_TAPS, TapRegistry
+from .weights import Conv, FRNetWeights, Linear
+
+__all__ = ["FRNetTorch", "scatter_max", "frustum2pixel", "pixel2point", "TorchLayers"]
+
+
+def _t(a: np.ndarray) -> torch.Tensor:
+    a = np.ascontiguousarray(a)
+    return torch.from_numpy(a if a.flags.writeable else a.copy())
+
+
+class TorchLayers:
+    """The weights as torch tensors (shared by the literal model and the rewrites)."""
+
+    def __init__(self, weights: FRNetWeights):
+        self.w = weights
+        self.lin = {lin.name: (_t(lin.weight), _t(lin.bias), lin.relu) for lin in weights.linears}
+        self.conv = {c.name: (_t(c.weight), _t(c.bias), c.stride, c.padding, c.activation) for c in weights.convs}
+        self.pre_scale = _t(weights.pre_norm_scale)
+        self.pre_shift = _t(weights.pre_norm_shift)
+
+    def linear(self, x: torch.Tensor, layer: Linear) -> torch.Tensor:
+        w, b, relu = self.lin[layer.name]
+        y = torch.addmm(b, x, w)
+        return F.relu(y) if relu else y
+
+    def conv2d(self, x: torch.Tensor, layer: Conv, *, activation: bool = True) -> torch.Tensor:
+        w, b, stride, pad, act = self.conv[layer.name]
+        y = F.conv2d(x, w, b, stride=stride, padding=pad)
+        if activation and act == "hardswish":
+            y = F.hardswish(y)
+        elif activation and act == "sigmoid":
+            y = torch.sigmoid(y)
+        return y
+
+
+def scatter_max(src: torch.Tensor, inverse_map: torch.Tensor, m: int) -> torch.Tensor:
+    """``ScatterElements(reduction=max)`` into ``-FLT_MAX`` rows (every row of a frame owns >= 1 point)."""
+    out = torch.full((m, src.shape[1]), torch.finfo(src.dtype).min, dtype=src.dtype)
+    return out.scatter_reduce(0, inverse_map.unsqueeze(-1).expand(-1, src.shape[1]), src, reduce="amax",
+                              include_self=True)
+
+
+def frustum2pixel(feats: torch.Tensor, voxel_coors: np.ndarray, hw: tuple, stride: int) -> torch.Tensor:
+    """``ScatterND`` of voxel rows into a zero ``(1, h, w, C)`` image at ``voxel_coors // stride`` (last voxel wins
+    on duplicates), returned NCHW."""
+    H, W = hw
+    h, w = H // stride, W // stride
+    lin = (voxel_coors[:, 1] // stride) * w + voxel_coors[:, 2] // stride
+    last = np.full(h * w, -1, np.int64)
+    np.maximum.at(last, lin, np.arange(len(lin), dtype=np.int64))
+    pix = np.flatnonzero(last >= 0)
+    img = torch.zeros(h * w, feats.shape[1], dtype=feats.dtype)
+    img[_t(pix)] = feats[_t(last[pix])]
+    return img.reshape(1, h, w, -1).permute(0, 3, 1, 2).contiguous()
+
+
+def pixel2point(x: torch.Tensor, coors: np.ndarray, stride: int) -> torch.Tensor:
+    """``Gather`` of the NHWC rows of ``x`` (NCHW) at each point's cell ``// stride``."""
+    _, c, h, w = x.shape
+    flat = x.permute(0, 2, 3, 1).reshape(h * w, c)
+    return flat[_t((coors[:, 1] // stride) * w + coors[:, 2] // stride)]
+
+
+class FRNetTorch:
+    """``FRNetTorch(weights)(points, coors, voxel_coors, inverse_map, taps=TapRegistry())`` -> pred_probs."""
+
+    def __init__(self, weights: FRNetWeights, *, threads: Optional[int] = None):
+        if threads:
+            torch.set_num_threads(int(threads))
+        self.weights = weights
+        self.layers = TorchLayers(weights)
+
+    def __call__(self, *args: Any, **kwargs: Any) -> np.ndarray:
+        return self.forward(*args, **kwargs)
+
+    @torch.no_grad()
+    def forward(self, points: np.ndarray, coors: np.ndarray, voxel_coors: np.ndarray, inverse_map: np.ndarray,
+                taps: TapRegistry = NULL_TAPS) -> np.ndarray:
+        W, L = self.weights, self.layers
+        hw = W.frustum_hw
+        m = int(inverse_map.max()) + 1
+        voxel_coors = np.asarray(voxel_coors)[:m]
+        coors = np.asarray(coors)
+        pts = _t(np.asarray(points, np.float32))
+        inv = _t(np.asarray(inverse_map, np.int64))
+
+        def tap(name: str, value: torch.Tensor) -> torch.Tensor:
+            if taps.wants(name):
+                taps.tap(name, value.numpy())
+            return value
+
+        # ---- FrustumFeatureEncoder
+        dist = torch.linalg.norm(pts[:, :3], dim=1, keepdim=True)
+        vsum = torch.zeros(m, 4).index_add_(0, inv, pts)
+        vcnt = torch.zeros(m, 4).index_add_(0, inv, torch.ones_like(pts))
+        f_cluster = pts[:, :3] - (vsum / vcnt)[inv][:, :3]
+        x = torch.cat([pts, dist, f_cluster], 1) * L.pre_scale + L.pre_shift
+        tap("encoder.input", x)
+        pf = []
+        for i, lin in enumerate(W.ffe):
+            x = L.linear(x, lin)
+            pf.append(tap(f"encoder.pf{i}", x))
+        vmax = tap("encoder.voxel_max", scatter_max(x, inv, m))
+        vfeat = tap("encoder.voxel_feats", L.linear(vmax, W.compression))
+
+        # ---- backbone: stem, point stem, fusion stem
+        img = tap("backbone.frustum", frustum2pixel(vfeat, voxel_coors, hw, 1))
+        for c in W.stem:
+            img = L.conv2d(img, c)
+        tap("backbone.stem", img)
+        p = L.linear(torch.cat([pixel2point(img, coors, 1), pf[-1]], 1), W.point_stem)
+        tap("backbone.point_stem", p)
+        seg = tap("backbone.segmax0", scatter_max(p, inv, m))
+        x = L.conv2d(torch.cat([frustum2pixel(seg, voxel_coors, hw, 1), img], 1), W.fusion_stem)
+        tap("backbone.fusion_stem", x)
+        maps, pts_out = [x], [p]
+
+        # ---- four stages: BasicBlocks, point fusion, pixel fusion, attention
+        for st in W.stages:
+            li, s = st.index, st.stride
+            for bi, (c1, c2, ds) in enumerate(st.blocks):
+                y = L.conv2d(L.conv2d(x, c1), c2)
+                idt = L.conv2d(x, ds) if ds is not None else x
+                x = tap(f"backbone.layer{li}.block{bi}", F.hardswish(y + idt))
+            tap(f"backbone.layer{li}", x)
+            p = L.linear(torch.cat([pixel2point(x, coors, s), p], 1), st.point_fusion)
+            tap(f"backbone.point_fusion{li}", p)
+            seg = tap(f"backbone.segmax{li}", scatter_max(p, inv, m))
+            fuse = L.conv2d(torch.cat([frustum2pixel(seg, voxel_coors, hw, s), x], 1), st.pixel_fusion)
+            tap(f"backbone.pixel_fusion{li}", fuse)
+            att = tap(f"backbone.attention{li}", L.conv2d(L.conv2d(fuse, st.attention[0]), st.attention[1]))
+            x = tap(f"backbone.stage{li}", fuse * att + x)
+            maps.append(x)
+            pts_out.append(p)
+
+        # ---- decoder (align-corners bilinear up-sampling to H x W, fuse convs) and the point branch
+        ups = [tap(f"decoder.up{k}", F.interpolate(mp, size=hw, mode="bilinear", align_corners=True))
+               for k, mp in enumerate(maps)]
+        y = tap("decoder.fuse1", L.conv2d(torch.cat(ups, 1), W.fuse_layer1))
+        y = tap("decoder.fuse2", L.conv2d(y, W.fuse_layer2))
+        q = tap("decoder.point_fuse1", L.linear(torch.cat(pts_out, 1), W.point_fuse[0]))
+        q = tap("decoder.point_fuse2", L.linear(q, W.point_fuse[1]))
+
+        # ---- FRHead
+        h = pixel2point(y, coors, 1)
+        for i, (lin, skip) in enumerate(zip(W.head_mlps, (q, pf[2], pf[1], pf[0]))):
+            h = tap(f"head.mlp{i}", L.linear(h, lin) + skip)
+        logits = tap("head.logits", L.linear(h, W.conv_seg))
+        return tap("pred_probs", torch.softmax(logits, -1)).numpy()

code/tt_frnet/reference/ort.py

@@ -0,0 +1,91 @@
+# SPDX-License-Identifier: Apache-2.0
+"""ONNX Runtime golden: the deployed ``frnet_<variant>.onnx`` of AutowareFoundation/lidar_frnet@v2.0 run as-is (CPU
+execution provider; re-runs are bit-identical, S:frnet:332) on the four unpadded network inputs, exactly as
+the node binds them (``points`` float32 (N, 4), ``coors`` int64 (N, 3), ``voxel_coors`` int64 (M, 3),
+``inverse_map`` int64 (N,)).
+
+With ``intermediates=True`` the in-memory graph also exposes the ONNX tensors that hold the same values as the torch
+reference's taps (the mapping is ``FRNetWeights.onnx_taps``, recovered from the graph by ``reference/weights.py``),
+so ``tests/test_reference_cpu.py`` compares module by module. Adding graph outputs changes nothing that is
+computed. Choose which taps to expose with ``tap_patterns`` (fnmatch); OT128 module taps at 146k points are ~1 GB.
+
+onnxruntime is a research / test dependency only (not in the image): it is imported lazily, and the tests skip
+where it is absent.
+"""
+from __future__ import annotations
+
+import fnmatch
+from typing import Any, Dict, Optional, Sequence
+
+import numpy as np
+
+from ..ttaw.golden import NULL_TAPS, TapRegistry
+from .config import variant_files
+from .weights import MODULE_TAPS, FRNetWeights, load_weights
+
+__all__ = ["OrtFRNet", "onnxruntime_available"]
+
+
+def onnxruntime_available() -> bool:
+    import importlib.util
+
+    return importlib.util.find_spec("onnxruntime") is not None and importlib.util.find_spec("onnx") is not None
+
+
+def _session(model_bytes: bytes, threads: int):
+    import onnxruntime as ort
+
+    so = ort.SessionOptions()
+    so.intra_op_num_threads = int(threads)
+    so.inter_op_num_threads = 1
+    so.log_severity_level = 3
+    return ort.InferenceSession(model_bytes, so, providers=["CPUExecutionProvider"])
+
+
+def _with_outputs(path: Any, tensors: Sequence[str]) -> bytes:
+    """The ONNX model at ``path`` with ``tensors`` added as graph outputs (in memory)."""
+    import onnx
+    from onnx import TensorProto, helper
+
+    model = onnx.load(str(path))
+    existing = {o.name for o in model.graph.output}
+    for t in tensors:
+        if t not in existing:
+            model.graph.output.append(helper.make_tensor_value_info(t, TensorProto.FLOAT, None))
+            existing.add(t)
+    return model.SerializeToString()
+
+
+class OrtFRNet:
+    """The deployed graph of one variant under ONNX Runtime (module docstring)."""
+
+    def __init__(self, weights_dir: Any, variant: str, *, threads: int = 2, intermediates: bool = False,
+                 tap_patterns: Sequence[str] = MODULE_TAPS, weights: Optional[FRNetWeights] = None):
+        self.variant = variant
+        self.path = variant_files(weights_dir, variant)["onnx"]
+        self.weights = weights or load_weights(weights_dir, variant)
+        self.taps: Dict[str, str] = {}
+        if intermediates:
+            self.taps = {name: tensor for name, tensor in self.weights.onnx_taps.items()
+                         if name != "pred_probs" and any(fnmatch.fnmatchcase(name, p) for p in tap_patterns)}
+            model_bytes = _with_outputs(self.path, list(self.taps.values()))
+        else:
+            model_bytes = self.path.read_bytes()
+        self.session = _session(model_bytes, threads)
+        self.output_names = [o.name for o in self.session.get_outputs()]
+
+    def __call__(self, *args: Any, **kwargs: Any) -> np.ndarray:
+        return self.forward(*args, **kwargs)
+
+    def forward(self, points: np.ndarray, coors: np.ndarray, voxel_coors: np.ndarray, inverse_map: np.ndarray,
+                taps: TapRegistry = NULL_TAPS) -> np.ndarray:
+        feeds = {"points": np.ascontiguousarray(points, np.float32), "coors": np.ascontiguousarray(coors, np.int64),
+                 "voxel_coors": np.ascontiguousarray(voxel_coors, np.int64),
+                 "inverse_map": np.ascontiguousarray(inverse_map, np.int64)}
+        outs = dict(zip(self.output_names, self.session.run(None, feeds)))
+        for name, tensor in self.taps.items():
+            if taps.wants(name):
+                taps.tap(name, outs[tensor])
+        probs = outs["pred_probs"]
+        taps.tap("pred_probs", probs)
+        return probs

code/tt_frnet/reference/pipeline.py

@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The CPU reference end to end: ``host/`` pre-processing -> network (torch fp32, ONNX Runtime, or the TT dataflow
+of ``rewrites.py``) -> ``host/`` post-processing -> ``FRNetSegmentation``. The same host code feeds the device path
+(``api.py``), so CPU and TT always see identical network inputs.
+
+    ref = FRNetReference(weights_dir, "ot128", backend="torch", threads=4)
+    run = ref.run("samples/synthetic_ot128.npz", calibration={"preset": "synthetic_ot128"})
+    run.output.to_dict()        # the stored smoke reference (samples/<stem>.<variant>.reference.json)
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any, Mapping, Optional, Sequence
+
+import numpy as np
+
+from ..host.frame import FRNetPreprocessor, Prepared
+from ..host.postprocess import FRNetPostprocessor, FRNetSegmentation
+from ..host.preprocess import NetworkInputs, build_plan
+from ..ttaw.golden import NULL_TAPS, TapRegistry
+from .config import FRNetConfig
+from .weights import MODULE_TAPS, FRNetWeights, check_variant_grid, load_weights
+
+__all__ = ["FRNetReference", "ReferenceRun", "BACKENDS"]
+
+BACKENDS = ("torch", "ort", "plan")
+MODEL_NAME = "frnet-p150"
+
+
+@dataclass
+class ReferenceRun:
+    prepared: Prepared
+    pred_probs: Optional[np.ndarray]       # (N, 27) float32, network-input order; None for a skipped frame
+    output: FRNetSegmentation
+
+
+class FRNetReference:
+    """One variant's CPU reference (module docstring). ``backend``: ``torch`` (``model.py``, literal graph),
+    ``ort`` (the deployed ONNX; research / tests only), ``plan`` (``rewrites.py``, the TT dataflow; optionally with
+    an ``emulation``)."""
+
+    def __init__(self, weights_dir: Any, variant: str, *, backend: str = "torch", threads: int = 4,
+                 intermediates: bool = False, tap_patterns: Sequence[str] = MODULE_TAPS,
+                 weights: Optional[FRNetWeights] = None, emulation: Any = None):
+        if backend not in BACKENDS:
+            raise ValueError(f"backend must be one of {BACKENDS}")
+        self.variant = variant
+        self.backend = backend
+        self.config = FRNetConfig.from_model_dir(weights_dir, variant)
+        self.weights = weights or load_weights(weights_dir, variant)
+        check_variant_grid(self.weights, self.config.frustum_h, self.config.frustum_w)
+        pre_norm = (self.weights.pre_norm_scale, self.weights.pre_norm_shift)
+        self.pre = FRNetPreprocessor(self.config, pre_norm=pre_norm, plan=backend == "plan")
+        self.post = FRNetPostprocessor(self.config)
+        if backend == "torch":
+            import torch
+
+            from .model import FRNetTorch
+
+            torch.set_num_threads(int(threads))
+            self.net = FRNetTorch(self.weights)
+        elif backend == "ort":
+            from .ort import OrtFRNet
+
+            self.net = OrtFRNet(weights_dir, variant, threads=threads, intermediates=intermediates,
+                                tap_patterns=tap_patterns, weights=self.weights)
+        else:
+            import torch
+
+            from .rewrites import FRNetPlanTorch
+
+            torch.set_num_threads(int(threads))
+            self.net = FRNetPlanTorch(self.weights, emulation=emulation)
+
+    def network(self, inputs: NetworkInputs, *, plan: Any = None, taps: TapRegistry = NULL_TAPS) -> np.ndarray:
+        """``pred_probs`` (N, 27) of the network inputs (network-input point order)."""
+        if self.backend == "plan":
+            plan = plan if plan is not None else build_plan(
+                inputs, self.config, pre_norm=(self.weights.pre_norm_scale, self.weights.pre_norm_shift))
+            return plan.unsort(self.net(plan, taps))
+        return self.net(inputs.points, inputs.coors, inputs.voxel_coors, inputs.inverse_map, taps=taps)
+
+    def run(self, points: Any, calibration: Optional[Mapping[str, Any]] = None, *,
+            params: Optional[Mapping[str, Any]] = None, taps: TapRegistry = NULL_TAPS,
+            model_name: str = MODEL_NAME) -> ReferenceRun:
+        prep = self.pre(points, calibration)
+        meta = {"variant": self.variant, "reference": self.backend, **prep.stats()}
+        if prep.skipped is not None:
+            out = self.post.empty(model=model_name, frame_id=prep.frame_id, meta=meta)
+            return ReferenceRun(prep, None, out)
+        probs = self.network(prep.inputs, plan=prep.plan, taps=taps)
+        out = self.post(probs, points=prep.inputs.points, num_points_raw=prep.inputs.num_points_raw,
+                        input_index=prep.input_index, params=dict(params or {}), model=model_name,
+                        frame_id=prep.frame_id, meta=meta)
+        return ReferenceRun(prep, probs, out)

code/tt_frnet/reference/rewrites.py

@@ -0,0 +1,275 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The TT device dataflow of FRNet (PORT_LOG.md "trace layout"), run on the CPU in fp32 and proven equal to the
+literal graph (``reference/model.py``) by ``tests/test_reference_rewrites_host.py``. Each rewrite is exact in real
+arithmetic; in float32 they differ only by summation order:
+
+- **points sorted by cell and padded** to ``N_cap`` (``host/preprocess.py`` ``FramePlan``): the network is
+  permutation-equivariant in the points; padded rows form one trailing sentinel segment whose results nothing reads;
+- **host features**: the 8-d decoration and ``pre_norm`` in float64, one rounding (``decorate_points``);
+- **segmented max** (6 sites: ``ScatterElements(max)``) as the exact **log-step** form of PLAN.md 1.3 (the K1
+  kernel's stock-op fallback): round ``r`` takes ``x = max(x, x[idx_r])`` with the host shift table ``idx_r``,
+  then one gather of each segment's last row (``seg_last``) -> ``(M_cap, C)``; :func:`segmax_logstep`;
+- **frustum2pixel** (6 ``ScatterND``) as a gather: ``image[pixel] = table[pix2vox_s[pixel]]`` from the voxel table
+  with one appended **zero row** (index ``M_cap``) for empty pixels -- ``ttaw.ops.gather.scatter_rows`` semantics
+  (C19); duplicates at s > 1 resolved on the host (last voxel wins);
+- **pixel2point** as a gather of NHWC rows (``point2pix_s``);
+- **concat-free convs**: ``conv(concat(a, b))`` = ``conv_a(a) + conv_b(b)`` for the fusion convs (256 = 128 + 128)
+  and the 5-way K-split ``fuse_layer1`` (640 = 5 x 128, probe P14: 8.3 vs 14.2 ms);
+- **concat-free point layers**: ``linear(concat(a, b))`` = ``a @ W_a + b @ W_b + bias`` (point stem, point fusion,
+  ``point_fuse_layer1``), which also lets the two halves keep different dtypes;
+- **align-corners bilinear** as interpolation matrices ``A_h X A_w^T`` (C18 ``interp_matrix``, float64 -> float32).
+
+:class:`Emulation` optionally rounds layer operands the way the device stores them (bf16, or the TF32-like
+mantissa of the device's fp32 matmul, probe P12) -- a first-order CPU model to size the fp32 islands of the
+port before the device runs (``code/scripts/precision_study.py``); it models no math-fidelity phases or tile
+layouts, so its numbers are indicative only.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Dict, List, Optional, Sequence
+
+import numpy as np
+import torch
+import torch.nn.functional as F
+
+from ..host.preprocess import FramePlan, segmax_rounds, segmax_shift_tables
+from ..ttaw.golden import NULL_TAPS, TapRegistry
+from ..ttaw.ops.upsample import interp_matrix
+from .model import TorchLayers, _t
+from .weights import Conv, FRNetWeights, Linear
+
+__all__ = ["Emulation", "round_mode", "segmax_logstep", "FRNetPlanTorch", "plan_taps_to_literal"]
+
+MODES = ("fp32", "bf16", "tf32")
+
+
+def round_mode(x: torch.Tensor, mode: str) -> torch.Tensor:
+    """``fp32``: unchanged; ``bf16``: round to nearest even bfloat16; ``tf32``: truncate to 10 mantissa bits (the
+    probe P12 truncation model of the device's fp32 matmul operands; the device measured 1.4-1.75x its error)."""
+    if mode == "fp32":
+        return x
+    if mode == "bf16":
+        return x.to(torch.bfloat16).to(torch.float32)
+    if mode == "tf32":
+        return (x.contiguous().view(torch.int32) & ~0x1FFF).view(torch.float32)
+    raise ValueError(f"unknown rounding mode {mode!r}; one of {MODES}")
+
+
+@dataclass(frozen=True)
+class Emulation:
+    """Operand rounding per site (``fp32`` = exact float32). ``conv_act`` / ``conv_weight``: inputs and weights
+    (+ bias) of the 53 convs; ``point_act`` / ``point_weight``: inputs and weights of the 17 point / voxel linears;
+    ``gather``: rows produced by the embedding gathers (frustum2pixel, pixel2point: ``ttnn.embedding`` tables are
+    bf16-only, so ``bf16`` is the stock-op device); ``upsample``: the decoder's interpolation operands. ``overrides``:
+    per-layer modes by layer name (e.g. ``{"head.logits": ("tf32", "fp32")}``: (act, weight))."""
+
+    conv_act: str = "fp32"
+    conv_weight: str = "fp32"
+    point_act: str = "fp32"
+    point_weight: str = "fp32"
+    gather: str = "fp32"
+    upsample: str = "fp32"
+    overrides: Dict[str, tuple] = field(default_factory=dict)
+
+    @classmethod
+    def parse(cls, spec: str) -> "Emulation":
+        """``"conv_act=bf16,point_act=tf32,gather=bf16"`` (unset sites stay fp32)."""
+        kw = {}
+        for part in filter(None, (p.strip() for p in spec.split(","))):
+            k, _, v = part.partition("=")
+            if k not in cls.__dataclass_fields__ or k == "overrides" or v not in MODES:
+                raise ValueError(f"bad emulation item {part!r}")
+            kw[k] = v
+        return cls(**kw)
+
+    @property
+    def exact(self) -> bool:
+        return all(getattr(self, k) == "fp32" for k in ("conv_act", "conv_weight", "point_act", "point_weight",
+                                                       "gather", "upsample")) and not self.overrides
+
+    def label(self) -> str:
+        items = [f"{k}={getattr(self, k)}" for k in ("conv_act", "conv_weight", "point_act", "point_weight",
+                                                      "gather", "upsample") if getattr(self, k) != "fp32"]
+        items += [f"{k}:{v}" for k, v in sorted(self.overrides.items())]
+        return ",".join(items) or "fp32"
+
+
+def segmax_logstep(x: torch.Tensor, plan: FramePlan, rounds: Optional[int] = None) -> torch.Tensor:
+    """Per-voxel max of the sorted, padded rows ``x`` ``(N_cap, C)`` -> ``(M_cap, C)``: ``rounds`` (default: just
+    enough for the frame's longest segment) log-step rounds over the host shift tables, then a gather of every
+    segment's last row (PLAN.md 1.3). Exact (max is order-independent)."""
+    rounds = segmax_rounds(plan.max_segment_length) if rounds is None else int(rounds)
+    for idx in segmax_shift_tables(plan.seg_id, rounds):
+        x = torch.maximum(x, x[_t(idx.astype(np.int64))])
+    return x[_t(plan.seg_last.astype(np.int64))]
+
+
+def _with_zero_row(table: torch.Tensor) -> torch.Tensor:
+    return torch.cat([table, table.new_zeros(1, table.shape[1])], 0)
+
+
+def _nhwc(x: torch.Tensor) -> torch.Tensor:
+    _, c, h, w = x.shape
+    return x.permute(0, 2, 3, 1).reshape(h * w, c)
+
+
+def _nchw(rows: torch.Tensor, h: int, w: int) -> torch.Tensor:
+    return rows.reshape(1, h, w, -1).permute(0, 3, 1, 2).contiguous()
+
+
+class FRNetPlanTorch:
+    """``FRNetPlanTorch(weights)(plan, taps)`` -> ``pred_probs`` of the sorted, padded rows ``(N_cap, 27)``
+    (``plan.unsort`` gives the network-input order). Taps use the names of ``reference/model.py``; point taps are in
+    sorted padded order, voxel taps have ``M_cap`` rows (:func:`plan_taps_to_literal` aligns them)."""
+
+    def __init__(self, weights: FRNetWeights, *, emulation: Optional[Emulation] = None,
+                 rounds: Optional[int] = None):
+        self.weights = weights
+        self.layers = TorchLayers(weights)
+        self.emu = emulation or Emulation()
+        self.rounds = rounds
+        H, W = weights.frustum_hw
+        self._interp = {}
+        for st in [1] + [s.stride for s in weights.stages]:
+            ah = interp_matrix(H // st, H, mode="linear", align_corners=True)
+            aw = interp_matrix(W // st, W, mode="linear", align_corners=True)
+            self._interp[st] = (_t(ah.astype(np.float32)), _t(aw.astype(np.float32)))
+        # K-split weights of the concat-consuming layers
+        self._split_lin = {}
+        for lin, parts in ((weights.point_stem, (128, 256)), (weights.point_fuse[0], (128,) * 5),
+                           *((st.point_fusion, (128, 128)) for st in weights.stages)):
+            w_t, b_t, _ = self.layers.lin[lin.name]
+            if w_t.shape[0] != sum(parts):
+                raise ValueError(f"{lin.name}: {w_t.shape[0]} inputs, expected {sum(parts)}")
+            self._split_lin[lin.name] = (list(torch.split(w_t, list(parts), 0)), b_t)
+        self._split_conv = {}
+        for conv, parts in ((weights.fusion_stem, (128, 128)), (weights.fuse_layer1, (128,) * 5),
+                            *((st.pixel_fusion, (128, 128)) for st in weights.stages)):
+            w_t, b_t, stride, pad, act = self.layers.conv[conv.name]
+            if w_t.shape[1] != sum(parts):
+                raise ValueError(f"{conv.name}: {w_t.shape[1]} inputs, expected {sum(parts)}")
+            self._split_conv[conv.name] = (list(torch.split(w_t, list(parts), 1)), b_t, stride, pad, act)
+
+    # ---- rounded layer applications ---------------------------------------------------------------------------
+    def _modes(self, name: str, act: str, weight: str) -> tuple:
+        return self.emu.overrides.get(name, (act, weight))
+
+    def linear(self, parts: Sequence[torch.Tensor], layer: Linear) -> torch.Tensor:
+        act_m, w_m = self._modes(layer.name, self.emu.point_act, self.emu.point_weight)
+        if layer.name in self._split_lin:
+            ws, b = self._split_lin[layer.name]
+        else:
+            w, b, _ = self.layers.lin[layer.name]
+            ws = [w]
+        if len(parts) != len(ws):
+            raise ValueError(f"{layer.name}: {len(parts)} input parts for {len(ws)} weight parts")
+        y = round_mode(b, w_m).expand(parts[0].shape[0], -1).clone()
+        for x, w in zip(parts, ws):
+            y = y + round_mode(x, act_m) @ round_mode(w, w_m)
+        return F.relu(y) if layer.relu else y
+
+    def conv(self, parts: Sequence[torch.Tensor], layer: Conv) -> torch.Tensor:
+        act_m, w_m = self._modes(layer.name, self.emu.conv_act, self.emu.conv_weight)
+        if layer.name in self._split_conv:
+            ws, b, stride, pad, act = self._split_conv[layer.name]
+        else:
+            w, b, stride, pad, act = self.layers.conv[layer.name]
+            ws = [w]
+        if len(parts) != len(ws):
+            raise ValueError(f"{layer.name}: {len(parts)} input parts for {len(ws)} weight parts")
+        y = None
+        for i, (x, w) in enumerate(zip(parts, ws)):
+            part = F.conv2d(round_mode(x, act_m), round_mode(w, w_m), round_mode(b, w_m) if i == 0 else None,
+                            stride=stride, padding=pad)
+            y = part if y is None else y + part
+        if act == "hardswish":
+            y = F.hardswish(y)
+        elif act == "sigmoid":
+            y = torch.sigmoid(y)
+        return y
+
+    def gather(self, table: torch.Tensor, index: np.ndarray) -> torch.Tensor:
+        return round_mode(table, self.emu.gather)[_t(index.astype(np.int64))]
+
+    def upsample(self, x: torch.Tensor, stride: int) -> torch.Tensor:
+        if stride == 1:
+            return x
+        ah, aw = self._interp[stride]
+        m = self.emu.upsample
+        x = round_mode(x, m)
+        return torch.einsum("hi,bcij,wj->bchw", round_mode(ah, m), x, round_mode(aw, m))
+
+    # ---- the device graph -------------------------------------------------------------------------------------
+    def __call__(self, plan: FramePlan, taps: TapRegistry = NULL_TAPS) -> np.ndarray:
+        return self.forward(plan, taps)
+
+    @torch.no_grad()
+    def forward(self, plan: FramePlan, taps: TapRegistry = NULL_TAPS) -> np.ndarray:
+        Wt = self.weights
+        H, W = plan.frustum_hw
+        if (H, W) != Wt.frustum_hw:
+            raise ValueError(f"plan grid {plan.frustum_hw} != weights grid {Wt.frustum_hw}")
+
+        def tap(name: str, value: torch.Tensor) -> torch.Tensor:
+            if taps.wants(name):
+                taps.tap(name, value.numpy())
+            return value
+
+        def to_image(table: torch.Tensor, s: int) -> torch.Tensor:
+            return _nchw(self.gather(_with_zero_row(table), plan.pix2vox[s]), H // s, W // s)
+
+        def to_points(img: torch.Tensor, s: int) -> torch.Tensor:
+            return self.gather(_nhwc(img), plan.point2pix[s])
+
+        x = tap("encoder.input", _t(plan.features))
+        pf: List[torch.Tensor] = []
+        for i, lin in enumerate(Wt.ffe):
+            x = self.linear([x], lin)
+            pf.append(tap(f"encoder.pf{i}", x))
+        vmax = tap("encoder.voxel_max", segmax_logstep(x, plan, self.rounds))
+        vfeat = tap("encoder.voxel_feats", self.linear([vmax], Wt.compression))
+        img = tap("backbone.frustum", to_image(vfeat, 1))
+        for c in Wt.stem:
+            img = self.conv([img], c)
+        tap("backbone.stem", img)
+        p = tap("backbone.point_stem", self.linear([to_points(img, 1), pf[-1]], Wt.point_stem))
+        seg = tap("backbone.segmax0", segmax_logstep(p, plan, self.rounds))
+        x = tap("backbone.fusion_stem", self.conv([to_image(seg, 1), img], Wt.fusion_stem))
+        maps, pts_out = [x], [p]
+        for st in Wt.stages:
+            li, s = st.index, st.stride
+            for bi, (c1, c2, ds) in enumerate(st.blocks):
+                y = self.conv([self.conv([x], c1)], c2)
+                idt = self.conv([x], ds) if ds is not None else x
+                x = tap(f"backbone.layer{li}.block{bi}", F.hardswish(y + idt))
+            tap(f"backbone.layer{li}", x)
+            p = tap(f"backbone.point_fusion{li}", self.linear([to_points(x, s), p], st.point_fusion))
+            seg = tap(f"backbone.segmax{li}", segmax_logstep(p, plan, self.rounds))
+            fuse = tap(f"backbone.pixel_fusion{li}", self.conv([to_image(seg, s), x], st.pixel_fusion))
+            att = tap(f"backbone.attention{li}", self.conv([self.conv([fuse], st.attention[0])], st.attention[1]))
+            x = tap(f"backbone.stage{li}", fuse * att + x)
+            maps.append(x)
+            pts_out.append(p)
+        strides = [1] + [st.stride for st in Wt.stages]
+        ups = [tap(f"decoder.up{k}", self.upsample(mp, s)) for k, (mp, s) in enumerate(zip(maps, strides))]
+        y = tap("decoder.fuse1", self.conv(ups, Wt.fuse_layer1))
+        y = tap("decoder.fuse2", self.conv([y], Wt.fuse_layer2))
+        q = tap("decoder.point_fuse1", self.linear(pts_out, Wt.point_fuse[0]))
+        q = tap("decoder.point_fuse2", self.linear([q], Wt.point_fuse[1]))
+        h = to_points(y, 1)
+        for i, (lin, skip) in enumerate(zip(Wt.head_mlps, (q, pf[2], pf[1], pf[0]))):
+            h = tap(f"head.mlp{i}", self.linear([h], lin) + skip)
+        logits = tap("head.logits", self.linear([h], Wt.conv_seg))
+        return tap("pred_probs", torch.softmax(logits, -1)).numpy()
+
+
+def plan_taps_to_literal(name: str, value: np.ndarray, plan: FramePlan) -> np.ndarray:
+    """A tap of :class:`FRNetPlanTorch` in the layout of ``reference/model.py``: point tensors (``N_cap`` rows)
+    unsorted to the network-input order, voxel tensors (``M_cap`` rows) cut to ``M``, maps unchanged."""
+    v = np.asarray(value)
+    if v.ndim == 2 and v.shape[0] == plan.n_cap:
+        return plan.unsort(v)
+    if v.ndim == 2 and v.shape[0] == plan.m_cap:
+        return v[: plan.m]
+    return v

code/tt_frnet/reference/weights.py

@@ -0,0 +1,437 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The deployed ``frnet_<variant>.onnx`` of AutowareFoundation/lidar_frnet@v2.0 read as DATA, the way the TT port
+reads it: the vendored ``ttaw.weights.OnnxWeights`` (initializers addressed by the node that consumes them; nothing
+is executed). 120 of the 186 initializer *names* differ between the OT128 and QT128 exports (``onnx::Conv_1487`` vs
+``onnx::Conv_1485``, S:frnet:98-101), node names do not, so every weight is found from its node.
+
+Where BatchNorm is folded (float64, one final rounding to float32; ``ttaw.weights.fold_bn_linear``):
+
+- every point layer ``MatMul (no bias) -> BatchNormalization(eps 1e-3) -> Relu`` (``ffe_layers.0-2``,
+  ``point_stem``, ``point_fusion_layers.0-3``, ``point_fuse_layer1-2``, ``mlps.0-3``) becomes one
+  ``y = relu(x @ W' + b')`` -- exactly the port's ``ttnn.linear(..., bias, activation="relu")``;
+- ``pre_norm`` (BatchNorm1d over the 8 decorated features) is kept as ``(scale, shift)`` float32 (``x * scale +
+  shift``): the port applies it on the host with the feature decoration (PLAN.md 2.8), so the first device layer is
+  a plain linear;
+- the 53 convs already carry the exporter's BN fold (Conv with bias, S:frnet:245); the three Gemms
+  (``ffe_layers.3``, ``compression_layers.0``, ``conv_seg``) have their own bias.
+
+The structure is recovered from the graph, never re-typed: stage count, BasicBlocks per stage, strides, the
+frustum grid of every frustum2pixel (the ``ConstantOfShape`` shape of its ``ScatterND``), every activation (the
+conv's consumer), and every wiring assumption of the reference model (concat orders, skip connections, the
+decoder's resize inputs, the head's residual skips, the Softmax axis) is asserted on the graph
+(:func:`load_weights` raises ``ValueError`` when one does not hold). Each layer records the ONNX tensor that holds
+its output, so ``reference/ort.py`` exposes the same taps from ONNX Runtime.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Sequence, Tuple
+
+import numpy as np
+
+from ..ttaw.weights import BatchNorm, OnnxWeights, file_sha256, fold_bn_linear
+from .config import BLOCKS, NUM_CLASSES, ONNX_SHA256, STRIDES, variant_files
+
+__all__ = ["Linear", "Conv", "Stage", "FRNetWeights", "load_weights", "check_variant_grid", "MODULE_TAPS",
+           "OPT_IN_TAPS"]
+
+VE = "/model/voxel_encoder"
+BB = "/model/backbone"
+
+
+@dataclass(frozen=True)
+class Linear:
+    """``y = act(x @ weight + bias)``; ``weight`` (in, out) float32 (BN folded where the graph has one)."""
+
+    name: str
+    weight: np.ndarray
+    bias: np.ndarray
+    relu: bool
+    onnx_node: str
+    onnx_output: str                 # ONNX tensor equal to this layer's output (after the ReLU when there is one)
+    bn_node: Optional[str] = None
+
+    @property
+    def in_features(self) -> int:
+        return int(self.weight.shape[0])
+
+    @property
+    def out_features(self) -> int:
+        return int(self.weight.shape[1])
+
+
+@dataclass(frozen=True)
+class Conv:
+    """``y = act(conv2d(x, weight, bias, stride, padding))`` (NCHW, BN already folded by the exporter)."""
+
+    name: str
+    weight: np.ndarray               # (Cout, Cin, k, k) float32
+    bias: np.ndarray                 # (Cout,) float32
+    stride: int
+    padding: int
+    activation: Optional[str]        # "hardswish" | "sigmoid" | None (a residual add follows)
+    onnx_node: str
+    onnx_output: str                 # ONNX tensor equal to the output (after the activation when there is one)
+
+    @property
+    def kernel(self) -> int:
+        return int(self.weight.shape[-1])
+
+    @property
+    def in_channels(self) -> int:
+        return int(self.weight.shape[1])
+
+    @property
+    def out_channels(self) -> int:
+        return int(self.weight.shape[0])
+
+
+@dataclass(frozen=True)
+class Stage:
+    """One backbone stage ``layer<index>`` (1-based) plus its point / pixel fusion (``*_layers.<index - 1>``)."""
+
+    index: int
+    stride: int                      # cumulative stride of the stage output (1, 2, 4, 8)
+    blocks: Tuple[Tuple[Conv, Conv, Optional[Conv]], ...]   # (conv1 + HardSwish, conv2, downsample) per BasicBlock
+    block_outputs: Tuple[str, ...]   # ONNX tensor of each block output (after add + HardSwish)
+    point_fusion: Linear             # [pixel2point(x_s), p] 256 -> 128 + BN + ReLU
+    pixel_fusion: Conv               # [frustum2pixel(segmax(p)), x_s] 256 -> 128 + HardSwish
+    attention: Tuple[Conv, Conv]     # 128 -> 128 + HardSwish, 128 -> 128 + Sigmoid
+    segmax_output: str               # ONNX tensor: per-voxel max of point_fusion's output (M, 128)
+    output: str                      # ONNX tensor: x = pixel_fusion * attention + x_s
+
+
+@dataclass(frozen=True)
+class FRNetWeights:
+    variant: str
+    frustum_hw: Tuple[int, int]
+    strides: Tuple[int, ...]
+    pre_norm: BatchNorm
+    pre_norm_scale: np.ndarray       # (8,) float32: pre_norm(x) == x * scale + shift
+    pre_norm_shift: np.ndarray
+    ffe: Tuple[Linear, ...]          # 8 -> 64 -> 128 -> 256 (ReLU, BN folded), 256 -> 256 (Gemm, no activation)
+    compression: Linear              # 256 -> 16 + ReLU (on the per-voxel max)
+    stem: Tuple[Conv, ...]           # 16 -> 64 -> 128 -> 128, each + HardSwish
+    point_stem: Linear               # [pixel2point(stem), pf3] 384 -> 128 + BN + ReLU
+    point_stem_segmax: str           # ONNX tensor of the per-voxel max of point_stem (M, 128)
+    fusion_stem: Conv                # [frustum2pixel(segmax(p0)), stem] 256 -> 128 + HardSwish
+    stages: Tuple[Stage, ...]
+    fuse_layer1: Conv                # concat of the 5 up-sampled maps 640 -> 256 + HardSwish
+    fuse_layer2: Conv                # 256 -> 128 + HardSwish
+    point_fuse: Tuple[Linear, Linear]   # 640 -> 256, 256 -> 128 (+ BN + ReLU)
+    head_mlps: Tuple[Linear, ...]    # 128 -> 128, 128 -> 256, 256 -> 128, 128 -> 64 (+ BN + ReLU), then + skip
+    conv_seg: Linear                 # 64 -> 27
+    onnx_taps: Dict[str, str]        # tap name -> ONNX tensor (module and opt-in taps)
+    onnx_path: str = ""
+    onnx_sha256: str = ""
+    raw_parameters: int = 0          # learned values in the ONNX initializers
+    folded_parameters: int = field(default=0)   # values after the BN folds (what the device holds)
+
+    @property
+    def convs(self) -> List[Conv]:
+        out = list(self.stem) + [self.fusion_stem]
+        for st in self.stages:
+            for c1, c2, ds in st.blocks:
+                out += [c1, c2] + ([ds] if ds is not None else [])
+            out += [st.pixel_fusion, *st.attention]
+        return out + [self.fuse_layer1, self.fuse_layer2]
+
+    @property
+    def linears(self) -> List[Linear]:
+        return [*self.ffe, self.compression, self.point_stem, *(s.point_fusion for s in self.stages),
+                *self.point_fuse, *self.head_mlps, self.conv_seg]
+
+    @property
+    def num_classes(self) -> int:
+        return self.conv_seg.out_features
+
+
+# Module taps of the reference (tests/test_reference_cpu.py compares every one with ORT; the TT port's per-module
+# gates use the same names). 1-based stage numbers: stage l = ONNX layer<l> + *_layers.<l - 1>.
+MODULE_TAPS: Tuple[str, ...] = (
+    "encoder.input", "encoder.pf0", "encoder.pf1", "encoder.pf2", "encoder.pf3", "encoder.voxel_max",
+    "encoder.voxel_feats", "backbone.frustum", "backbone.stem", "backbone.point_stem", "backbone.fusion_stem",
+    *(f"backbone.{kind}{i}" for i in range(1, 5)
+      for kind in ("layer", "point_fusion", "pixel_fusion", "attention", "stage")),
+    "decoder.fuse1", "decoder.fuse2", "decoder.point_fuse1", "decoder.point_fuse2",
+    *(f"head.mlp{i}" for i in range(4)), "head.logits", "pred_probs",
+)
+# Opt-in taps (TapRegistry include patterns): every BasicBlock output, the per-voxel max of every point branch
+# (the K1 / segmented-max sites), the five up-sampled decoder inputs.
+OPT_IN_TAPS: Tuple[str, ...] = (
+    "backbone.layer*.block*", "backbone.segmax*", "decoder.up*",
+)
+
+
+def _fail(msg: str) -> None:
+    raise ValueError(f"frnet ONNX structure: {msg}")
+
+
+def _single(w: OnnxWeights, tensor: str, op_type: str) -> Any:
+    try:
+        return w.consumer_of(tensor, op_type)
+    except KeyError as e:
+        _fail(str(e))
+
+
+def _point_layer(w: OnnxWeights, matmul: str, name: str) -> Linear:
+    mm = w.node(matmul)
+    if mm.op_type != "MatMul" or not w.has(mm.inputs[1]) or w.has(mm.inputs[0]):
+        _fail(f"{matmul} is not x @ W with a constant W")
+    bn = _single(w, mm.outputs[0], "BatchNormalization")
+    relu = _single(w, bn.outputs[0], "Relu")
+    wf, bf = fold_bn_linear(w.matmul_weight(matmul), None, w.batchnorm(bn.name), layout="in_out")
+    return Linear(name, wf, bf, True, mm.name, relu.outputs[0], bn.name)
+
+
+def _gemm(w: OnnxWeights, node: str, name: str) -> Linear:
+    g = w.gemm(node)
+    if g.trans_a or g.alpha != 1.0 or g.beta != 1.0 or g.bias is None:
+        _fail(f"{node}: unexpected Gemm attributes")
+    weight = np.asarray(g.weight, np.float32)
+    weight = weight.T if g.trans_b else weight
+    out = w.node(node).outputs[0]
+    relus = [c for c in w.consumers(out) if c.op_type == "Relu"]
+    relu = len(relus) == 1 and len(w.consumers(out)) == 1
+    return Linear(name, np.ascontiguousarray(weight), np.asarray(g.bias, np.float32), relu, node,
+                  relus[0].outputs[0] if relu else out)
+
+
+def _conv(w: OnnxWeights, node: str, name: str) -> Conv:
+    c = w.conv(node)
+    k = c.kernel_shape
+    if (c.group != 1 or set(c.dilations) != {1} or len(k) != 2 or k[0] != k[1] or len(set(c.strides)) != 1
+            or len(set(c.pads)) != 1 or c.bias is None or c.auto_pad not in ("NOTSET", "")):
+        _fail(f"{node}: unexpected Conv attributes {c.kernel_shape} {c.strides} {c.pads} {c.dilations} {c.group}")
+    out = w.node(node).outputs[0]
+    cons = w.consumers(out)
+    act, tensor = None, out
+    if len(cons) == 1 and cons[0].op_type in ("HardSwish", "Sigmoid"):
+        act, tensor = cons[0].op_type.lower(), cons[0].outputs[0]
+    return Conv(name, np.asarray(c.weight, np.float32), np.asarray(c.bias, np.float32), int(c.strides[0]),
+                int(c.pads[0]), act, node, tensor)
+
+
+def _producer_op(w: OnnxWeights, tensor: str) -> str:
+    p = w.producer(tensor)
+    return p.op_type if p is not None else ""
+
+
+def _scatter_max_of(w: OnnxWeights, tensor: str) -> str:
+    found = [c for c in w.consumers(tensor) if c.op_type == "ScatterElements" and c.attrs.get("reduction") == "max"]
+    if len(found) != 1:
+        _fail(f"{tensor} does not feed exactly one ScatterElements(max)")
+    return found[0].outputs[0]
+
+
+def load_weights(model_dir: Any, variant: str, *, check_sha256: bool = True) -> FRNetWeights:
+    """Parse ``frnet_<variant>.onnx`` of a weights directory (module docstring)."""
+    path = variant_files(model_dir, variant)["onnx"]
+    digest = file_sha256(path)
+    if check_sha256 and digest != ONNX_SHA256[variant]:     # refused before it is parsed
+        raise ValueError(f"{path.name}: sha256 {digest} is not the pinned v2.0 file {ONNX_SHA256[variant]}")
+    w = OnnxWeights(path)
+    if w.input_names != ("points", "coors", "voxel_coors", "inverse_map") or w.output_names != ("pred_probs",):
+        _fail(f"inputs {w.input_names} / outputs {w.output_names}")
+    taps: Dict[str, str] = {}
+
+    # ---- frustum grids of the six frustum2pixel (ScatterND) sites, in graph order
+    grids = []
+    for sc in w.nodes("ScatterND"):
+        shape_node = w.producer(w.producer(sc.inputs[0]).inputs[0])
+        dims = [w.array(t).tolist() for t in shape_node.inputs[:3] if w.has(t)]
+        if len(dims) != 3 or dims[0] != [1]:
+            _fail(f"{sc.name}: cannot read its (1, H, W, C) image shape")
+        grids.append((int(dims[1][0]), int(dims[2][0])))
+    if len(grids) != 2 + len(STRIDES):
+        _fail(f"{len(grids)} ScatterND sites, expected {2 + len(STRIDES)}")
+    H, W = grids[0]
+    strides = tuple(H // h for h, _ in grids[2:])
+    if grids[1] != (H, W) or strides != STRIDES or any(W // s != g[1] or H // s != g[0]
+                                                       for s, g in zip(strides, grids[2:])):
+        _fail(f"frustum grids {grids} do not follow strides {STRIDES}")
+
+    # ---- encoder
+    bn_pre = w.find_node(f"{VE}/pre_norm/*", "BatchNormalization")
+    pre = w.batchnorm(bn_pre.name)
+    taps["encoder.input"] = bn_pre.outputs[0]
+    ffe = [_point_layer(w, f"{VE}/ffe_layers.{i}/ffe_layers.{i}.0/MatMul", f"encoder.ffe{i}") for i in range(3)]
+    for i in range(1, 3):
+        if w.node(ffe[i].onnx_node).inputs[0] != ffe[i - 1].onnx_output:
+            _fail(f"ffe_layers.{i} does not consume ffe_layers.{i - 1}")
+    if w.node(ffe[0].onnx_node).inputs[0] != bn_pre.outputs[0]:
+        _fail("ffe_layers.0 does not consume pre_norm")
+    ffe3 = _gemm(w, f"{VE}/ffe_layers.3/Gemm", "encoder.ffe3")
+    if ffe3.relu or w.node(ffe3.onnx_node).inputs[0] != ffe[2].onnx_output:
+        _fail("ffe_layers.3 is not a plain Gemm on ffe_layers.2")
+    ffe.append(ffe3)
+    for i, lin in enumerate(ffe):
+        taps[f"encoder.pf{i}"] = lin.onnx_output
+    taps["encoder.voxel_max"] = _scatter_max_of(w, ffe3.onnx_output)
+    compression = _gemm(w, f"{VE}/compression_layers/compression_layers.0/Gemm", "encoder.compression")
+    if not compression.relu or w.node(compression.onnx_node).inputs[0] != taps["encoder.voxel_max"]:
+        _fail("compression_layers is not Gemm + ReLU on the per-voxel max")
+    taps["encoder.voxel_feats"] = compression.onnx_output
+
+    # ---- stem and point stem
+    stem = tuple(_conv(w, f"{BB}/stem/stem.{i}/Conv", f"backbone.stem{j}") for j, i in enumerate((0, 3, 6)))
+    if any(c.activation != "hardswish" or c.stride != 1 for c in stem):
+        _fail("stem convs must be stride 1 + HardSwish")
+    taps["backbone.frustum"] = w.node(stem[0].onnx_node).inputs[0]
+    if _producer_op(w, taps["backbone.frustum"]) != "Transpose":
+        _fail("the stem does not read the transposed frustum image")
+    taps["backbone.stem"] = stem[-1].onnx_output
+    point_stem = _point_layer(w, f"{BB}/point_stem/point_stem.0/MatMul", "backbone.point_stem")
+    cat = w.producer(w.node(point_stem.onnx_node).inputs[0])
+    if cat.op_type != "Concat" or len(cat.inputs) != 2 or cat.inputs[1] != ffe3.onnx_output:
+        _fail("point_stem input is not concat[pixel2point(stem), ffe_layers.3]")
+    taps["backbone.point_stem"] = point_stem.onnx_output
+    point_stem_segmax = _scatter_max_of(w, point_stem.onnx_output)
+    taps["backbone.segmax0"] = point_stem_segmax
+    fusion_stem = _conv(w, f"{BB}/fusion_stem/fusion_stem.0/Conv", "backbone.fusion_stem")
+    cat = w.producer(w.node(fusion_stem.onnx_node).inputs[0])
+    if fusion_stem.activation != "hardswish" or cat.op_type != "Concat" or cat.inputs[1] != stem[-1].onnx_output:
+        _fail("fusion_stem is not HardSwish(conv(concat[frustum2pixel(p0), stem]))")
+    taps["backbone.fusion_stem"] = fusion_stem.onnx_output
+
+    # ---- stages
+    stages: List[Stage] = []
+    x_in, p_prev = fusion_stem.onnx_output, point_stem.onnx_output
+    for li in range(1, 5):
+        conv1s = w.nodes("Conv", f"{BB}/layer{li}/layer{li}.*/conv1/Conv")
+        if len(conv1s) != BLOCKS[li - 1]:
+            _fail(f"layer{li} has {len(conv1s)} BasicBlocks, expected {BLOCKS[li - 1]}")
+        blocks, outs = [], []
+        for bi in range(len(conv1s)):
+            pre_b = f"{BB}/layer{li}/layer{li}.{bi}"
+            c1 = _conv(w, f"{pre_b}/conv1/Conv", f"backbone.layer{li}.block{bi}.conv1")
+            c2 = _conv(w, f"{pre_b}/conv2/Conv", f"backbone.layer{li}.block{bi}.conv2")
+            ds = None
+            if w.nodes("Conv", f"{pre_b}/downsample/*"):
+                ds = _conv(w, f"{pre_b}/downsample/downsample.0/Conv", f"backbone.layer{li}.block{bi}.downsample")
+            if c1.activation != "hardswish" or c2.activation is not None or (ds is not None and ds.activation):
+                _fail(f"{pre_b}: unexpected activations")
+            if w.node(c1.onnx_node).inputs[0] != x_in or (ds is not None and w.node(ds.onnx_node).inputs[0] != x_in):
+                _fail(f"{pre_b} does not read the previous block output")
+            add = _single(w, c2.onnx_output, "Add")
+            idt = ds.onnx_output if ds is not None else x_in
+            if set(add.inputs) != {c2.onnx_output, idt}:
+                _fail(f"{pre_b}: the residual add does not take conv2 + identity")
+            hs = _single(w, add.outputs[0], "HardSwish")
+            blocks.append((c1, c2, ds))
+            outs.append(hs.outputs[0])
+            taps[f"backbone.layer{li}.block{bi}"] = hs.outputs[0]
+            x_in = hs.outputs[0]
+        x_s = outs[-1]
+        taps[f"backbone.layer{li}"] = x_s
+        pf = _point_layer(w, f"{BB}/point_fusion_layers.{li - 1}/point_fusion_layers.{li - 1}.0/MatMul",
+                          f"backbone.point_fusion{li}")
+        cat = w.producer(w.node(pf.onnx_node).inputs[0])
+        if cat.op_type != "Concat" or len(cat.inputs) != 2 or cat.inputs[1] != p_prev:
+            _fail(f"point_fusion_layers.{li - 1} input is not concat[pixel2point(x), p]")
+        seg = _scatter_max_of(w, pf.onnx_output)
+        taps[f"backbone.point_fusion{li}"] = pf.onnx_output
+        taps[f"backbone.segmax{li}"] = seg
+        pix = _conv(w, f"{BB}/pixel_fusion_layers.{li - 1}/pixel_fusion_layers.{li - 1}.0/Conv",
+                    f"backbone.pixel_fusion{li}")
+        cat = w.producer(w.node(pix.onnx_node).inputs[0])
+        if pix.activation != "hardswish" or cat.op_type != "Concat" or cat.inputs[1] != x_s:
+            _fail(f"pixel_fusion_layers.{li - 1} is not HardSwish(conv(concat[frustum2pixel(p), x]))")
+        a0 = _conv(w, f"{BB}/attention_layers.{li - 1}/attention_layers.{li - 1}.0/Conv",
+                   f"backbone.attention{li}.conv0")
+        a1 = _conv(w, f"{BB}/attention_layers.{li - 1}/attention_layers.{li - 1}.3/Conv",
+                   f"backbone.attention{li}.conv1")
+        if (a0.activation != "hardswish" or a1.activation != "sigmoid" or w.node(a0.onnx_node).inputs[0] !=
+                pix.onnx_output or w.node(a1.onnx_node).inputs[0] != a0.onnx_output):
+            _fail(f"attention_layers.{li - 1} is not sigmoid(conv(hswish(conv(pixel_fusion))))")
+        mul = _single(w, a1.onnx_output, "Mul")
+        if set(mul.inputs) != {pix.onnx_output, a1.onnx_output}:
+            _fail(f"stage {li}: the attention product is not pixel_fusion * attention")
+        add = _single(w, mul.outputs[0], "Add")
+        if set(add.inputs) != {mul.outputs[0], x_s}:
+            _fail(f"stage {li}: the output is not pixel_fusion * attention + x")
+        taps[f"backbone.pixel_fusion{li}"] = pix.onnx_output
+        taps[f"backbone.attention{li}"] = a1.onnx_output
+        taps[f"backbone.stage{li}"] = add.outputs[0]
+        stride = STRIDES[li - 1]
+        stages.append(Stage(li, stride, tuple(blocks), tuple(outs), pf, pix, (a0, a1), seg, add.outputs[0]))
+        x_in, p_prev = add.outputs[0], pf.onnx_output
+    got = [1] + [st.blocks[0][0].stride for st in stages[1:]]
+    if tuple(int(np.prod(got[: i + 1])) for i in range(4)) != STRIDES:
+        _fail(f"stage strides {got} do not give {STRIDES}")
+
+    # ---- decoder
+    fuse1 = _conv(w, f"{BB}/fuse_layer1/conv/Conv", "decoder.fuse1")
+    fuse2 = _conv(w, f"{BB}/fuse_layer2/conv/Conv", "decoder.fuse2")
+    cat = w.producer(w.node(fuse1.onnx_node).inputs[0])
+    sources = [fusion_stem.onnx_output] + [st.output for st in stages]
+    if cat.op_type != "Concat" or len(cat.inputs) != 5:
+        _fail("fuse_layer1 does not read a 5-way concat")
+    for k, (t, src) in enumerate(zip(cat.inputs, sources)):
+        rs = w.producer(t)
+        if rs.op_type != "Resize" or rs.inputs[0] != src or rs.attrs.get("mode") != "linear" or \
+                rs.attrs.get("coordinate_transformation_mode") != "align_corners":
+            _fail(f"decoder input {k} is not an align_corners bilinear Resize of {src}")
+        taps[f"decoder.up{k}"] = t
+    if fuse1.activation != "hardswish" or fuse2.activation != "hardswish" or \
+            w.node(fuse2.onnx_node).inputs[0] != fuse1.onnx_output:
+        _fail("fuse_layer1 / fuse_layer2 are not HardSwish convs in sequence")
+    taps["decoder.fuse1"] = fuse1.onnx_output
+    taps["decoder.fuse2"] = fuse2.onnx_output
+    pf1 = _point_layer(w, f"{BB}/point_fuse_layer1/point_fuse_layer1.0/MatMul", "decoder.point_fuse1")
+    pf2 = _point_layer(w, f"{BB}/point_fuse_layer2/point_fuse_layer2.0/MatMul", "decoder.point_fuse2")
+    cat = w.producer(w.node(pf1.onnx_node).inputs[0])
+    if cat.op_type != "Concat" or list(cat.inputs) != [point_stem.onnx_output] + [s.point_fusion.onnx_output
+                                                                                 for s in stages]:
+        _fail("point_fuse_layer1 input is not concat[p0, p1, p2, p3, p4]")
+    if w.node(pf2.onnx_node).inputs[0] != pf1.onnx_output:
+        _fail("point_fuse_layer2 does not consume point_fuse_layer1")
+    taps["decoder.point_fuse1"] = pf1.onnx_output
+    taps["decoder.point_fuse2"] = pf2.onnx_output
+
+    # ---- head: h = pixel2point(fuse2); h = mlp_i(h) + skip_i; logits; softmax
+    mlps = [_point_layer(w, f"/model/mlps.{i}/mlps.{i}.0/MatMul", f"head.mlp{i}") for i in range(4)]
+    skips = [pf2.onnx_output, ffe[2].onnx_output, ffe[1].onnx_output, ffe[0].onnx_output]
+    prev = None
+    for i, (lin, skip) in enumerate(zip(mlps, skips)):
+        add = _single(w, lin.onnx_output, "Add")
+        if set(add.inputs) != {lin.onnx_output, skip}:
+            _fail(f"head mlps.{i} skip is not {skip}")
+        if prev is not None and w.node(lin.onnx_node).inputs[0] != prev:
+            _fail(f"head mlps.{i} does not consume the previous residual sum")
+        taps[f"head.mlp{i}"] = add.outputs[0]
+        prev = add.outputs[0]
+    conv_seg = _gemm(w, "/model/conv_seg/Gemm", "head.logits")
+    if conv_seg.relu or w.node(conv_seg.onnx_node).inputs[0] != prev or conv_seg.out_features != NUM_CLASSES:
+        _fail("conv_seg is not the final 64 -> 27 Gemm on the last residual sum")
+    sm = _single(w, conv_seg.onnx_output, "Softmax")
+    if sm.attrs.get("axis", -1) not in (-1, 1) or sm.outputs[0] != "pred_probs":
+        _fail("the output is not Softmax(axis=-1) of conv_seg")
+    taps["head.logits"] = conv_seg.onnx_output
+    taps["pred_probs"] = "pred_probs"
+
+    scale = pre.scale.astype(np.float32)
+    shift = pre.shift.astype(np.float32)
+    raw = int(sum(int(np.asarray(w.array(t)).size) for t in w.initializer_names()
+                  if np.issubdtype(np.asarray(w.array(t)).dtype, np.floating)))
+    out = FRNetWeights(variant=variant, frustum_hw=(H, W), strides=strides, pre_norm=pre, pre_norm_scale=scale,
+                       pre_norm_shift=shift, ffe=tuple(ffe), compression=compression, stem=stem,
+                       point_stem=point_stem, point_stem_segmax=point_stem_segmax, fusion_stem=fusion_stem,
+                       stages=tuple(stages), fuse_layer1=fuse1, fuse_layer2=fuse2, point_fuse=(pf1, pf2),
+                       head_mlps=tuple(mlps), conv_seg=conv_seg, onnx_taps=taps, onnx_path=str(path),
+                       onnx_sha256=digest, raw_parameters=raw)
+    folded = sum(c.weight.size + c.bias.size for c in out.convs) + sum(
+        lin.weight.size + lin.bias.size for lin in out.linears) + 16
+    object.__setattr__(out, "folded_parameters", int(folded))
+    missing = [t for t in MODULE_TAPS if t not in taps]
+    if missing:
+        _fail(f"no ONNX tensor for taps {missing}")
+    return out
+
+
+def check_variant_grid(weights: FRNetWeights, frustum_h: int, frustum_w: int) -> None:
+    """The ml_package grid must equal the grid baked into the ONNX (S:frnet:95-98: 1024 OT128 / 256 QT128)."""
+    if weights.frustum_hw != (frustum_h, frustum_w):
+        raise ValueError(f"ml_package frustum {frustum_h}x{frustum_w} != the ONNX's {weights.frustum_hw}")
+

code/tt_frnet/samples/README.md

@@ -0,0 +1,27 @@
+# Sample inputs shipped with frnet-p150
+
+Only data generated by this bundle ships here: one **synthetic** LiDAR sweep per serve profile, made by
+`code/scripts/make_sample.py` (no dataset involved; Apache-2.0, like the code). Public-dataset frames (PandaSet,
+nuScenes) are used for validation and demo renders but are not in this repository: PandaSet-derived samples ship only
+once they are approved, and non-commercial data (nuScenes) and the Autoware demo rosbag's VLS-128 frames (licence
+unstated) never ship.
+
+Never use the `.bin` suffix here (or `.pt`, `.pth`, `.ckpt`, `.safetensors`): tt-model's staging silently drops
+those suffixes from `code/` (`CODE_IGNORE`, tt-model-manager `src/tt_kernel/build.py:328-332`). Store point clouds as
+`.npy` / `.npz` / `.pcd`.
+
+| file | content |
+|---|---|
+| `synthetic_ot128.npz` | `points`: structured float32 (x, y, z, intensity), 53,411 rows in the **sensor frame** (x forward, y left, z up): a ray cast from a sensor 2 m above a flat road against box-shaped cars, building facades, poles and pedestrians; 128 rows over the OT128's +15 / -25 deg, 512 columns over 360 deg, 5 % dropout, 2 cm range noise, random 0-255 intensity (seed 0). With the calibration preset `synthetic_ot128` (a pure 2 m lift): 52,613 points after the ego crop box, no interpolated point, N = M = 52,613 cells |
+| `synthetic_qt128.npz` | the same scene seen by a QT128-like sensor: 128 rows over +-52.6 deg, 600 columns; 45,783 rows. With the preset `synthetic_qt128`: 41,806 points after the crop, N = 69,553 after Autoware's interpolation, M = 18,926 cells |
+| `synthetic_<v>.<v>.reference.json` | the stored fp32 CPU-reference output of the sample for profile `<v>`: the `/predict` body of `tt_frnet.reference.pipeline` with ONNX Runtime on the deployed ONNX (the reference of record), reduced to what the container smoke compares: `labels` (Autoware `class_id`), `scores` (its `probability`), `filter_keep` (the filtered cloud) and the counts |
+
+`server/smoke_test.py` posts the sample of the served variant with its preset and requires >= 98.5 % label agreement
+with the stored reference (PLAN.md 2.8 argmax gate); `tests/test_e2e_device.py::test_shipped_sample` gates the same
+on the device. Regenerate the references with `code/scripts/make_sample.py` whenever the reference, the weights or
+the post-processing changes.
+
+The scans are geometric toys: the network has never seen such scenes (it labels most of the synthetic road
+`other_flat_surface` on the OT128 profile and finds few car points). They exercise the full input contract and the
+device path, and they are the smoke test's input; they say nothing about accuracy on real data. The model card's
+accuracy rows come from public datasets.

code/tt_frnet/samples/synthetic_ot128.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f95d27f004dc7e8ed4f14e9a14da38d07775657e3f92f615fdd429e69eaa5e49
+size 679575

code/tt_frnet/samples/synthetic_qt128.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01c8aa6c2740b903e9133942e12cb9a091a05ea11b9cf6ed03fe1af61aef13c7
+size 579254

code/tt_frnet/server/__init__.py

@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: Apache-2.0
+"""HTTP serving of frnet-p150, thin bindings of the vendored ``ttaw.server``: ``app.py`` (the ASGI app tt-model
+runs with uvicorn), ``client.py`` (stdlib request builder, runnable as a script), ``smoke_test.py`` (stdlib PASS/FAIL
+check of a running server: ETH dispatch and the 12x10 grid asserted, agreement with the stored CPU reference)."""

code/tt_frnet/server/app.py

@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: Apache-2.0
+"""ASGI serving app of frnet-p150 on one Tenstorrent Blackhole p150: the HTTP contract of the Autoware
+collection (vendored ``ttaw.server.app``, BUNDLE_CONVENTIONS.md section 7) bound to :class:`FRNet`.
+
+Served by tt-model-manager as ``kind: tt-dit-server``::
+
+    python -m uvicorn --host 0.0.0.0 --port <p> --lifespan on tt_frnet.server.app:app
+
+Routes: ``GET /``, ``/health`` and ``/v1/health`` (always 200: ``ok`` / ``starting`` / ``error``), ``/info``,
+``/v1/models`` (stub), ``POST /predict``; errors 400 / 422 / 503 / 500 (SERVING.md section 3). Everything that
+touches the device happens in the lifespan: weights -> device (ETH dispatch, 12x10) -> graph -> trace capture of
+every warm-up variant, so uvicorn's ``Application startup complete`` (the line ``tt-model serve`` waits for) means
+warm; SIGTERM (``tt-model stop``, 120 s) closes the model under the lock. ``/predict`` calls the Python API, so it
+returns exactly what ``model(...)`` returns. Importing this module has no side effects beyond importing fastapi and
+pydantic (the image's ``verify:`` imports it without a device); the environment is read in the lifespan only.
+
+Model-specific request fields (e.g. PointPainting ``rois``): subclass ``PredictRequest``, pass ``request_model=`` and
+``decode_extra=`` (which adds the decoded field to the call kwargs) to :class:`ServerSpec`, and list the keyword in
+``EXTRA_INPUTS`` of the model class. Host tests swap the model before starting the app:
+``app.state.ttaw.model_factory = Stub``; ``app.state.ttaw.predict(request)`` is the route handler itself.
+"""
+from pathlib import Path
+
+from .. import __version__
+from ..api import FRNet
+from ..ttaw.server.app import PredictRequest, ServerSpec, create_app, parse_mesh_shape
+
+__all__ = ["SPEC", "app", "PredictRequest", "parse_mesh_shape"]
+
+SPEC = ServerSpec(
+    model_name=FRNet.MODEL_NAME,
+    env_prefix=FRNet.ENV_PREFIX,
+    model_cls=FRNet,
+    task="LiDAR 3D semantic segmentation",
+    default_weights=FRNet.DEFAULT_REPO,
+    owner="changh95",
+    io="one LiDAR sweep in the sensor frame (x, y, z, raw 0-255 intensity) in, a class id and probability for every point (27 T4 classes; Autoware's interpolated points included) out",
+    autoware={"package": "autoware_lidar_frnet",
+              "path": "perception/autoware_lidar_frnet",
+              "autoware_universe": "9ceaccf026c31ffc5319bc9eeb4bd7bede0af3fd"},
+    source={"repo": "https://huggingface.co/changh95/frnet-p150", "license": "Apache-2.0"},
+    calib_dir=Path(__file__).resolve().parents[1] / "calib",
+    version=__version__,
+    description="FRNet (Autoware lidar_frnet) on one Tenstorrent Blackhole p150. "
+                "Not an OpenAI-compatible API.",
+)
+
+app = create_app(SPEC)