Media attribution: frnet-p150 demo renders
Every render shows the p150 output: the port running on one Tenstorrent Blackhole p150 (tt-nn; ETH dispatch, 1
command queue, 12x10 grid; the serve.env precision knobs), called through the Python API
(FRNet.from_pretrained(variant=...), then model(points, calibration=...)), which returns exactly what
POST /predict returns: Autoware's per-point class_id after Autoware's pre-processing (ego crop box, projection,
interpolation) and post-processing (first-max argmax). Only the raw points the node scores are drawn (Autoware's
interpolated points are not in the dataset views). The caption inside each image gives the frame's agreement with the
fp32 CPU reference of the same Autoware network (the deployed ONNX through ONNX Runtime with the port's bit-exact
Autoware pre- and post-processing; research/frnet/public_data/runs). The *_tt_vs_cpu* views show the p150 classes,
the reference's classes and the points whose class differs. The dataset views compare the p150 classes with the
dataset's own labels mapped to the FRNet classes (dark grey: no T4 counterpart); the weights were not trained on
these datasets or sensors.
Colours: the Autoware FRNet palette of ml_package_frnet_ot128.param.yaml (27 classes). Renderers:
code/scripts/make_demo.py (tt_frnet.viz for the BEV comparisons) and, for the camera views, the research renderer
of the public-data set (research/frnet/public_data/scripts/frnet_public_render.py: same frames, panels and camera
projections as the CPU-reference renders), workspace code that is not shipped, because the datasets are not.
| file | source | license |
|---|---|---|
frnet_synthetic_ot128_tt_vs_cpu.png, frnet_synthetic_qt128_tt_vs_cpu.png |
the shipped samples code/tt_frnet/samples/synthetic_<variant>.npz, generated by code/scripts/make_sample.py (a ray cast against boxes; no dataset) |
Apache-2.0 |
frnet_ot128_pandaset_019_40_tt.png |
PandaSet seq 019 (San Francisco, Embarcadero), frame 40: Pandar64 sweep, front_camera/40.jpg, semseg labels |
CC BY 4.0 + PandaSet Dataset Terms |
frnet_ot128_pandaset_029_40_tt.png |
PandaSet seq 029 (San Francisco, residential hill and intersection), frame 40: same members | CC BY 4.0 + PandaSet Dataset Terms |
frnet_ot128_pandaset_019_40_tt_vs_cpu.png |
PandaSet seq 019, frame 40: Pandar64 sweep (bird's-eye view only) | CC BY 4.0 + PandaSet Dataset Terms |
frnet_ot128_pandaset_019_bev_tt.gif |
PandaSet seq 019, Pandar64 sweeps 00-75, every 5th frame (bird's-eye view only) | CC BY 4.0 + PandaSet Dataset Terms |
frnet_ot128_nuscenes_scene-0103_20_tt_NC.png |
nuScenes v1.0-mini, mini_val scene-0103 (Boston Seaport), key-frame 20: LIDAR_TOP, CAM_FRONT, nuScenes-lidarseg labels | CC BY-NC-SA 4.0 + nuScenes Terms of Use (non-commercial; the render carries the label) |
frnet_ot128_nuscenes_scene-0916_40_tt_vs_cpu_NC.png |
nuScenes v1.0-mini, mini_val scene-0916 (Singapore), key-frame 40: LIDAR_TOP (bird's-eye view only) | CC BY-NC-SA 4.0 + nuScenes Terms of Use (non-commercial; the render carries the label) |
What was changed in every dataset render: the PandaSet Pandar64 sweep re-expressed in an estimated sensor frame (its per-point motion compensation undone), the nuScenes LIDAR_TOP sweep rotated to x forward; bird's-eye views rendered; camera images downscaled to at most 960 px wide and dimmed; the points projected into the camera and coloured by class; caption and attribution bars added. Nothing is cropped or zoomed in on people or license plates.
PandaSet
Contains data from PandaSet (Scale AI and Hesai), https://pandaset.org, licensed under CC BY 4.0 and the PandaSet Dataset Terms. Changes: the Pandar64 sweep was re-expressed in an estimated sensor frame, the front-camera image was downscaled and dimmed, and LiDAR points and labels were overlaid; bird's-eye views rendered. Scale AI and Hesai do not endorse this work. Cite: P. Xiao et al., PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving, ITSC 2021.
The Dataset Terms forbid using the data to identify people and using Scale AI's or Hesai's names or logos beyond the attribution. No PandaSet data (points, images) is in this repository; only these renders are. A PandaSet sample is prepared but not shipped until its redistribution is approved.
nuScenes (non-commercial)
Rendered from the nuScenes dataset, © Motional AD Inc., CC BY-NC-SA 4.0 and the nuScenes Terms of Use (https://www.nuscenes.org/terms-of-use). Non-commercial use only; adaptations under the same license. Motional does not endorse this work. Cite: H. Caesar et al., nuScenes: A Multimodal Dataset for Autonomous Driving, CVPR 2020; W. K. Fong et al., Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking, RA-L 2022 (lidarseg labels).
The two _NC renders inherit CC BY-NC-SA 4.0: they are labelled "non-commercial, CC BY-NC-SA 4.0" wherever they
appear (the images carry the label). No nuScenes data (points, images, tensors) is in this repository.
Synthetic samples
code/tt_frnet/samples/synthetic_<variant>.npz and the two frnet_synthetic_* renders are generated by this
repository (code/scripts/make_sample.py, deterministic, seed 0): Apache-2.0.