--- tags: - blackhole - p150 - tt-dit-server - tt-model-cache - tt-model-container - tenstorrent - ttnn - tt-metal - tt-nn - robotics - vla - gr00t - tt-model-catalog pipeline_tag: robotics license: other license_name: nvidia-license license_link: https://huggingface.co/nvidia/GR00T-N1.6-3B/blob/main/LICENSE base_model: - nvidia/GR00T-N1.6-3B --- # GR00T-N1.6-3B-p150 NVIDIA Isaac GR00T N1.6 (3B vision-language-action policy) running on one Tenstorrent Blackhole p150a via tt-nn: one camera frame, proprioceptive state and a task instruction in, a 16-step chunk of GR1 arm, hand and waist actions out. Stage-2 release: the whole DiT denoise (32 blocks × 4 flow-matching steps) runs as one persistent megakernel streaming bfp8 weights from a DRAM arena; vision, LLM and adapter are TTNN ops (bf16) replayed from Metal traces; `GR00T_DIT_BACKEND=ttnn` selects the Stage-1 denoise. Weights: [nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B) · Paper: [arXiv:2503.14734](https://arxiv.org/abs/2503.14734) · Project page: [gr00t-n1_6](https://research.nvidia.com/labs/gear/gr00t-n1_6/) · Upstream code: [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) (`n1.6.1-release`) · Port: `code/models/experimental/gr00t` and the server `code/gr00t_p150` in this repo Runs on **p150** (mesh `P150`). Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1). ## Quickstart ```bash tt-model pull changh95/GR00T-N1.6-3B-p150 --with-weights tt-model serve changh95/GR00T-N1.6-3B-p150 ``` - Weights [`nvidia/GR00T-N1.6-3B`](https://huggingface.co/nvidia/GR00T-N1.6-3B) at `d0814e7ecb19` (6.57 GB) go to your HF cache; the image does not contain them. - Serves on port 20000 (or the next free port); ready when the log says `Application startup complete`. ### Run with tt-cli ```bash tt serve changh95/GR00T-N1.6-3B-p150 python3 - <<'EOF' import base64, json obs = json.load(open("media/demo_observation.json")) # GR1 demo step: raw state + instruction obs["images"] = {"ego_view_bg_crop_pad_res256_freq20": base64.b64encode(open("media/demo_ego_view.png", "rb").read()).decode()} json.dump(obs, open("req.json", "w")) EOF curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json tt model stop changh95/GR00T-N1.6-3B-p150 ``` - `POST /predict`: `images` (`{"ego_view_bg_crop_pad_res256_freq20": }`), `state` (`{group: [floats]}`: `left_arm` 7, `right_arm` 7, `left_hand` 6, `right_hand` 6, `waist` 3, raw joint values), `instruction` (≤ 24 tokens); optional `seed` (default 42) or `noise` (`[50, 128]`), `return_normalized`. - `GET /health`, `GET /info` (`stage` names the denoise backend), `GET /demo` (the shipped demo request and its golden actions). ### Response ```json {"actions": {"left_arm": [[-0.0289, 0.1453, 0.0224, -2.1383, -0.0165, -0.0225, -0.0428], "... 16 steps"], "right_arm": [[-0.0024, -0.1512, 0.0269, -2.1461, -0.0091, 0.0124, 0.0044], "..."], "left_hand": [[0.0135, -0.0092, -0.0047, 0.0070, 0.0455, 0.0281], "..."], "right_hand": [[-0.2253, -0.2421, -0.2345, -0.2311, -0.2338, 0.3385], "..."], "waist": [[0.0001, -0.0060, 0.0008], "..."]}, "action_horizon": 16, "embodiment": "gr1", "seq_len": 116, "seed": 42, "timing_ms": {"decode": 0.9, "encode": 1.4, "device": 39.6, "decode_actions": 0.2, "total": 42.1}} ``` - `actions` are physical, un-normalised joint targets, 16 steps × [7, 7, 6, 6, 3] dims, as `Gr00tPolicy.get_action` returns them for the GR1 data config (arm and hand groups are relative outputs composed onto the request's `state`, `waist` is absolute); `timing_ms.device` covers upload, the trace replays and the readback. ### Demo | Input (`media/demo_ego_view.png`, GR1 sim PickNPlace, 256×256) | Instruction | p150a actions vs the fp32 reference (`media/demo_actions_vs_golden.png`) | |:---:|:---|:---:| | ![](media/demo_ego_view.png) | `pick the pear from the counter and place it in the plate` | ![](media/demo_actions_vs_golden.png) | ### Accuracy and speed | Metric | Value | |---|---:| | Action chunk vs the fp32 reference (same observation, same initial noise) | PCC `left_arm` 0.99995 · `right_arm` 0.99990 · `right_hand` 0.99308 · `left_hand` / `waist` max\|d\| 0.059 / 0.013 (near-constant groups) · all 29 action dims 0.99923 (normalised valid slice 0.99922) | | Intermediate taps vs the fp32 reference (the 23 taps the megakernel path exposes, from pixels; DiT block internals are not observable inside the kernel) | 26 of 28 gated rows pass (29 of 31 rows incl. the determinism row and the two xfail info rows); 2 marginal misses on the DiT state-token row that no action reads (`action_decoder_out[k=1,2]` 0.999594 / 0.999598 vs gates 0.9997 / 0.99978), caused by bf16 error in the SigLIP2 tower; the Stage-1 card's `dit_out[k=1,2]` misses are now inside the kernel | | Megakernel vs the Stage-1 TTNN denoise, same device | `action_pred_normalized` PCC ≥ 0.999994 on every N1.6 reference sample | | Traced vs untraced, determinism | bit-identical (13 of 13 rows); 5 trace replays bit-identical; 2 CQs == 1 CQ | | Inference on p150a (warm, batch 1, median of 50) | **40.7 ms end-to-end** (p90 40.8) = host encode 1.4 + upload 1.4 + traces 37.8 (vision 10.8 · LLM 9.6 · adapter 2.2 · denoise 15.3 = 3.84 ms per step) + readback and decode 0.1 | | Served over HTTP by this image (warm, batch 1, 50 requests) | 39.6 ms device · 42.1 ms server-side incl. PNG decode (p90 42.5) | | Same policy on an RTX 5090 (same host, NVIDIA's unmodified `Gr00tPolicy`, eager PyTorch bf16 + flash-attn 2) | 63.1 ms end-to-end → p150a 1.55× faster; with `torch.compile` on the DiT the GPU reaches 33.1 ms → GPU 1.23× faster than this Stage-2 path | ### Caveats - Stage-2 path: the `denoise` trace is one persistent `generic_op` on all 110 Tensix cores (8 readers stream the 32-block weight set once per step from a 0.98 GB bfp8 DRAM arena to 96 compute cores; distributed LayerNorm, per-head flash SDPA, Euler update and decoder inside the kernel). Against the ttnn denoise on the same tree (e2e 56.2 ms, denoise 30.4) it saves 27.6 % / 49.6 %, short of the planned 30 % e2e; `GR00T_DIT_BACKEND=ttnn` selects that path. - Fidelity margin: the served chunk matches the fp32 reference at PCC 0.99923 (physical actions; 0.99922 on the normalised valid slice, where NVIDIA's own bf16 GPU run scores 0.99956), 2.3e-4 above the 0.999 boot self-check. - Costs of the megakernel: 64 KiB less allocatable L1 per core (the kernel-config ring is 136,192 B, the compute-core binaries take ≈ 131.7 KB of it), the VL adapter's intermediates in DRAM (+0.2 ms adapter trace), and no DiT-internal taps (10 head taps + the backbone taps remain gated). - One embodiment and layout: GR1 (embodiment slot 20), one `ego_view_bg_crop_pad_res256_freq20` camera, instruction ≤ 24 tokens; anything else is refused with 400. Batch 1. - Noise is seeded (seed 42, as the deployed policy); pass `seed` or `noise` for another draw. Outputs are the base checkpoint's actions for the GR1 sim data config. - Not an OpenAI-compatible API; `GET /v1/models` is a stub. - First boot is cold (weight conversion + kernel JIT incl. the megakernel, 58 s); later boots take 14 s. Validated on tt-metal `668c2907575`, single p150a only. Details: [`SERVING.md`](SERVING.md), [`GPU_COMPARISON.md`](GPU_COMPARISON.md). ### Licensing - Weights: [nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B), **NVIDIA License** (non-commercial; copy in [`LICENSE`](LICENSE)). Not redistributed here. - Tokenizer files (`code/gr00t_p150/assets/tokenizer/n16/`) are from Isaac-GR00T `n1.6.1-release` under the same NVIDIA License. - Port and serving code (`code/`): Apache-2.0, © 2026 Tenstorrent, distributed under the same non-commercial limitation as a derivative of the weights. ## Provenance The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image: | component | built from | | --- | --- | | tt-metal | [`668c290757550588d0ce46b180c344a462a2aaf5`](https://github.com/tenstorrent/tt-metal/commit/668c290757550588d0ce46b180c344a462a2aaf5) | | `code/` digest | `8e6bd13a23654f91` (sha256, first 16 hex digits) | | built | 2026-09-17T20:40:28+00:00 by tt-model 0.1.0 |