---
tags:
- blackhole
- p150
- tt-dit-server
- tt-model-cache
- tt-model-container
- tenstorrent
- ttnn
- tt-metal
- tt-nn
- robotics
- vla
- gr00t
- tt-model-catalog
pipeline_tag: robotics
license: other
license_name: nvidia-license
license_link: https://huggingface.co/nvidia/GR00T-N1.6-3B/blob/main/LICENSE
base_model:
- nvidia/GR00T-N1.6-3B
---

# gr00t-n1.6-3b-p150

NVIDIA Isaac GR00T N1.6 (3B vision-language-action policy: SigLIP2 ViT (252², 81 tokens per image) → 16-layer Qwen3-1.7B backbone → 32-block alternating cross/self-attention AdaLN DiT, 4 Euler flow-matching steps) running on one Tenstorrent Blackhole p150a via tt-nn: one camera frame + proprioceptive state + a task instruction in, a 16-step chunk of GR1 arm / hand / waist actions out. Stage-1 release: every op is a TTNN op replayed from four Metal traces (DiT matmul weights bfp8, everything else bf16); the persistent-megakernel denoise is in progress and is not in this image.
Weights: [nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B) · Paper: [arXiv:2503.14734](https://arxiv.org/abs/2503.14734) (GR00T N1 report) · Project page: [research.nvidia.com/labs/gear/gr00t-n1_6](https://research.nvidia.com/labs/gear/gr00t-n1_6/) · Upstream code: [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) (`n1.6.1-release`) · Port: `code/models/experimental/gr00t` in this repo (tt-metal `models/experimental/gr00t`, snapshot `5dc0517e`) + `code/gr00t_p150` (server)

Runs on **p150** (mesh `P150`).

Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1).

## Quickstart

```bash
tt-model pull  changh95/GR00T-N1.6-3B-p150 --with-weights
tt-model serve changh95/GR00T-N1.6-3B-p150
```

- Weights [`nvidia/GR00T-N1.6-3B`](https://huggingface.co/nvidia/GR00T-N1.6-3B) at `d0814e7ecb19` (2 bf16 safetensors shards, 6.57 GB) go to your HF cache; the image does not contain them.
- Serves on port 20000 (or the next free port); ready when the log says `Application startup complete`.

### Run with tt-cli

```bash
tt serve changh95/GR00T-N1.6-3B-p150
python - <<'EOF'
import base64, json
obs = json.load(open("media/demo_observation.json"))          # GR1 PickNPlace demo step: raw state + instruction
obs["images"] = {"ego_view_bg_crop_pad_res256_freq20": base64.b64encode(open("media/demo_ego_view.png", "rb").read()).decode()}
json.dump(obs, open("req.json", "w"))
EOF
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/GR00T-N1.6-3B-p150
```

- `POST /predict`: `images` (`{camera key: base64 PNG/JPEG}` — exactly one camera, `ego_view_bg_crop_pad_res256_freq20`, for the GR1 embodiment; any size, letterboxed / cropped / resized to 252×252 server-side by the reference eval chain), `state` (`{group: [floats]}`: `left_arm` 7, `right_arm` 7, `left_hand` 6, `right_hand` 6, `waist` 3 — raw joint values), `instruction` (task text, ≤ 24 BPE tokens in this layout); optional `embodiment` (`gr1`, the only layout in this image), `seed` (initial flow-matching noise; default = the deployed policy's fixed seed 42 → deterministic), `noise` (explicit `[50, 128]` initial noise), `state_dtype` (`float32` default), `return_normalized`.
- `GET /health`, `GET /info` (full contract, device facts, warm-up fidelity), `GET /demo` (the shipped demo request + its golden actions).

### Response

```json
{"actions": {"left_arm":   [[-0.0289, 0.1453, 0.0224, -2.1383, -0.0165, -0.0260, -0.0414], "... 16 steps × 7"],
             "right_arm":  [[-0.0024, -0.1502, 0.0280, -2.1461, -0.0091, 0.0132, 0.0044], "..."],
             "left_hand":  [[0.0097, -0.0177, 0.0041, 0.0078, 0.0539, 0.0281], "..."],
             "right_hand": [[-0.2339, -0.2466, -0.2431, -0.2311, -0.2423, 0.3333], "..."],
             "waist":      [[-0.0010, -0.0060, 0.0008], "..."]},
 "action_horizon": 16, "action_keys": ["left_arm", "right_arm", "left_hand", "right_hand", "waist"],
 "action_dims": {"left_arm": 7, "right_arm": 7, "left_hand": 6, "right_hand": 6, "waist": 3},
 "normalized": false, "embodiment": "gr1", "embodiment_id": 20, "version": "n16", "model": "GR00T-N1.6-3B-p150", "layout": "gr1",
 "images": {"ego_view_bg_crop_pad_res256_freq20": {"frames": 1, "received_hw": [256, 256]}}, "seq_len": 116, "prompt_tokens": 12,
 "state_dtype": "float32", "noise_source": "seed", "seed": 42,
 "timing_ms": {"decode": 0.8, "encode": 1.5, "device": 56.5, "decode_actions": 0.1, "total": 58.9}}
```

- `actions` are **physical, un-normalised** joint targets, 16 steps × [7, 7, 6, 6, 3] dims, exactly what `Gr00tPolicy.get_action` returns for the GR1 data config: the arm and hand groups are the checkpoint's RELATIVE outputs already composed onto the request's `state` (per-step min/max un-normalisation + the raw last state, clipped), `waist` is absolute. `seq_len` is the LLM prompt length the request produced (116 for the demo; padded to 128 on device).
- `timing_ms`: `decode` = base64 + PNG decode + validation, `encode` = the port's host preprocessing (image chain, prompt + tokenizer, sin/cos state), `device` = input upload + the four trace replays + one blocking readback, `decode_actions` = un-normalisation, `total` = handler wall time.

### Demo

| Input (`media/demo_ego_view.png`, GR1 sim PickNPlace, 256×256) | Instruction |
|:---:|:---|
| ![](media/demo_ego_view.png) | `pick the pear from the counter and place it in the plate` |

Served action chunk vs the fp32 reference for this observation (same initial noise), one panel per action dimension: `media/demo_actions_vs_golden.png`.

![](media/demo_actions_vs_golden.png)

### Accuracy and speed

| Metric | Value |
|---|---:|
| Action-chunk PCC vs the fp32 golden (official `Gr00tPolicy` code on CPU, same observation, same noise) | `left_arm` 0.99995 (gate 0.9999) · `right_arm` 0.99991 (0.9998) · `right_hand` 0.99312 (0.99); `left_hand` 0.801 and `waist` 0.772 reported-only (near-constant groups, golden range < 0.25 — max\|d\| 0.059 / 0.017 within tolerance); all 29 action dims concatenated **0.999209**, `action_pred_valid` 0.99921 (gate 0.998) |
| Intermediate taps vs golden (45 canonical taps, untraced, from pixels) | **52 of 56 gated rows pass; 4 marginal misses** (57 of 61 rows incl. the determinism row and the four xfail info rows), all on DiT state-token row 0 (which no action reads): `dit_out[k=1]` 0.999556 vs gate 0.99961, `dit_out[k=2]` 0.999496 vs 0.99964, `action_decoder_out[k=1]` 0.999579 vs 0.9997, `action_decoder_out[k=2]` 0.999556 vs 0.99978 (excluding row 0: 0.99984 / 0.99984 / 0.99991 / 0.99992). Root cause: bf16 HiFi2 SigLIP2 tower error propagation (`vit_block_last` 0.99276), not the DiT — `test_tt_action_head` passes on golden inputs |
| Traced == untraced, determinism, stale state | bit-identical (13/13 rows; 5 replays + 1 traced-after-untraced; A→B→A→B alternation passes) |
| Inference on p150a, `bench_e2e` (warm, batch 1, one 256² frame → 252², S 116→128, median of 50) | **59.4 ms end-to-end** (p90 60.1) = host encode 2.0 + upload 3.8 + 4 traces 53.3 (vision 11.5 · LLM 9.5 · adapter 2.0 · denoise 30.4) + D2H 0.1 + decode 0.2 |
| Served over HTTP from this image (container, warm, 50 requests after 5 warm-ups, seed path, quiet host) | **56.8 ms device** (p90 57.3, min 56.5, max 57.6) · **59.4 ms server-side total** incl. PNG decode (p90 59.8) · 61.0 ms client wall on localhost (p90 61.6) |
| Same policy on an RTX 5090 (same host, NVIDIA's unmodified code, eager PyTorch bf16 + flash-attn 2, batch 1) | `get_action` e2e 63.1 ms (backbone 16.8, action head 36.2) → **p150a 1.06× faster e2e**; with `torch.compile(DiT, max-autotune)` the GPU reaches 33.1 ms (head 13.5) → **GPU 1.8× faster** than this Stage-1 path; NVIDIA's published RTX 5090 numbers: eager 58 ms, compiled 37 ms |

### Caveats

- **Stage-1 path.** Vision / LLM / adapter / denoise are TTNN ops captured in four Metal traces; DiT block matmul weights are bfp8_b, everything else bf16 (`mixed_dit`). The persistent-megakernel denoise (Stage 2) is at rung K1 — weight streaming measured at 464 GB/s bf16 / 414 GB/s bfp8 over the full 32-block N1.6 weight set — and is **not** in this image.
- **N1.6 fidelity margin.** The served chunk reproduces the fp32 golden at PCC 0.99921 on the normalised valid slice (NVIDIA's own bf16 GPU run scores 0.99956 there), 2e-4 above the 0.999 boot / smoke gate; the four intermediate-tap misses above are disclosed verbatim. N1.5 (`changh95/GR00T-N1.5-3B-p150`) passes every tap.
- **One embodiment, one layout.** The image is built for GR1 (`gr1`, embodiment slot 20, one `ego_view_bg_crop_pad_res256_freq20` camera, instruction ≤ 24 BPE tokens → LLM sequence padded to 128); other embodiments / cameras / longer instructions are refused (400). Batch 1; requests are serialised on one lock.
- **Noise is seeded** (seed 42, as the deployed policy); pass `seed` or `noise` for a different draw. Outputs are the base checkpoint's actions for the GR1 sim data config — a real robot needs NVIDIA's post-training.
- Not an OpenAI-compatible API; `GET /v1/models` is a stub so the tt-model ready card does not 404.
- Validated on tt-metal main `668c2907575` (`v0.79.0-dev20260914-1`), single p150a only. First boot converts the checkpoint to device layout and compiles every kernel (137 s cold, see SERVING.md) into `~/.cache/tt-model/gr00t-n1.6-3b-p150/{weights,cache}` (~8 GB); later boots take 13–21 s (13 s with the weight tier still in the host page cache, 21 s an hour later).
- GPU comparison: RTX 5090 rows (2026-09-14) are NVIDIA's unmodified `Gr00tPolicy` path (`Isaac-GR00T` `n1.6.1-release`, PyTorch eager bf16 + flash-attn 2, `cudnn.benchmark` on, deterministic mode off), medians of 20 timed iterations after 5 warm-ups (`torch.cuda.synchronize()` around every call), GPU otherwise idle; the p150a row is the Stage-1 trace path incl. upload / readback. p150a power was not measured, so no efficiency comparison is made. Full table: [`GPU_COMPARISON.md`](GPU_COMPARISON.md).

### Licensing

- Weights: [nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B), **NVIDIA License** (non-commercial: research or evaluation use only; verbatim copy in [`LICENSE`](LICENSE)). Not redistributed here; fetched into your HF cache by `tt-model serve` / `scripts/download_weights.sh`.
- Tokenizer files under `code/gr00t_p150/assets/tokenizer/n16/` (Qwen2 BPE + Eagle special tokens) are copied from [NVIDIA/Isaac-GR00T `n1.6.1-release`](https://github.com/NVIDIA/Isaac-GR00T) (`gr00t/model/modules/nvidia/Eagle-Block2A-2B-v2`), whose `LICENSE` is the same NVIDIA License.
- Port and serving code (`code/models/experimental/gr00t`, `code/gr00t_p150`, `code/scripts`): Apache-2.0 (SPDX headers), © 2026 Tenstorrent; distributed here under the same non-commercial use limitation as the weights (NVIDIA License §3.2). tt-metal (`code/models/common`, `code/models/tt_dit`, `code/models/demos`, and the image): Apache-2.0.
- Demo frame and state (`media/`, `code/gr00t_p150/demo/n16`): one GR1 PickNPlace step from the Isaac-GR00T demo data; the fp32 / bf16 reference actions shipped beside it were computed with NVIDIA's code on the pinned weights.

## Provenance

The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image:

| component | built from |
| --- | --- |
| tt-metal | [`668c290757550588d0ce46b180c344a462a2aaf5`](https://github.com/tenstorrent/tt-metal/commit/668c290757550588d0ce46b180c344a462a2aaf5) |
| `code/` digest | `7a456e1065da7159` (sha256, first 16 hex digits) |
| built | 2026-09-14T16:25:52+00:00 by tt-model 0.1.0 |