---
tags:
- blackhole
- p150
- tt-dit-server
- tt-model-cache
- tt-model-container
- tenstorrent
- ttnn
- tt-metal
- tt-nn
- robotics
- vla
- gr00t
- tt-model-catalog
pipeline_tag: robotics
license: other
license_name: nvidia-license
license_link: https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/LICENSE
base_model:
- nvidia/GR00T-N1.5-3B
---

# gr00t-n1.5-3b-p150

NVIDIA Isaac GR00T N1.5 (3B vision-language-action policy: SigLIP ViT (224², 256 tokens/image) → linear projector → 12-layer Qwen3-1.7B backbone → 4-block VL self-attention → 16-block cross/self AdaLN DiT, 4 Euler flow-matching steps) running on one Tenstorrent Blackhole p150a via tt-nn: one camera frame + proprioceptive state + a task instruction in, a 16-step chunk of GR1 arm/hand joint targets out. Stage-1 release: every op is a TTNN op replayed from four Metal traces (DiT block matmul weights bfp8, everything else bf16); the persistent-megakernel denoise is in progress and not in this image.
Weights: [nvidia/GR00T-N1.5-3B](https://huggingface.co/nvidia/GR00T-N1.5-3B) (NVIDIA License, non-commercial) · Paper: [arXiv:2503.14734](https://arxiv.org/abs/2503.14734) (GR00T N1 report; the N1.5 changes are described in the [N1.5 model card](https://huggingface.co/nvidia/GR00T-N1.5-3B)) · Upstream code: [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) (tag `n1.5-release`) · Port: `code/models/experimental/gr00t` in this repo (tt-metal-tree layout, frozen snapshot `5dc0517ee3b`) + the policy server `code/gr00t_p150`

Runs on **p150** (mesh `P150`).

Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1).

## Quickstart

```bash
tt-model pull  changh95/GR00T-N1.5-3B-p150 --with-weights
tt-model serve changh95/GR00T-N1.5-3B-p150
```

- Weights [`nvidia/GR00T-N1.5-3B`](https://huggingface.co/nvidia/GR00T-N1.5-3B) at `869830fc749c` (3 bf16 safetensors shards, 5.45 GB) go to your HF cache; the image does not contain them.
- Serves on port 20000 (or the next free port); ready when the log says `Application startup complete`.

### Run with tt-cli

```bash
tt serve changh95/GR00T-N1.5-3B-p150
python3 - <<'EOF'
import base64, json
body = json.load(open("media/demo_observation.json"))          # GR1 PickNPlace demo step: raw joint state + instruction
body.pop("_note", None)
body["images"] = {"ego_view": base64.b64encode(open("media/demo_ego_view.png", "rb").read()).decode()}
json.dump(body, open("req.json", "w"))
EOF
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/GR00T-N1.5-3B-p150
```

- `POST /predict`: `images` (`{"ego_view": <base64 PNG/JPEG>}` — the one GR1 camera; any size with sides in 64–4096 px, the server applies the checkpoint's own eval chain: 0.95 centre crop → bilinear 224×224 → Eagle2.5 normalisation), `state` (`{group: [[floats]]}`: `left_arm` 7, `right_arm` 7, `left_hand` 6, `right_hand` 6 — raw joint values, one time step), `instruction` (task text, ≤ 102 BPE tokens); optional `embodiment` (`gr1`, the only layout in this image), `seed` (initial flow-matching noise; default 0 = the deployed policy's seed → deterministic) or `noise` (an explicit `[16][32]` initial noise), `state_dtype` (`float64` default, as the GR1 dataset), `return_normalized` (also return the model-space chunk).
- `GET /health`, `GET /info` (the full input/output contract, weights/tree provenance, warm-up latency and fidelity), `GET /demo` (the shipped demo request and its fp32 reference actions).

### Response

```json
{"actions": {"left_arm":   [[0.0069, 0.2107, 0.0483, -1.8136, -0.1264, -0.0548, 0.0147], "... 16 steps"],
             "right_arm":  [[-0.0177, -0.1731, 0.3702, -1.7429, 0.4968, -0.0567, 0.9849], "..."],
             "left_hand":  [[0.0081, 0.0155, 0.0033, 0.0064, 0.0169, 0.0438], "..."],
             "right_hand": [[-1.4880, -1.4915, -1.4781, -1.4810, -2.9609, 2.9904], "..."]},
 "action_horizon": 16, "action_keys": ["left_arm", "right_arm", "left_hand", "right_hand"], "action_dims": {"left_arm": 7, "right_arm": 7, "left_hand": 6, "right_hand": 6},
 "normalized": false, "embodiment": "gr1", "embodiment_id": 24, "version": "n15", "model": "GR00T-N1.5-3B-p150", "layout": "gr1",
 "images": {"ego_view": {"frames": 1, "received_hw": [256, 256]}}, "seq_len": 296, "prompt_tokens": 14, "state_dtype": "float64",
 "noise_source": "client", "seed": null,
 "timing_ms": {"decode": 1.5, "encode": 2.6, "device": 43.0, "decode_actions": 0.1, "total": 47.2}}
```

- `actions` are **physical, un-normalised** joint targets (radians) in the GR1 arms-and-hands action space of the checkpoint's `experiment_cfg/metadata.json` statistics, 16 steps × [7, 7, 6, 6] dims — what `Gr00tPolicy.get_action` returns for this checkpoint; the values above are abridged from the shipped demo request (`GET /demo`) with its golden initial noise. `seq_len` is the LLM prompt length the request produced (296 for the demo; padded to 384 on device). `timing_ms.device` covers input upload, the four trace replays and the action readback; `total` is the whole handler.

### Demo

| Input (`media/demo_ego_view.png`, GR1 sim `robot_sim.PickNPlace` traj 0 / step 100, 256×256) | Instruction | p150a actions vs the fp32 reference (`media/demo_actions.png`) |
|:---:|:---|:---:|
| ![](media/demo_ego_view.png) | `pick the pear from the counter and place it in the plate` | ![](media/demo_actions.png) |

### Accuracy and speed

| Metric | Value |
|---|---:|
| Action chunk vs the fp32 reference (`Gr00tPolicy`, same observation, same initial noise) | PCC `left_arm` 0.99995 · `right_arm` 0.99997 · `right_hand` 0.99999 (gates 0.9999; max\|d\| 0.021 / 0.017 / 0.029 rad) · `left_hand` max\|d\| 0.018 rad ≤ 0.046 (PCC 0.943 reported only: the golden chunk is near-constant there, spread 0.1 rad — NVIDIA's own bf16 GPU path scores 0.952) · normalised chunk `action_pred_normalized` 0.99998 (max\|d\| 0.014 ≤ 0.027) |
| Intermediate taps vs the fp32 reference (untraced path from pixels, 57 gated taps + determinism row) | **58/58 rows pass**; tightest: `vit_block_first` 0.99994 (gate 0.9999), `backbone_features` 0.99586 (0.995), `vl_self_attention_out` 0.99119 (0.991), `vit_post_ln` 0.99506 (0.993) |
| Traced == untraced, determinism, stale state | traced `action_pred` / `backbone_features` / hoisted K,V bit-equal to the untraced run; 5 trace replays bit-identical; mutated inputs (`n_text` > L_max, other shapes) refused before any device write; A→B→A→B alternating observations on one captured model pass |
| Inference on p150a (port benchmark, warm, batch 1, one 224² image, LLM S 296→384, median of 50; `benchmarks/results/e2e_stage1_n15.json`) | **43.4 ms end-to-end** (p90 44.1) = host encode 1.5 + upload 3.8 + 4 traces 37.8 (vision 8.7 · LLM 8.7 · adapter 4.5 · denoise 16.0) + readback 0.1 + decode 0.2 |
| Served over HTTP by this image (`tt-model serve`, warm, batch 1, the demo request, 10 warm-ups + 50 timed; `scripts/bench_http.py`) | **42.4 ms device · 45.4 ms server-side** (medians; p90 43.2 / 49.2; min 41.7 / 44.5) = image decode 1.1 + host encode 1.6 + device 42.4 (upload, 4 trace replays, readback) + un-normalise 0.1; 47.6 ms client wall on the same host (p90 51.8). Under a heavily loaded host (two concurrent tt-metal builds, load average 16–110) the same loop read 44.3 / 44.5 ms device and 53.6 / 58.1 ms server-side medians with p90s of 55 / 91–97 ms — the host-side stages, not the device, absorb the contention. The served actions are bit-identical to the port's device test and to the host run |
| Same policy on an RTX 5090 (same host, NVIDIA's unmodified `Gr00tPolicy`, eager PyTorch bf16 autocast + flash-attn 2, batch 1) | `get_action` e2e **58.0 ms** (preprocessing 2.2, model 55.4 = backbone 20.8 + action head 34.6) → the p150a is 1.34× faster end-to-end (43.4 vs 58.0) and 1.33× on the model (41.7 device vs 55.4); NVIDIA's N1.5 README quotes 47.9 ms on an H100 (backbone 23.2 + head 4 × 6.2) |

### Caveats

- **Stage-1 path.** Vision / LLM / adapter / denoise are TTNN ops captured in four Metal traces; DiT block matmul weights are bfp8_b, everything else bf16 (`mixed_dit`). The persistent-megakernel denoise (Stage 2) is at rung K1 — its weight-streaming kernel measures 464 GB/s bf16 / 414 GB/s bfp8 (90.6 % of DRAM peak) on the full DiT block set — and is **not** in this image.
- **One embodiment, one layout.** The image is built for GR1 arms-and-hands (`gr1`, embodiment slot 24, one `ego_view` camera, ≤ 102 text tokens → LLM sequence padded to 384); other embodiments, cameras or longer instructions are refused with 400, nothing is truncated. Batch 1; requests are serialised on one lock.
- **Noise is seeded** (seed 0, as the deployed policy); pass `seed` or `noise` for another draw. The reference noise of the shipped demo is not seed-reproducible, so the smoke test sends it as `noise`. Outputs are the base checkpoint's actions for the GR1 sim data config — a real robot needs NVIDIA's post-training.
- Not an OpenAI-compatible API; `GET /v1/models` is a stub so the tt-model ready card does not 404.
- Validated on tt-metal main `668c2907575` (`v0.79.0-dev20260914-1`), single p150a only, with this exact image (`tt-model serve` → smoke test → 50-request benchmark → `tt-model stop`, three boots). **First boot is cold**: the checkpoint is converted into the port's device layout under `~/.cache/tt-model/gr00t-n1.5-3b-p150/weights` (6.4 GB, 9 s) and the kernels of the four traces are JIT-compiled into `…/cache` (411 MB, ~87 s incl. trace capture) — `tt-model serve` reported READY after 1 min 43 s; the next boots reused both caches and were ready in 20 s (loaded host) and 9.7 s (quiet host). Every boot re-checks the warm-up actions against the shipped fp32 reference (PCC 0.999982 here) and refuses to serve below 0.999.
- GPU comparison: the RTX 5090 row is NVIDIA's unmodified `Gr00tPolicy` path (`Isaac-GR00T` `n1.5-release`, torch 2.7.1+cu128, transformers 4.51.3, flash-attn 2.8.3), best median of 3 × (20 warm-ups + 20 timed calls), GPU idle, no `torch.compile` / TensorRT; the p150a rows are the Stage-1 trace path incl. upload and readback. p150a power was not measured, so no efficiency comparison is made. Full table: [`GPU_COMPARISON.md`](GPU_COMPARISON.md).

### Licensing

- Weights: [nvidia/GR00T-N1.5-3B](https://huggingface.co/nvidia/GR00T-N1.5-3B), **NVIDIA License** — use limited to research or evaluation (non-commercial), redistribution only under the same licence (copy in [`LICENSE`](LICENSE)). Not redistributed here; `tt-model serve` fetches the pinned snapshot into your HF cache.
- Tokenizer files under `code/gr00t_p150/assets/tokenizer/n15/` (Qwen2 BPE + Eagle2.5 special tokens) are copied from [NVIDIA/Isaac-GR00T `n1.5-release`](https://github.com/NVIDIA/Isaac-GR00T/tree/n1.5-release) (Apache-2.0).
- Port and serving code (`code/models/experimental/gr00t`, `code/gr00t_p150`, `code/scripts`): Apache-2.0 (SPDX headers), © 2026 Tenstorrent; as a derivative work of the weights it is distributed here under the same non-commercial use limitation (NVIDIA License §3.2). tt-metal / tt-nn: Apache-2.0.
- Demo frame and state (`media/`): one step of the GR1 `robot_sim.PickNPlace` demo data shipped with Isaac-GR00T.

## Provenance

The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image:

| component | built from |
| --- | --- |
| tt-metal | [`668c290757550588d0ce46b180c344a462a2aaf5`](https://github.com/tenstorrent/tt-metal/commit/668c290757550588d0ce46b180c344a462a2aaf5) |
| `code/` digest | `a2771515bca98527` (sha256, first 16 hex digits) |
| built | 2026-09-14T15:17:00+00:00 by tt-model 0.1.0 |