--- tags: - blackhole - p150 - tt-dit-server - tt-model-cache - tt-model-container - tenstorrent - ttnn - tt-metal - tt-nn - robotics - vla - gr00t - tt-model-catalog pipeline_tag: robotics license: other license_name: nvidia-license license_link: https://huggingface.co/nvidia/GR00T-N1.7-3B/blob/main/LICENSE base_model: - nvidia/GR00T-N1.7-3B --- # GR00T-N1.7-3B-p150 NVIDIA Isaac GR00T N1.7 (3B vision-language-action policy) running on one Tenstorrent Blackhole p150a via tt-nn, in the DROID layout: two cameras × two frames (t−15 and t) at 16:9, a 17-dim Franka state and a task instruction in, a 40-step chunk of end-effector, gripper and joint actions out. Stage-2 release: the whole DiT denoise runs as one persistent megakernel streaming bfp8 weights from a DRAM arena; vision (tuned Qwen3-VL tower), LLM and adapter are TTNN ops in Metal traces; `GR00T_DIT_BACKEND=ttnn` selects the Stage-1 denoise. Weights: [nvidia/GR00T-N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B) · Paper: [arXiv:2503.14734](https://arxiv.org/abs/2503.14734) · Upstream code: [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) (`main`, the `n1.7-release` line) · Port: `code/models/experimental/gr00t` and the server `code/gr00t_p150` in this repo Runs on **p150** (mesh `P150`). Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1). ## Quickstart ```bash tt-model pull changh95/GR00T-N1.7-3B-p150 --with-weights tt-model serve changh95/GR00T-N1.7-3B-p150 ``` - Weights [`nvidia/GR00T-N1.7-3B`](https://huggingface.co/nvidia/GR00T-N1.7-3B) at `2fc962b973bc` (6.91 GB) go to your HF cache; the image does not contain them. - Serves on port 20000 (or the next free port); ready when the log says `Application startup complete`. ### Run with tt-cli ```bash tt serve changh95/GR00T-N1.7-3B-p150 python3 - <<'EOF' import base64, json b64 = lambda f: base64.b64encode(open(f"media/{f}.png", "rb").read()).decode() obs = json.load(open("media/demo_observation.json")) # DROID demo step: raw state + instruction obs["images"] = {cam: [b64(f"{cam}_tm15"), b64(f"{cam}_t0")] # per camera: the frame at t-15, then the frame at t for cam in ("exterior_image_1_left", "wrist_image_left")} json.dump(obs, open("req.json", "w")) EOF curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json tt model stop changh95/GR00T-N1.7-3B-p150 ``` - `POST /predict`: `images` (`{"exterior_image_1_left": [, ], "wrist_image_left": [...]}`, exactly two frames per camera, 16:9 frames), `state` (`{group: [floats]}`: `eef_9d` 9 = xyz + rot6d, `gripper_position` 1, `joint_position` 7, raw values), `instruction` (≤ 51 tokens); optional `seed` (default 42) or `noise` (`[40, 132]`), `return_normalized`. - `GET /health`, `GET /info` (`stage` says what is served), `GET /demo` (the shipped demo request with its reference noise, and its fp32 reference actions). ### Response ```json {"actions": {"eef_9d": [[0.5284, 0.0181, 0.4569, -0.0786, 0.3927, -0.9163, 0.9923, 0.1193, -0.034], "... 40 steps"], "gripper_position": [[0.0], [0.002], "..."], "joint_position": [[-0.0051, 0.034, 0.0013, -1.917, 0.1641, 1.6101, 0.0844], "..."]}, "action_horizon": 40, "embodiment": "oxe_droid_relative_eef_relative_joint", "seq_len": 469, "noise_source": "client", "seed": null, "timing_ms": {"decode": 3.4, "encode": 10.9, "device": 55.9, "decode_actions": 1.1, "total": 71.3}} ``` - The example is the demo request with the reference noise, as `GET /demo` returns it (`noise_source: "client"`, `seed: null`); `timing_ms` from a warm request of the same serve. - `actions` are physical, un-normalised targets, 40 steps × [9, 1, 7] dims, as `Gr00tPolicy.get_action` returns them for the DROID data config: `eef_9d` is the **absolute** end-effector pose (the relative SE(3) output composed onto the request's pose), `joint_position` the relative output added to the request's joints, `gripper_position` absolute; `timing_ms.device` covers upload, the four trace replays and the readback. ### Demo | Input (`media/demo_frames.png`: both cameras at t−15 and t; DROID sample episode 1, step 40, 320×180) | Instruction | p150a actions vs the fp32 reference (`media/demo_actions_vs_golden.png`) | |:---:|:---|:---:| | ![](media/demo_frames.png) | `Put the blue block in the green bowl` | ![](media/demo_actions_vs_golden.png) | ### Accuracy and speed | Metric | Value | |---|---:| | Action chunk vs the fp32 reference (same observation, same initial noise) | PCC `eef_9d` 0.99998 · `joint_position` 0.99996 · `gripper_position` max\|d\| 0.0031 (near-constant group) · all 17 dims × 40 steps 0.99996 · normalised valid slice 0.99995 | | Intermediate taps vs the fp32 reference (27 gated taps, from pixels; DiT internals are not observable inside the megakernel) | 30 of 30 rows pass; tightest `vit_deepstack_mergers[0]` 0.999849 vs gate 0.9998. On six more DROID goldens the actions pass on five: `t2_s120` has two pre-existing misses (`llm_layer_first` 0.98998 vs 0.991, `joint_position` PCC 0.99976 / max\|d\| 0.107 vs 0.9998 / 0.083); `t1_s120` adds one megakernel-only marginal tap, `pred_velocity[k=2]` dim 9 per-dim PCC 0.99588 vs 0.996 (its actions pass) | | Megakernel vs Stage-1 TTNN denoise (same device, 7 DROID samples) | `action_pred_normalized` PCC ≥ 0.999992 between the two backends; per-step velocities ≥ 0.99996 | | Traced vs untraced, determinism, stale state, 2 vs 1 command queue | bit-identical; 5 trace replays bit-identical; A→B→A→B reproduces A and B exactly; 2-CQ == 1-CQ on every probe (26 rows) | | Inference on p150a (port benchmark, warm, batch 1, median of 50) | **62.8 ms end-to-end** (p90 68.6) ≈ host encode 5.6 + upload 3.2 + traces 52.4 (vision 17.5 · LLM 12.4 · adapter 6.2 · denoise 16.5 = 4 × 4.12) + readback and decode 1.3. Stage-1 path on the same tree: 79.7 ms (denoise 30.7) | | Served over HTTP by this image (warm, batch 1, 50 requests) | 55.9 ms device (p90 56.3) · 69.1 ms server-side incl. four PNG decodes (p90 72.7) | | Same policy on an RTX 5090 (same host, NVIDIA's unmodified N1.7 policy, eager PyTorch bf16, idle machine) | `get_action` 59.8–65.6 ms model, 73–79 ms incl. host pre/post-processing → p150a 1.22× faster end-to-end; NVIDIA's published H100 TensorRT full pipeline: 27.9 ms (one camera) | ### Caveats - Stage-2 path: one persistent `generic_op` runs the whole denoise inside the fourth Metal trace, streaming the 32 blocks' bfp8 weights once per step from a ~1 GB DRAM arena over the 11×10 Tensix grid (96 compute cores, 8 DRAM readers, hubs), with distributed LayerNorm, per-head flash SDPA and the cross attention over the 448-key image set split three ways. Costs: 64 KiB less allocatable L1 per core (`worker_l1_size` 1,395,712), the VL adapter's intermediates in DRAM (+0.6 ms), compute binaries of ~131 KB sitting ~4.5 KB under the 136 KB kernel-config ring, and no DiT-intermediate taps. Gain vs the Stage-1 path on the same tree: −19 % end-to-end, −46 % denoise (the plan's −30 % e2e was not reached). Everything outside the DiT matmuls is bf16. - The vision tower is the port's tuned Qwen3-VL fork (HiFi3 matmuls on blocks 0–5; 17.5 ms vs 87 ms for the verbatim tower, `GR00T_VIT_FORK=0` selects it). - One embodiment and layout: DROID (`oxe_droid_relative_eef_relative_joint`, embodiment slot 24), exactly two cameras × two frames (t−15 and t), landscape frames that land on the 16×28-patch grid (width / height in [1.686, 1.814]: 16:9, e.g. 320×180), instruction ≤ 51 tokens; anything else is refused with 400. Batch 1. - Noise: default `seed` 42 (CPU generator, deterministic). The demo's reference noise (`code/gr00t_p150/demo/n17/noise.json`) was drawn on CUDA and is not seed-reproducible; send it as `noise` to reproduce the reference actions. - Not an OpenAI-compatible API; `GET /v1/models` is a stub. - First boot is cold (weight conversion + kernel JIT, 64 s); later boots take 14 s. Validated on tt-metal `668c2907575` (port snapshot `009de892c895`), single p150a only. Details: [`SERVING.md`](SERVING.md), [`GPU_COMPARISON.md`](GPU_COMPARISON.md). ### Licensing - Weights: [nvidia/GR00T-N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B), **NVIDIA License** (non-commercial; copy in [`LICENSE`](LICENSE)). Not redistributed here. - Tokenizer and image-processor files (`code/gr00t_p150/assets/tokenizer/n17/`) are byte-identical to [Qwen/Qwen3-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct) (Apache-2.0; hashes in `PROVENANCE.md` next to them). - Port and serving code (`code/`): Apache-2.0, © 2026 Tenstorrent, distributed under the same non-commercial limitation as a derivative of the weights. See [`LICENSE-NOTICE.md`](LICENSE-NOTICE.md). ## Provenance The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image: | component | built from | | --- | --- | | tt-metal | [`668c290757550588d0ce46b180c344a462a2aaf5`](https://github.com/tenstorrent/tt-metal/commit/668c290757550588d0ce46b180c344a462a2aaf5) | | `code/` digest | `9efa5512d0634561` (sha256, first 16 hex digits) | | built | 2026-09-17T20:42:14+00:00 by tt-model 0.1.0 |