# Serving GR00T N1.7-3B (`nvidia/GR00T-N1.7-3B`, DROID layout) on Blackhole with tt-model-manager

This repo is a **tt-model container package source**: `tt-model.yaml` + the code under `code/`. `tt-model package
--container` builds the OCI image (tt-metal from source + the port + the HTTP stack), `tt-model serve` runs it against
the card, `tt-model push` publishes it here. The server is `code/gr00t_p150/server/app.py` (FastAPI, `kind:
tt-dit-server`, shared with the N1.5 / N1.6 packages); the device model is the tt-nn port `code/models/experimental/gr00t`
(**Stage-2 path**: the whole DiT denoise as one persistent megakernel `generic_op` streaming a bfp8_b DRAM weight arena, inside
the fourth Metal trace; vision / LLM / adapter as TTNN ops in the other three traces; DiT matmul weights bfp8_b, everything
else bf16; the port's tuned Qwen3-VL vision tower). Weights are a pinned pointer, never in the image.

| | |
|---|---|
| tt-metal | main `668c290757550588d0ce46b180c344a462a2aaf5` (`v0.79.0-dev20260914-1-g668c290757`), built from source inside the image; torch pin 2.11.0 (+cpu in the image) |
| weights | `nvidia/GR00T-N1.7-3B` @ `2fc962b973bccdd5d8ce4f67cc63b264d6886495` — `model-0000{1,2}-of-00002.safetensors` (6,910,361,856 B of bf16 tensors, 1,031 tensors), `model.safetensors.index.json`, `config.json`, `processor_config.json`, `statistics.json`, `embodiment_id.json`, `LICENSE` (NVIDIA License, non-commercial) |
| app | `gr00t_p150.server.app:app` (uvicorn `--lifespan on`; the lifespan resolves the snapshot, resolves the `TTPolicy` from `GR00T_*`, opens the device the policy needs, builds `Gr00tTT`, captures the traces, runs `GR00T_WARMUP_RUNS` traced predictions and checks them against the shipped fp32 golden before READY) |
| device recipe | `models.experimental.gr00t.tt.model.open_model_device(policy, trace_region_size=64 MiB, l1_small_size=32768, num_command_queues=2, device_id)`: under the megakernel backend the device is opened with `worker_l1_size` 1,461,248 − 65,536 = **1,395,712 B** (the 64 KiB cut buys the 136,192 B kernel-config ring the megakernel binaries need) and the VL adapter's intermediates live in DRAM; 2 command queues (per-call input writes on CQ 1 overlap the traces, bit-equal to 1 CQ — `tests/tt/test_mk_2cq.py`); asserts the 11×10 compute / 8×1 DRAM grid of a p150. `GR00T_DIT_BACKEND=ttnn` opens the firmware-default L1 and serves the Stage-1 TTNN denoise |
| port source | `code/models/experimental/gr00t` = the frozen snapshot commit `009de892c895917ad6abe9a6c36518f00316ed4b` (ref `gr00t-p150-snapshot-mk`, parent = `668c2907575`) of the tt-metal working tree `models/experimental/gr00t`; the `tt/megakernel/` tree is the K4b checkpoint (16 files, digest `7df61c35e9dc7857`, `tests/tt/results/mk_tree_20260918-041439.sha256`, re-hashed by `verify:` inside the image); `code/gr00t_p150` (server) and `code/scripts` were written for these releases |
| only served configuration | version `n17`, embodiment `oxe_droid_relative_eef_relative_joint` (slot 24), static layout `droid` (2 cameras × 2 frames = 4 images × 112 tokens on a 16×28-patch grid, `L_max` 51 text tokens, LLM sequence padded to 512), `mixed_dit` dtype policy, `per_stage` trace layout, `dit_backend=megakernel` with the `bfp8_b` arena, 2 command queues, tuned tower (`GR00T_VIT_FORK=1`), batch 1 |

Below, `$ROOT` = the tt-models tooling checkout (`/home/deepgadget/experiments/tt-models` on the build host: `tt-model` CLI in
`$ROOT/.venv/bin`, rootless Docker env in `$ROOT/bin/docker-env.sh`), `$TREE` = the tt-metal tree with the built `python_env`
(`/home/deepgadget/experiments/gr00t/tt-metal`), `$REPO` = this directory.

## Layout

```
tt-model.yaml                 authoring manifest (schema 5.1) -- the whole build/serve interface
tt_kernel_manifest.json       GENERATED wire manifest: weights pointer, image tag/digest, runtime, serve, verify, built provenance
requirements.lock             GENERATED `pip freeze` of the image venv (Python 3.12, torch 2.11.0+cpu, torchvision 0.26.0+cpu, ttnn <scm>)
image/                        GENERATED OCI layout of the image (~3 GB)
README.md                     the card (generated by `tt-model package` from card.description / card.quickstart; front matter restored after push)
SERVING.md, GPU_COMPARISON.md, LICENSE, LICENSE-NOTICE.md     authored
media/{exterior_image_1_left,wrist_image_left}_{tm15,t0}.png   the four DROID demo frames (320x180; tm15 = t-15, t0 = current), media/demo_frames.png their 2x2 montage,
media/demo_observation.json   the matching raw state + instruction, media/demo_actions_vs_golden.png the served chunk vs the fp32 reference
code/models/experimental/gr00t/    the port: common/ (host preprocessing incl. the DROID eval chain + Qwen2-VL patching, prompts, normalisation,
                                   checkpoint, weight plan), reference/ (fp32 torch reference), tt/ (device model, layers, traces, the tuned
                                   Qwen3-VL tower tt/qwen3vl_vision.py, tt/action_head.py with both DiT backends, tt/megakernel/ = the persistent
                                   DiT megakernel: arena.py (DRAM weight arena), core_map.py, descriptors.py, dit_program.py, kernels/dit_kernel.cpp +
                                   kernels/ops/*.hpp), tests/ (CPU + device suites, the cited result JSONs and the WP summaries), benchmarks/ (+ results)
code/models/common/utility_functions.py, code/models/tt_dit/{utils,layers,parallel,encoders/qwen3vl}/..., code/models/demos/deepseek_v3_b1/{unified_kernel_descriptor.py, unified_kernels/{kernel_op_api,kernel_utils,dataflow_utils,mcast}.hpp}
                                   the 20 tt-metal files the port imports or #includes from the tree other than itself (staged by tt-model from `source.code`;
                                   unified_kernel_descriptor.py builds the megakernel's generic_op and dit_kernel.cpp #includes the four unified_kernels headers;
                                   vision_qwen3vl.py is the verbatim tower = the oracle behind GR00T_VIT_FORK=0)
code/gr00t_p150/              server/{app.py, schemas.py, smoke_test.py}, demo/n17/ (canonical DROID demo observation, reference noise + golden),
                              assets/tokenizer/n17/ (vendored Qwen2 tokenizer + Qwen2-VL preprocessor_config.json, PROVENANCE.md)
code/scripts/                 download_weights.sh (pinned snapshot -> HF cache), bench_http.py (HTTP latency, 50 requests)
```

In the image the code lands at `/opt/tt-metal/<path>` with `PYTHONPATH=/opt/tt-metal`, which is exactly the tree layout the port was
written for (`models.experimental.gr00t.*`, `models.tt_dit.*` absolute imports; `models` is a namespace package).

## 1. Run on the HOST (hardware validation, no Docker)

The tree venv `$TREE/python_env` (Python 3.10, torch 2.11.0+cpu, torchvision 0.26.0, ttnn editable, transformers 5.12.1, opencv
4.8.1) has everything except `fastapi` / `uvicorn`. Do not install into the tree venv; put the HTTP stack in a side directory and
prepend it to `PYTHONPATH`. `$REPO/code` must be FIRST on `PYTHONPATH` so `models.experimental.gr00t` resolves to the shipped snapshot
(the tree's `models/` is still needed for `models.tt_dit.*` on the host; in the image the staged `code/` is the only `models` tree).

```bash
export TREE=/home/deepgadget/experiments/gr00t/tt-metal
export PATH=$HOME/.local/bin:$PATH                                       # uv
HTTP=/home/deepgadget/experiments/gr00t/publish/http-side                # any writable dir outside the trees
uv pip install --python $TREE/python_env/bin/python --target $HTTP fastapi uvicorn "pydantic>=2"

cd $REPO
export PYTHONPATH=$PWD/code:$TREE:$TREE/ttnn:$TREE/tools:$HTTP
export TT_METAL_HOME=$TREE ARCH_NAME=blackhole
export TT_METAL_CACHE=$HOME/.cache/tt-metal-cache-gr00t                  # JIT kernels (warm on the build host)
export HF_MODEL=nvidia/GR00T-N1.7-3B
export TT_WEIGHTS_REVISION=2fc962b973bccdd5d8ce4f67cc63b264d6886495      # scripts/download_weights.sh puts it in ~/.cache/huggingface
export TT_MESH_SHAPE=1x1 TT_DEVICE_ID=0 TT_METAL_VISIBLE_DEVICES=0
export GR00T_VERSION=n17 GR00T_EMBODIMENT=oxe_droid_relative_eef_relative_joint GR00T_LAYOUT=droid
export GR00T_POLICY=mixed_dit GR00T_TRACE_LAYOUT=per_stage GR00T_VIT_FORK=1
export GR00T_DIT_BACKEND=megakernel GR00T_MK_ARENA_DTYPE=auto GR00T_NUM_CQS=2   # = the port's defaults; GR00T_DIT_BACKEND=ttnn for the Stage-1 A/B
export GR00T_TOKENIZER_DIR=$PWD/code/gr00t_p150/assets/tokenizer/n17
export GR00T_TT_CACHE=$HOME/.cache/gr00t-tt                              # host .pt plan tier + .tensorbin device tier (~2.9 GB for n17 under the megakernel)
export GR00T_WARMUP_RUNS=5

# import check, no device (what the image's verify.sh does):
$TREE/python_env/bin/python -c "import gr00t_p150.server.app as a; assert a.app; print('ok')"

# serve (resolves the policy, opens the device with the megakernel's L1 cut, loads the weights, builds the arena, captures the
# four traces, 5 warm-up predictions checked against the golden, then READY).
# The one Blackhole card of the build host is shared with other agents: run server + client + shutdown inside ONE lock hold.
DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
  $TREE/python_env/bin/python -m uvicorn --host 127.0.0.1 --port 20017 --lifespan on gr00t_p150.server.app:app & UV=$!
  python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:20017 --wait 1800 --out /tmp/gr00t-n17-smoke.json; RC=$?
  python3 code/scripts/bench_http.py --url http://127.0.0.1:20017 --n 50 --out /tmp/gr00t-n17-bench.json
  kill -TERM $UV; wait $UV; exit $RC'
```

Boot log landmarks (they drive `tt-model serve`'s boot checklist): `Loading weights: nvidia/GR00T-N1.7-3B @ 2fc962b9…` →
`Tokenizer files: …` → `Opening device 0 (trace_region_size=67108864, l1_small_size=32768, num_command_queues=2, dit_backend=megakernel,
mk_arena_dtype=bfp8_b)` → `Loading pipeline: Gr00tTT.from_pretrained(...)` → `Model built in N s (weights … s, 529 tensors, 3008 MB on device,
cache path warm|cold; dit_backend megakernel, arena bfp8_b, adapter_mem DRAM, head {mk_arena_plan_pack_s …, mk_upload_s …}; ttnn DiT weights
skipped: 534 tensors / 1032 MB)` → `Warming up: capture per_stage traces on the n17 demo observation, then 5 traced predict(s)` →
`Warmup k/5: … ms` → `Warm-up fidelity vs golden fp32: actions PCC 0.999959…` → `Warmup complete (…)` → uvicorn `Application startup
complete`. Under the megakernel the ttnn DiT weights are not uploaded (534 tensors / 1,032 MB skipped, `GET /info -> stage.weights`); the
megakernel's own bfp8_b arena (~1 GB) is packed on the host and uploaded during the model build (`head_timing_s`). Startup failures raise
and uvicorn exits non-zero (no CPU fallback); a warm-up PCC below `GR00T_WARMUP_PCC_MIN` (0.999) fails the boot. Stop with SIGTERM / Ctrl-C:
the lifespan releases the traces and device tensors and closes the device (`Releasing traces and device tensors`, `Closing device`).

Expected smoke line on the demo observation (the shipped image, cold-boot cycle of the container validation,
`publish/logs/smoke-mk-n17-c1-<stamp>.log` of the build host; the host run of `tests/tt/results/mk_default_summary.md` §7 gives the same fidelity):

```
PASS GR00T-N1.7-3B-p150 n17: PCC(actions)=0.999959 max|d|=0.0282 PCC(action_pred_valid)=0.999946 repeat_maxdiff=0.0e+00 seed_path_ok device_ms=56.51/56.35 total_ms=74.63/69.67 wall_ms=81 per_group[eef_9d=0.99998/0.0121 gripper_position=0.81268/0.0031 joint_position=0.99996/0.0282]
```

Host boot on the build host with a **fresh, empty** `TT_METAL_CACHE` and warm weight tiers (`mk_default_summary.md` §7,
`publish/logs/server_n17_20260918-043407.log`): `Model built in 9.5 s`, capture (JIT of every op + the megakernel) 39.3 s, boot to READY 52.7 s;
smoke `device_ms=56.72/56.18`, `total_ms=71.69/67.9`. With warm kernel and weight caches the host boot is under 10 s.
Offline overrides: `GR00T_WEIGHTS_DIR=<dir with the seven snapshot files>` (shimmed into a private hub cache so the port's
`configs.snapshot_dir("n17")` resolves to it); `GR00T_TOKENIZER_DIR=<dir>` (must also hold `preprocessor_config.json`);
`GR00T_TORCH_THREADS=<n>` (torch intra-op threads for the host preprocessing; unset in the image). Every variable is read in the
lifespan, never at import.

## 2. Package, serve, push (Docker)

Rootless Docker on the build host needs `source $ROOT/bin/docker-env.sh` first (PATH + `DOCKER_HOST`; the bare `docker` is podman).
**Run every `tt-model` command from this directory**: `source.tt_metal` and `extra_code[].root: code` resolve against the process CWD.
`--out` points outside any git checkout because `stage()` deletes `<out>/gr00t-n1.7-3b-p150` before rebuilding.

```bash
source $ROOT/bin/docker-env.sh
cd $REPO
# offline validation (must print VALID)
$ROOT/.venv/bin/python -c "from tt_kernel.container_manifest import load_container_manifest as L; m = L('tt-model.yaml', check_sources=True); print('VALID', m.name, m.weights_ref)"

$ROOT/.venv/bin/tt-model package --container tt-model.yaml --out /home/deepgadget/experiments/gr00t/publish/build   # ~2 h cold (tt-metal C++ build), minutes with the ccache mount; runs verify.sh
# serve -> smoke -> bench -> stop inside ONE device-lock hold (the container keeps the card after `serve` returns)
DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
  source $ROOT/bin/docker-env.sh
  $ROOT/.venv/bin/tt-model serve /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.7-3b-p150/tt_kernel_manifest.json || exit 1
  python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:20000 --out /tmp/gr00t-n17-smoke.json
  python3 code/scripts/bench_http.py --url http://127.0.0.1:20000 --n 50 --out /tmp/gr00t-n17-bench.json
  $ROOT/.venv/bin/tt-model stop /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.7-3b-p150/tt_kernel_manifest.json'   # org/name is a valid target only after pull/push
$ROOT/.venv/bin/tt-model push /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.7-3b-p150 --publish
```

`serve` pre-downloads the pinned files into `~/.cache/huggingface` (a metadata no-op when present), then
`docker run --user 0:0 --device /dev/tenstorrent --ipc host --mount /dev/hugepages-1G -v ~/.cache/huggingface:/hf
-v ~/.cache/tt-model/gr00t-n1.7-3b-p150/{cache,weights,tensors}:/{cache,weight-cache,tensor-cache} -p 20000:20000` with exactly
`HF_MODEL=nvidia/GR00T-N1.7-3B`, `MESH_DEVICE=P150`, `TT_MESH_SHAPE=1x1`, `HF_HOME=/hf`, `TT_METAL_CACHE=/cache`,
`TT_DIT_CACHE_DIR=/weight-cache`, `TT_CACHE_PATH=/tensor-cache` and the `serve.env` block of `tt-model.yaml`
(`TT_WEIGHTS_REVISION`, `GR00T_VERSION=n17`, `GR00T_LAYOUT=droid`, `GR00T_DIT_BACKEND=megakernel`, `GR00T_MK_ARENA_DTYPE=auto`,
`GR00T_NUM_CQS=2`, `GR00T_VIT_FORK=1`, `GR00T_TT_CACHE=/weight-cache/gr00t-tt`, `GR00T_TOKENIZER_DIR=/opt/tt-metal/gr00t_p150/assets/tokenizer/n17`, …),
and waits ≤ 1800 s for `Application startup complete`. Serves on 20000 or the next free port (printed). tt-cli users after the push:
`tt serve changh95/GR00T-N1.7-3B-p150` / `tt model stop changh95/GR00T-N1.7-3B-p150`.

Caches on the host (persist across boots): `~/.cache/tt-model/gr00t-n1.7-3b-p150/cache` (JIT kernels incl. the megakernel, `TT_METAL_CACHE`),
`~/.cache/tt-model/gr00t-n1.7-3b-p150/weights/gr00t-tt/n17/...` (the port's host plan tier + `.tensorbin` device tier, 2.9 GB under the megakernel — the skipped ttnn DiT tensors are not
converted either; 8.4 GB on the Stage-1 path — written on the first boot), weights in `~/.cache/huggingface/hub/models--nvidia--GR00T-N1.7-3B`.

Measured on the build host with this image (`docs/publish/build-mk-n17.md` of the port project has every command and log):

| boot | wall from `tt-model serve` to READY | notes |
|---|---:|---|
| first (cold: empty `/cache` and `/weight-cache`) | 64 s | `Model built in 14.0 s` (checkpoint → device-layout plan + `.tensorbin` tier + the bfp8_b arena), JIT of every kernel of the four traces incl. the megakernel + trace capture 43.2 s, 5 warm-ups; smoke PASS, 50-request bench 55.9 / 69.1 ms device / total, clean stop 2.1 s |
| second (warm caches, minutes later) | 14 s | weight tier and kernels from the persistent caches (`Model built in 4.8 s`, capture 3.3 s); smoke PASS, 50-request bench 55.9 / 67.7 ms device / total, clean stop 2.5 s |

`push` makes `code/` and `image/` on the Hub exactly the staged trees and replaces `README.md` with the generated card (everything worth
keeping lives in `card.description` / `card.quickstart`); `tt-model.yaml`, `SERVING.md`, `GPU_COMPARISON.md`, `LICENSE`,
`LICENSE-NOTICE.md`, `media/` survive (re-uploaded after every push). The front matter (`license: other`, `license_name: nvidia-license`,
`license_link`, `pipeline_tag: robotics`, `base_model`, extra tags) is restored afterwards with `huggingface_hub.metadata_update`.

## 3. Request / response contract

| route | returns |
|---|---|
| `GET /health` | `{"status": "ok" \| "starting", "model": "GR00T-N1.7-3B-p150", "device": "blackhole:0"}` — 200 always; `ok` only after warm-up |
| `GET /info` | model / version / task (40 actions) / hardware / **`stage`** (`served: stage2-megakernel`, `dit_backend`, `mk_arena_dtype` (resolved: `bfp8_b`), `num_command_queues`, `cq1_uploads`, `worker_l1_size` (1395712), `adapter_mem` (`DRAM`), `weights` {529 tensors, 3007.7 MB, `skipped_ttnn_dit` {534, 1031.9 MB}}, `head_timing_s`, a note) / `policy` (the `TTPolicy` actually used, incl. `dit_backend`, `mk_arena_dtype`, `vit_fork`) / `weights` (repo, revision, snapshot dir actually loaded) / `source` (port snapshot commit, server version, tt-metal commit, ttnn version) / `inputs` (the full observation contract: camera keys, `video_delta_indices [-15, 0]`, `frame_order`, `vit_grid_thw [1, 16, 28]`, `frame_sizes` acceptance rule, state groups, `L_max`) / `outputs` (the DROID composition rule) / `noise.default_seed` (42) / `limits` / `device_facts` / `warmup_latency_ms` / `warmup_fidelity_vs_golden` / `license` |
| `GET /demo` | `{"request": <ready-to-post demo body incl. the four frames and the reference noise>, "expected": <golden actions + gates>}` |
| `GET /v1/models` | OpenAI-shaped stub (`{"object": "list", "data": [{"id": "nvidia/GR00T-N1.7-3B", ...}]}`) so the tt-model ready card does not 404; this is not a chat API |
| `POST /predict` | one observation → one 40-step action chunk (below) |

Serving knobs (all read once in the lifespan; the image pins them in `serve.env`):

| variable | image | meaning |
|---|---|---|
| `GR00T_DIT_BACKEND` | `megakernel` | `TTPolicy.dit_backend`: `megakernel` (the DiT denoise as one persistent `generic_op`, Stage 2, the port default since 2026-09-18) or `ttnn` (the Stage-1 TTNN op sequence, for A/B; opens the firmware-default L1). Reported in `/info -> stage` |
| `GR00T_MK_ARENA_DTYPE` | `auto` (→ `bfp8_b`) | the megakernel's DRAM weight arena: `auto` = the port's per-version table (`bfp8_b` for n17), `bf16`, `bfp8_b` |
| `GR00T_NUM_CQS` | `2` | command queues of the device: `2` = per-call input writes on CQ 1 overlapping the traces (bit-equal to `1`, `tests/tt/test_mk_2cq.py`) |
| `GR00T_VIT_FORK` | `1` | `1` = the tuned Qwen3-VL tower, `0` = the verbatim tt_dit tower (5× slower, the oracle) |
| `GR00T_POLICY`, `GR00T_TRACE_LAYOUT` | `mixed_dit`, `per_stage` | dtype policy / trace layout (the validated pair) |

`POST /predict` request (JSON; unknown fields → 422):

| field | type | meaning |
|---|---|---|
| `images` | `{camera key: [base64 PNG/JPEG at t−15, base64 PNG/JPEG at t]}` | exactly the two cameras `exterior_image_1_left` and `wrist_image_left`, **exactly two frames each, oldest first** (the embodiment's video `delta_indices [-15, 0]`: at 15 fps the history frame is 1 s old). Frame sides in [64, 4096]; only frames whose eval chain (`SmallestMaxSize(256, INTER_AREA)` → `FractionalCenterCrop(0.95)` → `SmallestMaxSize(256)` → Qwen2-VL `smart_resize` to multiples of 32) lands on the served 16×28-patch grid are accepted, i.e. landscape frames with width / height in [1.686, 1.814] (16:9 such as 320×180, 1280×720, 1920×1080; 7:4; 9:5). Square, 4:3 and portrait frames → 400 naming the accepted ratios; frames of one camera must share a size |
| `state` | `{group: [D floats]}` (or `[[D floats]]`) | raw physical state: `eef_9d` 9 (end-effector xyz in metres + rotation as rot6d, one group as in the reference), `gripper_position` 1, `joint_position` 7 (rad); finite. Normalised on the server with the checkpoint's q01/q99 statistics + clip |
| `instruction` | str (aliases `language`, `prompt`) | task text, 1–2000 chars; ≤ 51 BPE tokens after the reference's `formalize_language` (lower-case, non-word characters stripped) in this layout (longer → 400: `n_text exceeds L_max`) |
| `embodiment` | str, optional | must be `oxe_droid_relative_eef_relative_joint` |
| `seed` | int ≥ 0, optional | CPU-generator seed of the `[1, 40, 132]` flow-matching noise; default 42 (`NOISE_SEED["n17"]`) |
| `noise` | `[40, 132]` or `[1, 40, 132]` floats, optional | explicit initial noise (the smoke test and `GET /demo` send the reference noise, which was drawn on CUDA and is not seed-reproducible); exclusive with `seed` |
| `state_dtype` | `float32` \| `float64`, optional | default `float32` for N1.7 (the DROID dataset's dtype) |
| `return_normalized` | bool, optional | also return `action_pred_normalized` (`[40, 132]` model-space chunk) and `action_pred_valid_hd` (`[40, 17]`) |

Response (200):

```json
{"actions": {"eef_9d": [[9 floats] x 40], "gripper_position": [[1] x 40], "joint_position": [[7] x 40]},
 "action_horizon": 40, "action_keys": ["eef_9d", "gripper_position", "joint_position"], "action_dims": {"eef_9d": 9, "gripper_position": 1, "joint_position": 7},
 "normalized": false, "embodiment": "oxe_droid_relative_eef_relative_joint", "embodiment_id": 24, "version": "n17", "model": "GR00T-N1.7-3B-p150", "layout": "droid",
 "images": {"exterior_image_1_left": {"frames": 2, "received_hw": [180, 320], "vit_grid_thw": [1, 16, 28]}, "wrist_image_left": {"frames": 2, "received_hw": [180, 320], "vit_grid_thw": [1, 16, 28]}},
 "seq_len": 469, "prompt_tokens": 8, "state_dtype": "float32", "noise_source": "seed", "seed": 42,
 "timing_ms": {"decode": 3.4, "encode": 10.9, "device": 55.9, "decode_actions": 1.1, "total": 71.3}}
```

`actions` are physical, un-normalised targets (float32 as decimal JSON), exactly what the reference `get_action` returns for the DROID
data config: `eef_9d` = the model's RELATIVE end-effector output (per-step q01/q99 un-normalisation from `statistics.json["relative_action"]`,
XYZ + rot6d → SE(3)) composed onto the request's `eef_9d` (`T_abs = T_state · T_rel`), i.e. an **absolute pose**; `joint_position` = the
relative joint output added to the request's `joint_position`; `gripper_position` ABSOLUTE (q01/q99). `timing_ms`: `decode` = base64 +
four PNG decodes + validation (incl. the frame-size rule), `encode` = the port's host preprocessing (cv2 eval chain on four frames,
Qwen2-VL patching, tokenizer, mRoPE tables), `device` = `predict_normalized` (input writes, `execute_trace` × 4 — the fourth replay is the
megakernel `generic_op` —, one blocking readback), `decode_actions` = un-normalisation + SE(3) composition, `total` = handler wall time.
Errors: **400** (wrong/missing camera or state group, one or three frames per camera, frame outside the accepted ratio band or [64, 4096]
sides, wrong state width, non-finite state, undecodable image, empty/too-long instruction, other embodiment, `seed` + `noise` together,
malformed noise), **422** (schema), **503** while starting, **500** `Type: text` on a device failure. Handlers are synchronous and serialised
on one lock; batch is 1.

## 4. Caveats

* **Stage-2 path: the DiT denoise is one persistent megakernel.** The fourth trace (`denoise`, 16.5 ms = 4 × 4.12 ms per Euler step)
  replays one `generic_op` that runs the action encoder, the 32 cross/self DiT blocks, the decoder and the Euler update for all four steps:
  the block weights (bfp8_b, ~1 GB DRAM arena, `tt/megakernel/arena.py`) are streamed once per step by 8 DRAM-bank reader cores and
  multicast through hub cores to 96 compute cores of the 11×10 grid (`core_map.py`), LayerNorm is distributed, attention is a per-head
  flash SDPA on the compute cores (`kernels/ops/head_sdpa.hpp`) with the long-key cross blocks (the 448-key image set) split three ways per
  head (`mk_k4b_summary.md` §0). What this costs: the device runs with 64 KiB less allocatable L1 per core (`worker_l1_size` 1,395,712 B,
  the kernel-config ring of the megakernel binaries — the compute-core program is ≈ 131 KB, ≈ 4.5 KB under the 136,192 B ring,
  `mk_k4b_summary.md` §5), the VL adapter's intermediates move to DRAM (`adapter` trace 6.2 vs 5.7 ms), and the DiT's per-step internals
  (`dit_out[k]`, block taps, AdaLN) are not observable from the served model — the tap comparison covers the backbone / adapter taps and the
  10 head taps (30 rows) instead of the Stage-1 path's 58. Gain vs the Stage-1 TTNN denoise on the same tree (`mk_default_summary.md` §5):
  e2e −19 % (64.4 vs 79.7 ms in the same session; 62.8 ms on the snapshot), device −20 % (55.7 vs 69.8 ms), denoise −46 % (16.5 vs
  30.7 ms); the plan's −30 % e2e gate is not met. `GR00T_DIT_BACKEND=ttnn` serves the Stage-1 path (`stage1-ttnn-traces`).
* **Tuned tower.** Vision / LLM / adapter are TTNN ops in three Metal traces (`vision` 17.5 ms, `llm` 12.4, `adapter` 6.2 — `bench_e2e` medians on
  the snapshot, `code/models/experimental/gr00t/benchmarks/results/bench_e2e_n17_20260918-050929.json`). The vision tower is the port's
  `TunedQwen3VLVisionTT` (HiFi2 + fp32-accumulate matmuls, **HiFi3 on blocks 0–5**, HiFi2 SDPA with (448, 448) chunks, `rotary_embedding_hf`,
  L1 activations, tuned mergers): 17.5 ms traced vs 87 ms for the verbatim tt-nn `Qwen3VlVisionModel` (`GR00T_VIT_FORK=0`, the oracle; its
  `vit_deepstack_mergers[0]` sits at 0.99965, below the 0.9998 gate — `docs/plan/n17-fidelity.md`). Unchanged from the Stage-1 package.
* **Fidelity margin.** Final actions reproduce the fp32 golden at PCC 0.99996 (all 17 dims × 40 steps) / 0.99995 on the normalised valid
  slice (boot/smoke gate 0.999); all 30 rows of the canonical tap comparison pass (27 gated taps; tightest `vit_deepstack_mergers[0]`
  0.999849 vs 0.9998, 4.9e-5 above the gate; lowest `vl_self_attention_out` 0.9745 vs 0.95). Megakernel vs the Stage-1 TTNN denoise on the
  same device: `action_pred_normalized` PCC ≥ 0.999992 on every DROID sample, per-step velocities ≥ 0.99996
  (`tests/tt/results/test_mk_e2e_backends_agree_*_n17_20260918-0428*.json`). On six additional DROID goldens the actions pass on five:
  `t2_s120` misses `llm_layer_first` (0.98998 vs 0.991) and `unnormalized_actions.joint_position` (PCC 0.99976 / max|d| 0.107 vs 0.9998 /
  0.083), a pre-existing sample-level miss also seen with the Stage-1 path and the verbatim tower; `t1_s120` misses one **megakernel-only**
  per-dim gate, `pred_velocity[k=2]` dim 9 PCC 0.99588 vs 0.996 (full-tensor PCC 0.99983 vs 0.9997 passes; the Stage-1 path passed the per-dim
  gate; recorded as a known marginal with floor 0.995 in `tests/tt/test_mk_e2e.py`; that sample's actions pass every gate)
  (`tests/tt/results/test_e2e_untraced_taps_vs_golden_{t1_s120,t2_s120}_n17_20260918-0418*.json`).
* **One embodiment, one layout, two cameras × two frames, batch 1.** DROID (`oxe_droid_relative_eef_relative_joint`), static layout
  `droid` (16×28 patches per image, `L_max` 51, `S_pad` 512). The other N1.7 layouts in `configs.py` (`g1_4x3`, `g1_synthetic_16x9`,
  `cam3_4x3`) have no traces in this image and no device validation. A client without history must send its oldest available frame
  twice (the reference wraps `iloc[-15]` at episode start, so results at t < 15 differ from NVIDIA's numbers either way).
* **Deterministic noise.** Actions are a deterministic function of (observation, seed | noise); two identical requests return
  bit-identical chunks; the 2-CQ device produces the same bits as a 1-CQ device on every probe (`test_mk_2cq_bit_equality_n17_20260918-041603.json`,
  26 rows). The default seed 42 does **not** reproduce the reference `initial_noise` (CUDA draw); `GET /demo` and the smoke test send the
  reference noise explicitly (`demo/n17/noise.json`, `seed_equivalent: null`).
* **Host cost.** N1.7's `encode` (four 256×455 frames through the cv2 chain, torchvision bicubic patching, tokenizer, cached mRoPE / vision
  tables) is 5.6 ms median / 11.6 p90 in the port benchmark on an idle host (8.5 / 19.2 in a run taken while a CPU suite was finishing) and
  8.7 ms median / 12.1 ms p90 inside the serving process, vs 1.5–2 ms for the GR1 packages; `decode` of four
  PNGs ≈ 3.4 ms. Both run outside the device lock. Inside the serving process torch's threads compete with the ttnn worker
  threads (`GR00T_TORCH_THREADS` is the knob; unset in this image). With the device time down to ~56 ms the host share of a served call is
  now about a fifth.
* **Megakernel-side open items** (kernel work, not changed by this package): the compute-core binaries are ≈ 4.5 KB under the kernel-config
  ring, so any further kernel growth needs a size check (`mk_k4b_summary.md` §5); building a second megakernel model in one process on the
  same open device after releasing the first hung the card once on N1.6 during development (`mk_default_summary.md` §10) — the server builds
  exactly one model per process, as every shipped path does.
* **First boot is cold**: the checkpoint is converted to the device layout into `/weight-cache` (2.9 GB) and every kernel incl. the megakernel
  is JIT-compiled into `/cache` (483 MB); both persist under `~/.cache/tt-model/gr00t-n1.7-3b-p150/`. Measured boot times: §2.
* **Python 3.12 image vs 3.10 host venv.** The image resolves the packages on 3.12 (`requirements.lock`); the host venv is 3.10. The
  code is 3.10/3.12 compatible; the image is the authoritative validation (§2 numbers).
* **Vendored tokenizer / processor files** (`code/gr00t_p150/assets/tokenizer/n17`, Apache-2.0 — see `PROVENANCE.md` there and
  `LICENSE-NOTICE.md`): verified bit-exact against `Qwen/Qwen3-VL-2B-Instruct` and against the golden `input_ids` / `pixel_values`.
* `tt-model curl` / `GET /v1/models` are OpenAI-shaped and are not this API; use the routes above.

## 5. Where the numbers come from

Everything in the card is measured and shipped under `code/models/experimental/gr00t/`: `benchmarks/results/bench_e2e_n17_20260918-050929.json`
(Stage-2 latency on the snapshot, 50 calls, idle host; `…-050724.json` the first run with a busier host), `benchmarks/results/bench_e2e_n17_20260918-032335.json`
(the Stage-1 TTNN path on the same tree, same day) and `…-043104.json` (the megakernel on the live tree in the same session),
`tests/tt/results/test_e2e_{predict_actions_vs_golden,traced_equals_untraced,untraced_taps_vs_golden}_n17_20260918-0452*.json` (canonical fidelity on
the snapshot commit `009de892c895`, traced == untraced), the six `test_e2e_untraced_taps_vs_golden_<sample>_n17_20260918-0418*.json` files (multi-sample,
K4b-checkpoint tree = the snapshot's code), `test_mk_e2e_backends_agree_*_n17_20260918-0428*.json` (megakernel vs TTNN denoise),
`test_mk_2cq_bit_equality_n17_20260918-041603.json`, `test_stale_state_*_n17_20260918-0420*.json`, `test_determinism_traced_and_untraced_n17_20260918-042138.json`,
`tests/tt/results/{mk_default_summary,mk_k4b_summary,mk_k5_summary,d2_summary,p_summary}.md` (the megakernel's design, measurements and decisions),
`code/gr00t_p150/demo/n17/expected.json` (fp32 golden + official bf16 reference actions for the demo step). The RTX 5090 rows are in
`GPU_COMPARISON.md`. The served numbers of this image (`timing_ms` over 50 requests, cold / warm boot) were measured on the build host as
described in §2 and recorded in `docs/publish/build-mk-n17.md` of the port project.