# Serving GR00T N1.5-3B (`nvidia/GR00T-N1.5-3B`) on Blackhole with tt-model-manager

This repo is a **tt-model container package source**: `tt-model.yaml` + the code under `code/`. `tt-model package
--container` builds the OCI image (tt-metal `668c2907575` built from source + the port + the HTTP stack),
`tt-model serve` runs it against the card, `tt-model push` publishes it to this HF repo. The server is
`code/gr00t_p150/server/app.py` (FastAPI, `kind: tt-dit-server`); what it runs on the device is the port's **Stage-1**
path — every op a TTNN op, captured once into four Metal traces (`vision`, `llm`, `adapter`, `denoise`) and replayed per
request; the persistent-megakernel denoise (Stage 2) is not part of this release.

| | |
|---|---|
| tt-metal tree the image is built from | `/home/deepgadget/experiments/gr00t/publish/tt-metal-668c2907575` — a clean local clone of tt-metal `main` @ `668c290757550588d0ce46b180c344a462a2aaf5` (`v0.79.0-dev20260914-1`), submodules populated, `git status` empty (the working clone `/home/deepgadget/experiments/gr00t/tt-metal` at the same commit has untracked dirs and would be recorded as dirty); torch pin 2.11.0 |
| port code shipped | `code/models/experimental/gr00t/` = tt-metal ref `gr00t-p150-snapshot`, commit `5dc0517ee3bec090b7ead025001a343c2f63aa7b` (parent `668c290`; frozen 2026-09-14T23:36 KST), minus `__pycache__`, `.omc` tool state and 2,001 of the 2,004 per-test result JSONs (the three N1.5 e2e results the card cites and the `d0/d1/d2/mk_k1` summaries stay) |
| weights | `nvidia/GR00T-N1.5-3B` @ `869830fc749c35f34771aa5209f923ac57e4564e`: `model-0000{1,2,3}-of-00003.safetensors` (5,448,327,040 B), `model.safetensors.index.json`, `config.json`, `experiment_cfg/metadata.json`, `LICENSE` — a pinned pointer, never in the image |
| app | `gr00t_p150.server.app:app` (uvicorn `--lifespan on`; the lifespan does weights → tokenizer → device → model → trace capture → warm-up → READY) |
| device recipe | `models.experimental.gr00t.tt.device.open_gr00t_device(trace_region_size=64 MiB, l1_small_size=32768, device_id=0)`; asserts the p150a's 11×10 compute grid and 8 DRAM banks; `TTPolicy(dtype_policy="mixed_dit", trace_layout="per_stage")` |
| serving shape | embodiment `gr1` (slot 24), layout `gr1`: 1 camera (`ego_view`) × 1 frame × 256 image tokens, ≤ 102 text tokens, LLM sequence padded to 384, state 7/7/6/6 → 16 × [7, 7, 6, 6] actions, batch 1 |

Below, `$PUB=/home/deepgadget/experiments/gr00t/publish` (this repo is `$PUB/GR00T-N1.5-3B-p150`),
`$TREE=/home/deepgadget/experiments/gr00t/tt-metal` (the built working clone, Python 3.10 venv),
`$ROOT=/home/deepgadget/experiments/tt-models` (tt-model-manager tooling).

## 1. Layout

```
tt-model.yaml                     authoring manifest (schema 5.1) -- the whole build/serve interface, incl. the model card text
SERVING.md                        this file
GPU_COMPARISON.md                 RTX 5090 vs p150a, matching definitions and caveats
LICENSE                           the NVIDIA License (weights); LICENSE-NOTICE.md says what is under which licence
media/demo_ego_view.png           the demo frame (256x256, GR1 sim robot_sim.PickNPlace traj 0 / step 100)
media/demo_observation.json       the matching raw joint state + instruction (request body minus the image)
media/demo_actions.png            served actions vs the fp32 reference for that request
code/models/experimental/gr00t/   the tt-nn port (tt-metal tree layout): common/ (host pipeline: preprocessing, prompts,
                                  normalisation, checkpoint, weight plan), tt/ (device model, layers, traces, megakernel WIP),
                                  reference/ (fp32 torch reference), tests/, benchmarks/
code/models/{common,tt_dit,demos}/   the 16 tt-metal files the port imports from the tree (source.code in tt-model.yaml)
code/gr00t_p150/                  the policy server: server/{app,schemas,smoke_test}.py, demo/n15/ (canonical demo
                                  observation + golden actions + noise), assets/tokenizer/n15/ (vendored Qwen2 tokenizer)
code/scripts/                     download_weights.sh, bench_http.py
requirements.lock                 the image venv (Python 3.12) as `uv pip freeze` -- written by `tt-model package`
tt_kernel_manifest.json, image/   written by `tt-model package`; uploaded by `tt-model push`
```

The port keeps its tt-metal-tree absolute imports (`models.experimental.gr00t.*`, `models.tt_dit.*`,
`models.common.*`): in the image the code lands at `/opt/tt-metal/<path>` with `PYTHONPATH=/opt/tt-metal`, and the
image's tt-metal copy excludes `models/`, so everything under `models/` comes from this repo's `code/` (the port) and
`source.code` (the 16 tree files). The server package `gr00t_p150` is shared with `changh95/GR00T-N1.6-3B-p150`; this
repo ships only the N1.5 assets (`demo/n15`, `assets/tokenizer/n15`).

## 2. Run on the HOST for validation (no Docker)

The tree venv (`$TREE/python_env`, Python 3.10, torch 2.11.0+cpu, ttnn editable) has everything except
`fastapi`/`uvicorn`. Do not install into the tree venv; put the HTTP stack in a side directory and prepend it to
`PYTHONPATH` (`$PUB/http-side` already holds fastapi 0.141.1 / uvicorn 0.53.0 / pydantic 2.13.5 / starlette 1.6.0):

```bash
export PATH=$HOME/.local/bin:$PATH
mkdir -p $PUB/http-side && uv pip install --python $TREE/python_env/bin/python --target $PUB/http-side fastapi uvicorn "pydantic>=2"

cd $PUB/GR00T-N1.5-3B-p150
export TT_METAL_HOME=$TREE ARCH_NAME=blackhole
export PYTHONPATH=$PWD/code:$TREE:$TREE/ttnn:$PUB/http-side      # code/ FIRST so models.experimental.gr00t is the snapshot
export TT_METAL_CACHE=$HOME/.cache/tt-metal-cache-gr00t           # warm JIT kernels of the port's device sessions
export HF_MODEL=nvidia/GR00T-N1.5-3B TT_WEIGHTS_REVISION=869830fc749c35f34771aa5209f923ac57e4564e
export GR00T_VERSION=n15 TT_MESH_SHAPE=1x1 TT_DEVICE_ID=0 TT_METAL_VISIBLE_DEVICES=0
export GR00T_TOKENIZER_DIR_N15=$PWD/code/gr00t_p150/assets/tokenizer/n15   # optional: the server picks the vendored dir itself
export GR00T_TT_CACHE=$HOME/.cache/gr00t-tt                                # host .pt plan tier + .tensorbin device tier (~6.4 GB for N1.5)

# import check, no device (what the image's verify.sh does):
$TREE/python_env/bin/python -c "import gr00t_p150.server.app as a, sys; assert a.app and 'ttnn' not in sys.modules; print('ok')"

# serve + smoke + stop -- ONE device-lock hold (the box has one p150a shared with other sessions):
DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
  '$TREE'/python_env/bin/python -m uvicorn --host 127.0.0.1 --port 20010 --lifespan on gr00t_p150.server.app:app & UV=$!
  python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:20010 --wait 1800 --out /tmp/gr00t-n15-smoke.json; RC=$?
  kill -TERM $UV; wait $UV; exit $RC'
```

Boot log landmarks (they drive `tt-model serve`'s checklist): `Loading weights: nvidia/GR00T-N1.5-3B @ 869830fc…` →
`Tokenizer files: …` → `Opening device 0 (trace_region_size=67108864, l1_small_size=32768)` → `Loading pipeline:
Gr00tTT.from_pretrained(n15, …)` → `Model built in N s (weights N s, 778 tensors, 3176 MB on device, cache path
warm|cold)` → `Warming up: capture per_stage traces on the n15 demo observation, then N traced predict(s)` →
`Warmup k/N: … ms (encode, device, decode)` → `Warm-up fidelity vs golden fp32: actions PCC … ` → `Warmup complete
(capture s, first ms, steady ms; boot s) -- per_stage traces ['vision', 'llm', 'adapter', 'denoise']` → uvicorn
`Application startup complete`. Any failure raises and uvicorn exits non-zero (no CPU fallback); the boot also **fails**
if the warm-up actions' PCC vs the shipped golden is below `GR00T_WARMUP_PCC_MIN` (0.999). SIGTERM / Ctrl-C:
`Releasing traces and device tensors` → `Closing device`.

Measured host boots (warm `~/.cache/gr00t-tt` and warm `TT_METAL_CACHE`; `$PUB/logs/server_n15_*.log`): first boot of a
process 14.6 s to READY (model built 9.3 s incl. 9.2 s reading the `.tensorbin` tier, warm + capture 2.5 s, 5 traced
warm-ups 44.4–44.8 ms); an immediately following boot 4.5 s (0.6 s model build with the tiers in the page cache); on the
frozen snapshot worktree 5.2 s (`server_n15_snapshot_20260914-235713.log`). Kernels are rebuilt once for a new
`TT_METAL_HOME` path (~42 s for the 355 kernels of the four traces, `docs/publish/snapshot.md` §2.3).

Expected smoke line (host, snapshot worktree, `$PUB/logs/smoke_n15_snapshot_20260914-235713.log`):

```
PASS GR00T-N1.5-3B-p150 n15: PCC(actions)=0.999982 max|d|=0.0289 PCC(action_pred_valid)=0.999976 repeat_maxdiff=0.0e+00
     seed_path_ok device_ms=42.95/42.16 total_ms=47.22/44.83 wall_ms=50
     per_group[left_arm=0.99995/0.0214 right_arm=0.99997/0.0172 left_hand=0.94335/0.0179 right_hand=0.99999/0.0289]
```

The per-group PCC / max|d| equal the port's own device test (`code/models/experimental/gr00t/tests/tt/results/
test_e2e_predict_actions_vs_golden_n15_20260914-230743.json`) to the printed precision; the traced policy is
deterministic, so every run reproduces them bit-exactly.

Offline overrides: `GR00T_WEIGHTS_DIR=<dir with the snapshot files>` (shimmed into a private hub cache so the port's
`snapshot_dir()` resolves to it), `GR00T_TOKENIZER_DIR=<dir>`.

Offline checks that need no device:

```bash
cd $PUB/GR00T-N1.5-3B-p150
# manifest + launcher preview (from the repo dir -- `root: code` is CWD-relative); must print VALID
$ROOT/.venv/bin/python -c "from tt_kernel.container_manifest import load_container_manifest; m = load_container_manifest('tt-model.yaml', check_sources=True); p = m.resolve_profile(); print('VALID', m.name, m.kind, p.hardware, p.mesh_device, m.weights_ref)"
# Python 3.12 resolution of runtime.packages (what the image does) -- must keep numpy<2 and torch 2.11.0+cpu
uv venv --python 3.12 $PUB/depcheck-n15 -q
uv pip install --python $PUB/depcheck-n15/bin/python --dry-run torch==2.11.0 fastapi uvicorn "pydantic>=2" pillow "numpy>=1.24.4,<2" \
   safetensors huggingface_hub "transformers==5.12.1" "opencv-python-headless==4.8.1.78" "torchvision==0.26.0" pytest \
   --extra-index-url https://download.pytorch.org/whl/cpu --index-strategy unsafe-best-match
#   -> torch 2.11.0+cpu, torchvision 0.26.0+cpu, numpy 1.26.4, transformers 5.12.1, opencv-python-headless 4.8.1.78 (cp312 wheel), pytest 9.1.1 (logs/uv-dryrun-n15.log)
```

## 3. Package, serve, push (Docker)

Rootless Docker on this box needs `source $ROOT/bin/docker-env.sh` first (PATH + `DOCKER_HOST`; the bare `docker`
on PATH is podman). **Run every `tt-model` command from this directory**: `source.tt_metal` and
`extra_code[].root: code` resolve against the process CWD. `--out` points outside any git checkout because `stage()`
deletes `<out>/gr00t-n1.5-3b-p150` before rebuilding. The manifest `name` is the slug `gr00t-n1.5-3b-p150`, so that is
the build directory, the cache directory (`~/.cache/tt-model/gr00t-n1.5-3b-p150/`) and the generated card's title.

```bash
source $ROOT/bin/docker-env.sh
cd $PUB/GR00T-N1.5-3B-p150

$ROOT/.venv/bin/tt-model package --container tt-model.yaml --out $PUB/build     # tt-metal C++ build (ccache) + venv + verify.sh
# device: serve -> smoke -> bench -> stop inside ONE lock hold (`serve` returns after READY and leaves the container
# running on the card; a second with-device.sh caller would probe a busy card and reset it)
DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
  source '$ROOT'/bin/docker-env.sh
  '$ROOT'/.venv/bin/tt-model serve '$PUB'/build/gr00t-n1.5-3b-p150/tt_kernel_manifest.json 2>&1 | tee '$PUB'/logs/serve-n15.log || exit 1
  PORT=$(grep -o "127.0.0.1:[0-9]*" '$PUB'/logs/serve-n15.log | head -1 | cut -d: -f2)
  python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:$PORT --out '$PUB'/logs/smoke-n15.json; RC=$?
  python3 code/scripts/bench_http.py --url http://127.0.0.1:$PORT --reps 50 --warmup 10 --out '$PUB'/logs/bench-http-n15.json
  '$ROOT'/.venv/bin/tt-model logs changh95/GR00T-N1.5-3B-p150 > '$PUB'/logs/container-n15.log 2>&1
  '$ROOT'/.venv/bin/tt-model stop changh95/GR00T-N1.5-3B-p150; exit $RC'
$ROOT/.venv/bin/tt-model push $PUB/build/gr00t-n1.5-3b-p150 --publish
```

`serve` pre-downloads the pinned files (a cache hit on this host), then `docker run --user 0:0 --device
/dev/tenstorrent --ipc host` with `/dev/hugepages-1G`, `~/.cache/huggingface` at `/hf` (rw, `HF_HOME=/hf`) and
`~/.cache/tt-model/gr00t-n1.5-3b-p150/{cache,weights,tensors}` at `/cache` (`TT_METAL_CACHE`, JIT kernels),
`/weight-cache` (`TT_DIT_CACHE_DIR`; the port's weight tiers go to `/weight-cache/gr00t-tt`, ~6.4 GB) and
`/tensor-cache`; it exports exactly `HF_MODEL=nvidia/GR00T-N1.5-3B`, `MESH_DEVICE=P150`, `TT_MESH_SHAPE=1x1` plus
`serve.env` from `tt-model.yaml`, publishes the first free port from 20000 and waits ≤ 1800 s for `Application startup
complete`. tt-cli users: `tt serve changh95/GR00T-N1.5-3B-p150` / `tt model stop changh95/GR00T-N1.5-3B-p150`
(after the push).

`push` makes `code/` and `image/` on the Hub exactly the staged trees and replaces `README.md` with the generated card
(everything worth keeping lives in `card.description` / `card.quickstart` of `tt-model.yaml`); `media/`, `SERVING.md`,
`GPU_COMPARISON.md`, `LICENSE`, `LICENSE-NOTICE.md`, `tt-model.yaml` and `.gitattributes` at the repo root survive; the
`license` / `pipeline_tag` / `base_model` front matter is restored afterwards with `huggingface_hub.metadata_update`.

## 4. Container validation (the image built from this manifest)

Image `tt-model/gr00t-n1.5-3b-p150:443501bd3b7a` (`sha256:443501bd3b7afce579bf743eeedd64809886653c29a9d7125614e8451cd61a7b`,
3.2 GB as 46 OCI blobs), built 2026-09-14 15:17–15:30 UTC from the manifest above (`tt-model package`: tt-metal C++ build
with a warm ccache ≈ 7 min, Python 3.12 venv, all 17 `verify:` lines + the launcher's own passed inside the image;
13 min 17 s in total). Validated on the p150a on 2026-09-15 00:56–01:02 KST: three `tt-model serve` boots of this exact
image, each followed by `smoke_test.py`, a 50-request `bench_http.py` and `tt-model stop`, all inside one device-lock hold
per run (`publish/logs/validate-n15.run2.log` = boots 1–2, `validate-n15.run3.log` = boot 3; per-boot files
`{serve,smoke,bench-http,container,info}-n15.{2,3}.*`; boot 1's per-file artefacts were overwritten by boot 3 and survive
only inside `validate-n15.run2.log`).

| boot | caches (`~/.cache/tt-model/gr00t-n1.5-3b-p150/`) | host load | `tt-model serve` → READY | container boot (lifespan) | smoke | `bench_http.py` (10 warm-up + 50): device / server-side median (p90) | `tt-model stop` |
|---|---|---|---|---|---|---|---|
| 1 (**cold**) | `weights/` and `cache/` empty before; 6.4 GB + 411 MB after | two concurrent tt-metal image builds + CPU tests (load avg 16–110) | **1 min 43 s** (103.6 s wall) | 100.9 s: device open ≈ 4 s → `Model built in 8.9 s (… cache path cold)` (checkpoint → host plan → `.tensorbin`) → trace capture **87.3 s** incl. the JIT of the four traces' kernels (`riscv-tt-elf-g++` lines in the container log) → 3 warm-ups 50.9 / 45.8 / 101.5 ms | **PASS** — PCC(actions) 0.999982, max\|d\| 0.0289; PCC(`action_pred_valid`) 0.999976; device 46.96 / 42.21 ms, total 54.42 / 48.21 ms | 44.27 (55.35) / 53.59 (91.12) ms; encode median 3.2, p90 41 ms; client wall 58.1 (94.5) | clean SIGTERM shutdown 4.9 s |
| 2 (warm, immediately after) | reused | same loaded host | 20.2 s (21.3 s wall) | 15.1 s: `Model built in 0.6 s (… cache path warm)`, capture 6.6 s, warm-ups 47.3 / 119.4 / 65.3 ms (encode spikes = host contention) | **PASS** — same fidelity; device 50.88 / 42.59 ms, total 67.44 / 45.45 ms | 44.51 (55.76) / 58.14 (96.86) ms; encode median 5.8, p90 38 ms; client wall 64.4 (104.2) | clean 3.1 s |
| 3 (warm, quiet host) | reused | load avg ≈ 9, no other device or build work | **9.7 s** (10.0 s wall) | 7.6 s: model 0.6 s, capture 3.2 s, warm-ups 44.5 → 44.3 ms | **PASS** — same fidelity; device 42.99 / 42.19 ms, total 47.31 / 44.95 ms, client wall 50 ms | **42.43 (43.18) / 45.40 (49.20) ms**, min 41.66 / 44.46; decode 1.06, encode 1.63, decode_actions 0.14; client wall 47.6 (51.8) | clean 2.0 s |

Fidelity is identical in every boot and identical to the host runs of §2 — the served `actions` JSON of the container is
byte-for-byte the host run's (`smoke-n15.3.json` vs `smoke_n15_snapshot_20260914-235713.json`) — and equal to the
port's device test `tests/tt/results/test_e2e_predict_actions_vs_golden_n15_20260914-230743.json` to the printed precision:
`left_arm` 0.99995 / 0.0214, `right_arm` 0.99997 / 0.0172, `left_hand` 0.94335 / 0.0179 (abs-gated), `right_hand`
0.99999 / 0.0289. The boot-3 benchmark is the card's "served" row; the device median 42.4 ms is 0.7 ms above the port's
own `bench_e2e` device figure (41.68 ms: upload + traces + readback) and the loaded-host runs show that the extra
latency and the p90 tails sit in the host-side stages (`encode`), not on the device.

`GET /info` inside the container reports `weights.snapshot_dir = /hf/hub/models--nvidia--GR00T-N1.5-3B/snapshots/869830fc…`
(the mounted HF cache), `source.tt_metal = {commit 668c290…, ttnn_dist_version 0.65.2.dev10011}`,
`source.port_snapshot_commit = 5dc0517ee3b…`, `device_facts` grid 11×10 / DRAM 8×1 (`info-n15.3.json`).

**Device-sharing note for this host.** The first validation attempt (00:32–00:40 KST) ran while another session's
container was still holding the card after that session's lock hold had ended; `bin/with-device.sh`'s probe hung and
reset the card twice, then refused to run. Since then every device entry goes through
`publish/scripts/run_validation_n15_when_free.sh`, which waits until `docker ps` shows no `tt-model-*` container and the
lock is free. Use the manifest path with `tt-model stop` / `logs` for an un-pushed package (`tt-model stop
changh95/GR00T-N1.5-3B-p150` only works after `pull`).

## 5. Request / response contract

| route | returns |
|---|---|
| `GET /health` | `{"status": "ok" \| "starting", "model": "GR00T-N1.5-3B-p150", "device": "blackhole:0"}` — 200 always; `ok` only after the warm-up |
| `GET /info` | model / version / task / hardware / `stage` (`stage1-ttnn-traces` + megakernel note) / `weights` (repo, revision, snapshot dir actually loaded) / `source` (port snapshot commit, server version, tt-metal commit, ttnn dist version) / `inputs` (the full observation contract) / `outputs` / `noise.default_seed` / `limits` / `device_facts` (11×10 grid, 8 DRAM banks) / `warmup_latency_ms` / `warmup_fidelity_vs_golden` / `license` |
| `GET /v1/models` | OpenAI-shaped stub `{"object": "list", "data": [{"id": "nvidia/GR00T-N1.5-3B", "object": "model", "owned_by": "changh95"}]}` so the tt-model ready card / `tt-model curl` do not 404; this is not a chat API |
| `GET /demo` | `{"request": <ready-to-post demo body>, "expected": <fp32 reference actions + gates>}` |
| `POST /predict` | one observation → one 16-step action chunk (below) |

`POST /predict` request (JSON; unknown fields → 422):

| field | type | meaning |
|---|---|---|
| `images` | `{"ego_view": <b64 PNG/JPEG>}` (or a one-element list per camera) | exactly the GR1 camera; any resolution with sides in [64, 4096] px; the server applies the checkpoint's eval chain (x/255 → 0.95 centre crop → bilinear antialiased 224×224 → Eagle2.5 normalisation), bit-exact vs the reference preprocessing |
| `state` | `{"left_arm": [[7]], "right_arm": [[7]], "left_hand": [[6]], "right_hand": [[6]]}` | raw physical joint state, one time step (`(D,)` or `(1, D)`), finite floats; sin/cos-encoded and padded to 64 dims on the host |
| `instruction` | `str` (aliases `language`, `prompt`) | task text, 1–2000 chars, ≤ 102 BPE tokens after the port's prompt build (`repr([instruction])` inside the Eagle2 chat prompt); longer → 400, never truncated |
| `embodiment` | `"gr1"`, optional | must equal the server's; anything else → 400 |
| `seed` | int ≥ 0, optional | CPU-generator seed of the `[1, 16, 32]` flow-matching noise; default 0 (the deployed policy's) |
| `noise` | `[[32 floats] × 16]`, optional | explicit initial noise; mutually exclusive with `seed` |
| `state_dtype` | `"float64"` (default) \| `"float32"` | dtype of the state before normalisation (the GR1 dataset stores float64) |
| `return_normalized` | bool, default false | also return the model-space chunk `action_pred_normalized` (`[16][32]`) and `action_pred_valid_hd` (`[16, 26]`) |

Response (200):

```json
{"actions": {"left_arm": [[7 floats] x 16], "right_arm": [...], "left_hand": [...], "right_hand": [...]},
 "action_horizon": 16, "action_keys": ["left_arm", "right_arm", "left_hand", "right_hand"], "action_dims": {"left_arm": 7, "right_arm": 7, "left_hand": 6, "right_hand": 6},
 "normalized": false, "embodiment": "gr1", "embodiment_id": 24, "version": "n15", "model": "GR00T-N1.5-3B-p150", "layout": "gr1",
 "images": {"ego_view": {"frames": 1, "received_hw": [256, 256]}}, "seq_len": 296, "prompt_tokens": 14, "state_dtype": "float64",
 "noise_source": "seed" | "client", "seed": 0 | null,
 "timing_ms": {"decode": 1.5, "encode": 2.6, "device": 43.0, "decode_actions": 0.1, "total": 47.2}}
```

`actions` are physical joint targets (radians) — `(y + 1) / 2 · (max − min) + min` with the checkpoint's
`experiment_cfg/metadata.json` GR1 statistics — exactly what `Gr00tPolicy.get_action` returns. `timing_ms`: `decode` =
base64 + image decode + validation; `encode` = the port's host preprocessing; `device` = `predict_normalized` (static-shape
asserts, input writes, 4 × `execute_trace`, one blocking readback); `decode_actions` = un-normalisation; `total` = the
whole handler. Errors: **400** structural problems (`{"detail": {"errors": [...], "inputs": {...}}}`), undecodable image,
bad state / instruction / noise, other embodiment; **503** while starting; **500** `Type: text` on a device failure.
Handlers are synchronous and serialised on one lock; batch 1.

## 6. Environment (read in the lifespan, never at import)

| variable | container value (`serve.env`) | meaning |
|---|---|---|
| `HF_MODEL` / `TT_WEIGHTS_REVISION` | `nvidia/GR00T-N1.5-3B` (launcher) / `869830fc…` | must be the version's pinned repo and sha (both are asserted against the port's own `configs.HF_REPOS` / `HF_SNAPSHOT_SHAS`) |
| `GR00T_VERSION` | `n15` | the server code is shared by both GR00T packages and never guesses |
| `GR00T_EMBODIMENT` / `GR00T_LAYOUT` | `gr1` / `gr1` | the only device-validated layout of this release |
| `GR00T_POLICY` / `GR00T_TRACE_LAYOUT` | `mixed_dit` / `per_stage` | `TTPolicy` knobs; `bf16` / `mixed` and `two` exist in the port but are not the benchmarked defaults |
| `GR00T_TRACE_REGION_SIZE` / `GR00T_L1_SMALL_SIZE` | default 67108864 / 32768 | the port's validated `open_gr00t_device` values |
| `GR00T_TOKENIZER_DIR_N15` | `/opt/tt-metal/gr00t_p150/assets/tokenizer/n15` | vendored Qwen2 tokenizer (not in the weights repo); `GR00T_TOKENIZER_DIR` overrides |
| `GR00T_TT_CACHE` | `/weight-cache/gr00t-tt` | host `.pt` plan tier + `.tensorbin` device tier (persisted under `~/.cache/tt-model/gr00t-n1.5-3b-p150/weights`) |
| `GR00T_PROJECT_ROOT` / `GR00T_GOLDEN_ROOT` | `/nonexistent` | the port's dev-box defaults; nothing on the serve path needs them |
| `GR00T_WARMUP_RUNS` / `GR00T_WARMUP_PCC_MIN` | `3` / default 0.999 | traced predicts before READY; boot fails below the PCC floor vs the shipped golden |
| `GR00T_DEFAULT_SEED`, `GR00T_TORCH_THREADS`, `GR00T_WEIGHTS_DIR` | unset | request default seed (0), torch intra-op threads, offline snapshot dir |
| `TT_MESH_SHAPE`, `TT_DEVICE_ID`, `TT_METAL_VISIBLE_DEVICES` | `1x1` (launcher), `0`, `0` | one chip; any other mesh shape is refused |

## 7. Caveats

- **Stage-1 path only.** TTNN ops in Metal traces; the persistent-megakernel denoise (K1 streaming kernel: 464 GB/s bf16 /
  414 GB/s bfp8, `code/models/experimental/gr00t/tests/tt/results/mk_k1_summary.md`) is not integrated into `Gr00tTT` and not served.
- **Shape-locked traces.** One embodiment (`gr1`), one camera, one frame, ≤ 102 text tokens (LLM S padded to 384), batch 1.
  Other embodiments / cameras / longer instructions → 400 before anything reaches the device. Concurrent requests queue on one lock.
- **Deterministic noise.** Actions are a deterministic function of (observation, seed | noise); the reference noise of the
  demo is not seed-reproducible, so the smoke test posts it as `noise`. Same observation + same noise → bit-identical actions.
- **Base checkpoint.** Outputs are the GR1 sim data-config actions of the released N1.5 checkpoint; a real robot needs NVIDIA's post-training.
- **First container boot is cold**: the port converts the 5.4 GB checkpoint into its `.pt` plan tier and the `.tensorbin`
  device tier under `/weight-cache/gr00t-tt` and JITs the kernels of the four traces into `/cache`; both persist under
  `~/.cache/tt-model/gr00t-n1.5-3b-p150/`, so later boots are seconds (measured values in §4).
- **Python 3.12 image vs 3.10 tree venv.** Host runs are an approximation; the image is authoritative (its `verify.sh`
  imports the app and the port with no device; the served smoke in §4 is the proof).
- **Weights licence.** `nvidia/GR00T-N1.5-3B` is under the NVIDIA License (non-commercial: research or evaluation use);
  the server reports it under `/info -> license`. See `LICENSE-NOTICE.md`.
- `tt-model curl` and the ready card's `/v1/models` hint are OpenAI-shaped and are not this API; use the routes above.