Restore card metadata (license, pipeline_tag, base_model, tags) and upload tt-model authoring files

README.md = the finished card (front matter: license other / nvidia-license / license_link, pipeline_tag robotics, base_model, descriptive tags + tt-model-catalog; quickstart paragraph simplified to the two bullets as in changh95/rf-detr-p150 and changh95/pi05-base-p150). Root files tt-model.yaml, SERVING.md, GPU_COMPARISON.md, LICENSE, LICENSE-NOTICE.md, requirements.lock and media/* from the staging dir; code/ and image/ untouched (as pushed by tt-model push).

.gitattributes

@@ -48,3 +48,4 @@ image/blobs/sha256/e72991f8dede710d8265ab8a49bd37bddfd9b4958934fa6f69e7f15c3c904
 image/blobs/sha256/20cad74cf2e0718048ce7cad5032e4837d231af0ead9812353883170b27a8c89 filter=lfs diff=lfs merge=lfs -text
 image/blobs/sha256/5a589f1aa835965af711da5a5217b53178226a35f12d46b268f651f27f50ab66 filter=lfs diff=lfs merge=lfs -text
 image/blobs/sha256/96d36b6a923d9a423d4d024e1d8b2865ba17630ff6c943b8b5399135262a392c filter=lfs diff=lfs merge=lfs -text
+media/demo_actions.png filter=lfs diff=lfs merge=lfs -text

GPU_COMPARISON.md

@@ -0,0 +1,130 @@
+# GR00T-N1.5-3B-p150 — Blackhole p150a vs RTX 5090 (same host, same weights, same observation)
+
+Dates: GPU rows 2026-09-14, p150a rows 2026-09-14/15. Facts only: every number is copied from the file named in its
+row. The two sides run **different code**: the GPU rows are NVIDIA's unmodified `Gr00tPolicy` (the official inference
+path, eager PyTorch); the p150a rows are this package's tt-nn port (Stage-1: TTNN ops replayed from four Metal traces).
+Read the caveats before comparing.
+
+## What was run
+
+| | RTX 5090 | Blackhole p150a |
+|---|---|---|
+| Code | NVIDIA [Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) tag `n1.5-release` (`4af2b622`), `Gr00tPolicy.get_action`, `FourierGr1ArmsOnlyDataConfig`, unmodified | `code/models/experimental/gr00t` (frozen snapshot `5dc0517ee3b`), `Gr00tTT` with `TTPolicy(dtype_policy="mixed_dit", trace_layout="per_stage")`, served by `code/gr00t_p150/server/app.py` |
+| Weights | `nvidia/GR00T-N1.5-3B` @ `869830fc749c35f34771aa5209f923ac57e4564e`, bf16 shards from the HF cache | same snapshot; converted once into the device layout (bf16 everywhere, bfp8_b for the 16 DiT blocks' matmul weights) |
+| Precision | official path: `torch.autocast(bfloat16)` + flash-attention 2 (`flash_attention_2` is hard-coded in the vendored Eagle2.5 config) | bf16 activations; HiFi2 matmul fidelity in the backbone/adapter, bfp8_b DiT weights; fp32 host pre/post-processing |
+| Observation | the port's canonical golden sample: one 256×256 `ego_view` frame (GR1 sim `robot_sim.PickNPlace` traj 0 / step 100), GR1 state (7/7/6/6), instruction "pick the pear from the counter and place it in the plate", batch 1, fixed initial noise | identical inputs (`code/gr00t_p150/demo/n15/`); LLM sequence 296 tokens padded to the static 384 |
+| GPU / host | NVIDIA GeForce RTX 5090 (sm_120), driver 580.126.18; settled at P1, 2985 MHz SM / 13,801 MHz mem, 190–250 W during the loop | one p150a (11×10 compute grid, 8 DRAM banks), same host, `/dev/tenstorrent/0` |
+| Software | Python 3.10.20, torch 2.7.1+cu128, torchvision 0.22.1+cu128, transformers 4.51.3, flash-attn 2.8.3 (official prebuilt wheel; its kernels verified on sm_120 against SDPA to 1 bf16 ulp), cuDNN 9.7.1 | tt-metal main `668c2907575` (`v0.79.0-dev20260914-1`), torch 2.11.0+cpu; image: Python 3.12 |
+| Loop | `time_n15.py`: 3 rounds × (20 warm-ups + 20 timed calls), `torch.cuda.synchronize()` around every call, GPU otherwise idle; **best median of the 3 rounds** (the round medians agree within 0.4 ms) | `benchmarks/bench_e2e.py`: 10 warm-ups + 50 timed calls, host wall clock (`perf_counter`), trace replays enqueued non-blocking with one synchronize per call; a second 50-call pass with a synchronize after every trace gives the per-trace split; **median** (p90 in brackets) |
+| Source files | `docs/research/ref-env-n15.md` §6 (project research notes; timing JSON `golden/n15/timing.json`) — not shipped in this repo | `code/models/experimental/gr00t/benchmarks/results/e2e_stage1_n15.json` (shipped); served numbers from this image: `SERVING.md` §"Container validation" |
+
+## Latency, batch 1 (ms)
+
+### RTX 5090, official eager bf16 + flash-attn 2 path (`ref-env-n15.md` §6)
+
+| stage | median ms |
+|---|---:|
+| `Gr00tPolicy.get_action` end-to-end (preprocess + model + un-normalise, includes a `deepcopy` of the observation) | **58.0** |
+| preprocessing only (`apply_transforms`: crop/resize, PIL, tokenizer, image processor — CPU) | 2.17 |
+| `model.get_action` (autocast bf16) | 55.4 |
+| backbone `EagleBackbone.forward` (ViT + mlp1 + 12 × Qwen3 + lm_head) | 20.8 |
+| — `extract_feature` (SigLIP ViT + mlp1) | 11.4 |
+| — Qwen3 12 layers + lm_head (difference) | ≈ 9.4 |
+| action head `get_action` | 34.6 |
+| — `process_backbone_output` (vlln + 4 VL self-attention blocks) | 1.68 |
+| — 4 × (action encoder + 16 DiT blocks + action decoder) (difference) | ≈ 32.9 ⇒ ≈ 8.2 per denoising step |
+
+An fp32 (no autocast) model-only run measured 64–100 ms across runs and was not re-measured (not a target). No
+`torch.compile` / TensorRT run was made for N1.5 (see caveats). NVIDIA's own N1.5 README quotes, for a single **H100**:
+VLM backbone 23.18 ms, action head 4 × 6.18 = 24.7 ms, full model 47.88 ms.
+
+### Blackhole p150a, Stage-1 trace path (`e2e_stage1_n15.json`, `stages.<name>.value` / `.p90`)
+
+| stage | median ms (p90) |
+|---|---:|
+| **e2e** = host encode + upload + 4 traces + readback + host decode | **43.36** (44.08) |
+| device = upload + traces + readback | 41.68 (42.25) |
+| 4 trace replays, one synchronize | 37.83 (37.87) |
+| — trace `vision` (SigLIP tower + projector) | 8.71 |
+| — trace `llm` (Qwen3-1.7B[:12]) | 8.75 |
+| — trace `adapter` (VL-LN + 4 VL-SA blocks + K/V hoist for the DiT cross-attention + state encoder) | 4.47 |
+| — trace `denoise` (4 Euler steps × (action encoder + 16 DiT blocks + decoder)) | 16.01 |
+| upload (host inputs → device buffers, incl. synchronize) | 3.77 (4.33) |
+| readback of `action_pred` (D2H) | 0.09 |
+| host encode (`Observation` → `ModelInputs`: image chain, prompt + tokenizer, sin/cos state) | 1.52 (1.82) |
+| host decode (un-normalise) | 0.17 |
+| e2e of the split pass (synchronize after every trace) | 43.92 (44.68) |
+
+Same run re-executed on the frozen snapshot worktree before packaging: e2e 43.01, traces 37.82 (vision 8.70 · llm 8.73 ·
+adapter 4.44 · denoise 15.99) — every trace median within 0.03 ms of the table above (`docs/publish/snapshot.md` §5.1).
+
+Served over HTTP by this image (`tt-model serve` → `scripts/bench_http.py`, 10 warm-ups + 50 timed demo requests, quiet
+host): **device 42.43 ms (p90 43.18) · server-side 45.40 ms (p90 49.20)**, client wall 47.6 ms on the same host —
+`SERVING.md` §4 has the three boots, incl. two under a heavily loaded host (device 44.3–44.5, server-side 53.6–58.1 ms
+medians) and the cold / warm boot times. The served device figure is 0.7 ms above `bench_e2e`'s device row (41.68 ms).
+
+## Fidelity vs the fp32 reference (same observation, same initial noise)
+
+The fp32 reference is NVIDIA's own `Gr00tPolicy` run on CPU in fp32 (math SDPA). Both bf16 paths are measured
+against it; PCC in float64 over the flattened tensor.
+
+| tensor | NVIDIA bf16 + FA2 on the RTX 5090 (`ref-env-n15.md` §5.1) | p150a Stage-1 path (`tests/tt/results/test_e2e_untraced_taps_vs_golden_n15_20260914-230740.json`) | gate applied to the p150a (multi-sample rule: twice the worst deviation of NVIDIA's bf16 path over 9 observations) |
+|---|---:|---:|---:|
+| `backbone_features` (post final RMSNorm) | 0.999006 | 0.995863 | 0.995 |
+| `vl_self_attention_out` (= DiT `encoder_hidden_states`) | 0.997315 | 0.991189 | 0.991 |
+| `vit_post_layernorm_out` | 0.999000 | 0.995057 | 0.993 |
+| `action_pred_normalized` (1×16×32) | 0.999991 | 0.999980 (max\|d\| 0.0143) | 0.9999 (+ abs 0.027) |
+| `unnormalized_action.left_arm` / `right_arm` / `right_hand` | 0.99998 / 0.99998 / 0.99999 | 0.999947 / 0.999972 / 0.999994 (max\|d\| 0.021 / 0.017 / 0.029 rad) | 0.9999 (+ abs 0.050 / 0.038 / 0.058) |
+| `unnormalized_action.left_hand` (range 0.099 rad in this sample — PCC not meaningful) | 0.9525 | 0.943352 (max\|d\| 0.018 rad) | abs 0.046 only |
+
+Over the multi-sample set (9 observations, `golden-multisample.md` §4.1) NVIDIA's bf16 path itself drops to
+`backbone_features` 0.99771, `vl_self_attention_out` 0.99573 and `action_pred_normalized` 0.999987; the p150a is held
+to gates derived from those minima. All 58 rows of the p150a run pass; the final action chunk agrees with the fp32
+reference to PCC ≥ 0.99995 on every gated group.
+
+## Comparison with matching definitions
+
+Ratio = RTX 5090 ms / p150a ms (> 1 means the p150a is faster).
+
+| row | RTX 5090 (official eager bf16 + FA2) | p150a (Stage-1 traces) | ratio |
+|---|---:|---:|---:|
+| end-to-end: `Gr00tPolicy.get_action` vs port e2e (both: host preprocessing + model + un-normalisation, batch 1) | 58.0 | 43.36 | **1.34** |
+| model only: `model.get_action` vs port device (upload + traces + readback) | 55.4 | 41.68 | **1.33** |
+| backbone: `EagleBackbone.forward` vs traces `vision` + `llm` | 20.8 | 17.46 | 1.19 |
+| action head: `ActionHead.get_action` (VL-SA + 4 denoising steps) vs traces `adapter` + `denoise` | 34.6 | 20.48 | 1.69 |
+| host preprocessing: `apply_transforms` vs port encode | 2.17 | 1.52 | 1.43 |
+
+Reading: at batch 1 the eager GPU path is kernel-launch bound (the action head is 60 % of its time at 1/3 of the FLOPs,
+≈ 16 blocks × ~15 kernels per denoising step); the p150a's traces replay the same graph with no launch overhead, which is
+where the 1.7× on the head comes from, while the two backbones are within 20 %.
+
+## Caveats — read before quoting a ratio
+
+1. **Eager GPU only.** No `torch.compile` or TensorRT run was made for N1.5. For N1.6 on the same GPU,
+   `torch.compile(DiT.forward, mode="max-autotune")` cut the official e2e from 63.1 ms to 33.1 ms (`ref-env-n16.md` §5)
+   and NVIDIA's N1.6 README lists 37 ms compiled on an RTX 5090; a compiled N1.5 would very likely be faster than this
+   Stage-1 p150a path. The p150a's own next step (the persistent-megakernel denoise, Stage 2) is not in this image either.
+2. **Different code, similar precision class.** GPU: bf16 autocast + FA2 (NVIDIA's shipped path). p150a: bf16 with
+   bfp8_b DiT weights and HiFi2 matmul fidelity. Both are compared against the same fp32 reference above; neither is fp32.
+3. **Stage boundaries differ slightly.** The GPU "backbone" includes `lm_head`; the p150a `adapter` trace holds the VL
+   self-attention (GPU: inside the action head) plus the K/V hoist and state encoder. Only the e2e and model rows are
+   like-for-like.
+4. **Loops differ** (3 × 20 timed calls best-median vs 50 timed calls median; `cuda.synchronize` per call vs one
+   host-side synchronize per call). Both exclude HTTP/JSON framing. The GPU e2e includes a `deepcopy` of the observation
+   (NVIDIA's code); the p150a e2e includes the 3.8 ms input upload.
+5. **Power / efficiency: not compared.** GPU draw during the loop was 190–250 W (idle not recorded in this run); p150a
+   power was not measured.
+6. **Batch 1, one sample.** Timings are for the canonical observation; the port's traces are shape-locked, so any
+   GR1 request with ≤ 102 text tokens takes the same device time (the sequence is padded to 384).
+
+## Reproduce
+
+```bash
+# p150a, served (any host with the package pulled):
+tt serve changh95/GR00T-N1.5-3B-p150
+python3 code/scripts/bench_http.py --url http://127.0.0.1:20000 --reps 50 --warmup 10      # server-side timing_ms + client wall
+# p150a, the port benchmark (needs a built tt-metal 668c2907575 tree, the pinned weights and the project's golden observation):
+python code/models/experimental/gr00t/benchmarks/bench_e2e.py --version n15 --out <dir>
+# RTX 5090: Isaac-GR00T n1.5-release + the environment of ref-env-n15.md §1.2; the timing script (tt_golden/time_n15.py)
+# lives in the project's research tree, not in this repo.
+```

LICENSE

@@ -0,0 +1,61 @@
+NVIDIA License
+1. Definitions
+“Licensor” means any person or entity that distributes its Work.
+“Work” means (a) the original work of authorship made available under this license, 
+which may include software, documentation, or other files, and (b) any additions to or 
+derivative works  thereof  that are made available under this license.
+The terms “reproduce,” “reproduction,” “derivative works,” and “distribution” have the 
+meaning as provided under U.S. copyright law; provided, however, that for the purposes
+of this license, derivative works shall not include works that remain separable from, or 
+merely link (or bind by name) to the interfaces of, the Work.
+Works are “made available” under this license by including in or with the Work either (a)
+a copyright notice referencing the applicability of this license to the Work, or (b) a copy 
+of this license.
+2. License Grant
+2.1 Copyright Grant. Subject to the terms and conditions of this license, each 
+Licensor grants to you a perpetual, worldwide, non-exclusive, royalty-free, 
+copyright license to use, reproduce, prepare derivative works of, publicly display, 
+publicly perform, sublicense and distribute its Work and any resulting derivative 
+works in any form.
+3. Limitations
+3.1 Redistribution. You may reproduce or distribute the Work only if (a) you do so 
+under this license, (b) you include a complete copy of this license with your 
+distribution, and (c) you retain without modification any copyright, patent, 
+trademark, or attribution notices that are present in the Work.
+3.2 Derivative Works. You may specify that additional or different terms apply to 
+the use, reproduction, and distribution of your derivative works of the Work (“Your
+Terms”) only if (a) Your Terms provide that the use limitation in Section 3.3 
+applies to your derivative works, and (b) you identify the specific derivative works 
+that are subject to Your Terms. Notwithstanding Your Terms, this license (including
+the redistribution requirements in Section 3.1) will continue to apply to the Work 
+itself.
+3.3 Use Limitation. The Work and any derivative works thereof only may be used 
+or intended for use non-commercially. Notwithstanding the foregoing, NVIDIA 
+Corporation and its affiliates may use the Work and any derivative works 
+commercially. As used herein, “non-commercially” means for research or 
+evaluation purposes only.
+3.4 Patent Claims. If you bring or threaten to bring a patent claim against any 
+Licensor (including any claim, cross-claim or counterclaim in a lawsuit) to enforce 
+any patents that you allege are infringed by any Work, then your rights under this 
+license from such Licensor (including the grant in Section 2.1) will terminate 
+immediately.
+3.5 Trademarks. This license does not grant any rights to use any Licensor’s or its 
+affiliates’ names, logos, or trademarks, except as necessary to reproduce the 
+notices described in this license.
+3.6 Termination. If you violate any term of this license, then your rights under this 
+license (including the grant in Section 2.1) will terminate immediately.
+4. Disclaimer of Warranty.
+THE WORK IS PROVIDED “AS IS” WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, 
+EITHER EXPRESS OR IMPLIED, INCLUDING WARRANTIES OR CONDITIONS OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE OR NON-
+INFRINGEMENT. YOU BEAR THE RISK OF UNDERTAKING ANY ACTIVITIES UNDER THIS 
+LICENSE.
+5. Limitation of Liability.
+EXCEPT AS PROHIBITED BY APPLICABLE LAW, IN NO EVENT AND UNDER NO LEGAL 
+THEORY, WHETHER IN TORT (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE 
+SHALL ANY LICENSOR BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY DIRECT, 
+INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF OR 
+RELATED TO THIS LICENSE, THE USE OR INABILITY TO USE THE WORK (INCLUDING BUT 
+NOT LIMITED TO LOSS OF GOODWILL, BUSINESS INTERRUPTION, LOST PROFITS OR 
+DATA, COMPUTER FAILURE OR MALFUNCTION, OR ANY OTHER DAMAGES OR LOSSES), 
+EVEN IF THE LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

LICENSE-NOTICE.md

@@ -0,0 +1,19 @@
+# Licence notice — `changh95/GR00T-N1.5-3B-p150`
+
+This repository packages NVIDIA's GR00T N1.5-3B policy for a Tenstorrent Blackhole p150a. It mixes material under
+two licences; nothing here grants more than its source does.
+
+| what | where in this repo | licence | source |
+|---|---|---|---|
+| Model weights `nvidia/GR00T-N1.5-3B` @ `869830fc749c35f34771aa5209f923ac57e4564e` | **not included** — `tt-model serve` / `code/scripts/download_weights.sh` fetch the pinned snapshot into your HF cache | **NVIDIA License** (copy in [`LICENSE`](LICENSE); original: [`LICENSE` in the weights repo](https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/LICENSE)) — use limited to research or evaluation (§3.3), redistribution only under the same licence with a copy included (§3.1), derivative works carry the same use limitation (§3.2) | NVIDIA |
+| Qwen2 tokenizer files (`vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`, `added_tokens.json`) | `code/gr00t_p150/assets/tokenizer/n15/` | Apache-2.0 (copied from `gr00t/model/backbone/eagle2_hg_model/` of [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) at tag `n1.5-release`, an Apache-2.0 repository) | NVIDIA / Alibaba Qwen |
+| Demo frame and joint state (`media/demo_ego_view.png`, `media/demo_observation.json`, `code/gr00t_p150/demo/n15/`) | `media/`, `code/gr00t_p150/demo/n15/` | one step of the GR1 `robot_sim.PickNPlace` demo data shipped with Isaac-GR00T (`demo_data/`, Apache-2.0 repository); the reference actions in `expected.json` were computed from those inputs with the weights above | NVIDIA |
+| The tt-nn port (`code/models/experimental/gr00t/`), the policy server (`code/gr00t_p150/`), the scripts (`code/scripts/`) and the tt-metal files copied into `code/models/{common,tt_dit,demos}/` | `code/` | **Apache-2.0** (SPDX headers; © 2026 Tenstorrent). Because the port's device-layout weights and the served outputs are derivative works of the NVIDIA weights, running this package is bound by the NVIDIA License's non-commercial use limitation (§3.2/§3.3) | Tenstorrent |
+| The container image (`image/`) | `image/` | tt-metal / tt-nn (Apache-2.0), Ubuntu 22.04 base and the Python packages listed in `requirements.lock` under their own licences; **contains no weights** | Tenstorrent, upstream projects |
+
+Model card front matter therefore declares `license: other`, `license_name: nvidia-license`,
+`license_link: https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/LICENSE` (HF has no identifier for this licence;
+it is *not* the "NVIDIA Open Model License" and *not* Apache-2.0).
+
+The GR00T N1 report is [arXiv:2503.14734](https://arxiv.org/abs/2503.14734); the N1.5 checkpoint is described on its
+[model card](https://huggingface.co/nvidia/GR00T-N1.5-3B).

README.md

@@ -4,8 +4,21 @@ tags:
 - p150
 - tt-dit-server
 - tt-model-cache
-- tt-model-catalog
 - tt-model-container
+- tenstorrent
+- ttnn
+- tt-metal
+- tt-nn
+- robotics
+- vla
+- gr00t
+- tt-model-catalog
+pipeline_tag: robotics
+license: other
+license_name: nvidia-license
+license_link: https://huggingface.co/nvidia/GR00T-N1.5-3B/blob/main/LICENSE
+base_model:
+- nvidia/GR00T-N1.5-3B
 ---
 
 # gr00t-n1.5-3b-p150
@@ -24,7 +37,8 @@ tt-model pull  changh95/GR00T-N1.5-3B-p150 --with-weights
 tt-model serve changh95/GR00T-N1.5-3B-p150
 ```
 
-`pull --with-weights` downloads the Docker image and the [`nvidia/GR00T-N1.5-3B`](https://huggingface.co/nvidia/GR00T-N1.5-3B) weights at `869830fc749c35f34771aa5209f923ac57e4564e` (into your HF cache; they are not in the image). `serve` starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`.
+- Weights [`nvidia/GR00T-N1.5-3B`](https://huggingface.co/nvidia/GR00T-N1.5-3B) at `869830fc749c` (3 bf16 safetensors shards, 5.45 GB) go to your HF cache; the image does not contain them.
+- Serves on port 20000 (or the next free port); ready when the log says `Application startup complete`.
 
 ### Run with tt-cli
 

SERVING.md

@@ -0,0 +1,274 @@
+# Serving GR00T N1.5-3B (`nvidia/GR00T-N1.5-3B`) on Blackhole with tt-model-manager
+
+This repo is a **tt-model container package source**: `tt-model.yaml` + the code under `code/`. `tt-model package
+--container` builds the OCI image (tt-metal `668c2907575` built from source + the port + the HTTP stack),
+`tt-model serve` runs it against the card, `tt-model push` publishes it to this HF repo. The server is
+`code/gr00t_p150/server/app.py` (FastAPI, `kind: tt-dit-server`); what it runs on the device is the port's **Stage-1**
+path — every op a TTNN op, captured once into four Metal traces (`vision`, `llm`, `adapter`, `denoise`) and replayed per
+request; the persistent-megakernel denoise (Stage 2) is not part of this release.
+
+| | |
+|---|---|
+| tt-metal tree the image is built from | `/home/deepgadget/experiments/gr00t/publish/tt-metal-668c2907575` — a clean local clone of tt-metal `main` @ `668c290757550588d0ce46b180c344a462a2aaf5` (`v0.79.0-dev20260914-1`), submodules populated, `git status` empty (the working clone `/home/deepgadget/experiments/gr00t/tt-metal` at the same commit has untracked dirs and would be recorded as dirty); torch pin 2.11.0 |
+| port code shipped | `code/models/experimental/gr00t/` = tt-metal ref `gr00t-p150-snapshot`, commit `5dc0517ee3bec090b7ead025001a343c2f63aa7b` (parent `668c290`; frozen 2026-09-14T23:36 KST), minus `__pycache__`, `.omc` tool state and 2,001 of the 2,004 per-test result JSONs (the three N1.5 e2e results the card cites and the `d0/d1/d2/mk_k1` summaries stay) |
+| weights | `nvidia/GR00T-N1.5-3B` @ `869830fc749c35f34771aa5209f923ac57e4564e`: `model-0000{1,2,3}-of-00003.safetensors` (5,448,327,040 B), `model.safetensors.index.json`, `config.json`, `experiment_cfg/metadata.json`, `LICENSE` — a pinned pointer, never in the image |
+| app | `gr00t_p150.server.app:app` (uvicorn `--lifespan on`; the lifespan does weights → tokenizer → device → model → trace capture → warm-up → READY) |
+| device recipe | `models.experimental.gr00t.tt.device.open_gr00t_device(trace_region_size=64 MiB, l1_small_size=32768, device_id=0)`; asserts the p150a's 11×10 compute grid and 8 DRAM banks; `TTPolicy(dtype_policy="mixed_dit", trace_layout="per_stage")` |
+| serving shape | embodiment `gr1` (slot 24), layout `gr1`: 1 camera (`ego_view`) × 1 frame × 256 image tokens, ≤ 102 text tokens, LLM sequence padded to 384, state 7/7/6/6 → 16 × [7, 7, 6, 6] actions, batch 1 |
+
+Below, `$PUB=/home/deepgadget/experiments/gr00t/publish` (this repo is `$PUB/GR00T-N1.5-3B-p150`),
+`$TREE=/home/deepgadget/experiments/gr00t/tt-metal` (the built working clone, Python 3.10 venv),
+`$ROOT=/home/deepgadget/experiments/tt-models` (tt-model-manager tooling).
+
+## 1. Layout
+
+```
+tt-model.yaml                     authoring manifest (schema 5.1) -- the whole build/serve interface, incl. the model card text
+SERVING.md                        this file
+GPU_COMPARISON.md                 RTX 5090 vs p150a, matching definitions and caveats
+LICENSE                           the NVIDIA License (weights); LICENSE-NOTICE.md says what is under which licence
+media/demo_ego_view.png           the demo frame (256x256, GR1 sim robot_sim.PickNPlace traj 0 / step 100)
+media/demo_observation.json       the matching raw joint state + instruction (request body minus the image)
+media/demo_actions.png            served actions vs the fp32 reference for that request
+code/models/experimental/gr00t/   the tt-nn port (tt-metal tree layout): common/ (host pipeline: preprocessing, prompts,
+                                  normalisation, checkpoint, weight plan), tt/ (device model, layers, traces, megakernel WIP),
+                                  reference/ (fp32 torch reference), tests/, benchmarks/
+code/models/{common,tt_dit,demos}/   the 16 tt-metal files the port imports from the tree (source.code in tt-model.yaml)
+code/gr00t_p150/                  the policy server: server/{app,schemas,smoke_test}.py, demo/n15/ (canonical demo
+                                  observation + golden actions + noise), assets/tokenizer/n15/ (vendored Qwen2 tokenizer)
+code/scripts/                     download_weights.sh, bench_http.py
+requirements.lock                 the image venv (Python 3.12) as `uv pip freeze` -- written by `tt-model package`
+tt_kernel_manifest.json, image/   written by `tt-model package`; uploaded by `tt-model push`
+```
+
+The port keeps its tt-metal-tree absolute imports (`models.experimental.gr00t.*`, `models.tt_dit.*`,
+`models.common.*`): in the image the code lands at `/opt/tt-metal/<path>` with `PYTHONPATH=/opt/tt-metal`, and the
+image's tt-metal copy excludes `models/`, so everything under `models/` comes from this repo's `code/` (the port) and
+`source.code` (the 16 tree files). The server package `gr00t_p150` is shared with `changh95/GR00T-N1.6-3B-p150`; this
+repo ships only the N1.5 assets (`demo/n15`, `assets/tokenizer/n15`).
+
+## 2. Run on the HOST for validation (no Docker)
+
+The tree venv (`$TREE/python_env`, Python 3.10, torch 2.11.0+cpu, ttnn editable) has everything except
+`fastapi`/`uvicorn`. Do not install into the tree venv; put the HTTP stack in a side directory and prepend it to
+`PYTHONPATH` (`$PUB/http-side` already holds fastapi 0.141.1 / uvicorn 0.53.0 / pydantic 2.13.5 / starlette 1.6.0):
+
+```bash
+export PATH=$HOME/.local/bin:$PATH
+mkdir -p $PUB/http-side && uv pip install --python $TREE/python_env/bin/python --target $PUB/http-side fastapi uvicorn "pydantic>=2"
+
+cd $PUB/GR00T-N1.5-3B-p150
+export TT_METAL_HOME=$TREE ARCH_NAME=blackhole
+export PYTHONPATH=$PWD/code:$TREE:$TREE/ttnn:$PUB/http-side      # code/ FIRST so models.experimental.gr00t is the snapshot
+export TT_METAL_CACHE=$HOME/.cache/tt-metal-cache-gr00t           # warm JIT kernels of the port's device sessions
+export HF_MODEL=nvidia/GR00T-N1.5-3B TT_WEIGHTS_REVISION=869830fc749c35f34771aa5209f923ac57e4564e
+export GR00T_VERSION=n15 TT_MESH_SHAPE=1x1 TT_DEVICE_ID=0 TT_METAL_VISIBLE_DEVICES=0
+export GR00T_TOKENIZER_DIR_N15=$PWD/code/gr00t_p150/assets/tokenizer/n15   # optional: the server picks the vendored dir itself
+export GR00T_TT_CACHE=$HOME/.cache/gr00t-tt                                # host .pt plan tier + .tensorbin device tier (~6.4 GB for N1.5)
+
+# import check, no device (what the image's verify.sh does):
+$TREE/python_env/bin/python -c "import gr00t_p150.server.app as a, sys; assert a.app and 'ttnn' not in sys.modules; print('ok')"
+
+# serve + smoke + stop -- ONE device-lock hold (the box has one p150a shared with other sessions):
+DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
+  '$TREE'/python_env/bin/python -m uvicorn --host 127.0.0.1 --port 20010 --lifespan on gr00t_p150.server.app:app & UV=$!
+  python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:20010 --wait 1800 --out /tmp/gr00t-n15-smoke.json; RC=$?
+  kill -TERM $UV; wait $UV; exit $RC'
+```
+
+Boot log landmarks (they drive `tt-model serve`'s checklist): `Loading weights: nvidia/GR00T-N1.5-3B @ 869830fc…` →
+`Tokenizer files: …` → `Opening device 0 (trace_region_size=67108864, l1_small_size=32768)` → `Loading pipeline:
+Gr00tTT.from_pretrained(n15, …)` → `Model built in N s (weights N s, 778 tensors, 3176 MB on device, cache path
+warm|cold)` → `Warming up: capture per_stage traces on the n15 demo observation, then N traced predict(s)` →
+`Warmup k/N: … ms (encode, device, decode)` → `Warm-up fidelity vs golden fp32: actions PCC … ` → `Warmup complete
+(capture s, first ms, steady ms; boot s) -- per_stage traces ['vision', 'llm', 'adapter', 'denoise']` → uvicorn
+`Application startup complete`. Any failure raises and uvicorn exits non-zero (no CPU fallback); the boot also **fails**
+if the warm-up actions' PCC vs the shipped golden is below `GR00T_WARMUP_PCC_MIN` (0.999). SIGTERM / Ctrl-C:
+`Releasing traces and device tensors` → `Closing device`.
+
+Measured host boots (warm `~/.cache/gr00t-tt` and warm `TT_METAL_CACHE`; `$PUB/logs/server_n15_*.log`): first boot of a
+process 14.6 s to READY (model built 9.3 s incl. 9.2 s reading the `.tensorbin` tier, warm + capture 2.5 s, 5 traced
+warm-ups 44.4–44.8 ms); an immediately following boot 4.5 s (0.6 s model build with the tiers in the page cache); on the
+frozen snapshot worktree 5.2 s (`server_n15_snapshot_20260914-235713.log`). Kernels are rebuilt once for a new
+`TT_METAL_HOME` path (~42 s for the 355 kernels of the four traces, `docs/publish/snapshot.md` §2.3).
+
+Expected smoke line (host, snapshot worktree, `$PUB/logs/smoke_n15_snapshot_20260914-235713.log`):
+
+```
+PASS GR00T-N1.5-3B-p150 n15: PCC(actions)=0.999982 max|d|=0.0289 PCC(action_pred_valid)=0.999976 repeat_maxdiff=0.0e+00
+     seed_path_ok device_ms=42.95/42.16 total_ms=47.22/44.83 wall_ms=50
+     per_group[left_arm=0.99995/0.0214 right_arm=0.99997/0.0172 left_hand=0.94335/0.0179 right_hand=0.99999/0.0289]
+```
+
+The per-group PCC / max|d| equal the port's own device test (`code/models/experimental/gr00t/tests/tt/results/
+test_e2e_predict_actions_vs_golden_n15_20260914-230743.json`) to the printed precision; the traced policy is
+deterministic, so every run reproduces them bit-exactly.
+
+Offline overrides: `GR00T_WEIGHTS_DIR=<dir with the snapshot files>` (shimmed into a private hub cache so the port's
+`snapshot_dir()` resolves to it), `GR00T_TOKENIZER_DIR=<dir>`.
+
+Offline checks that need no device:
+
+```bash
+cd $PUB/GR00T-N1.5-3B-p150
+# manifest + launcher preview (from the repo dir -- `root: code` is CWD-relative); must print VALID
+$ROOT/.venv/bin/python -c "from tt_kernel.container_manifest import load_container_manifest; m = load_container_manifest('tt-model.yaml', check_sources=True); p = m.resolve_profile(); print('VALID', m.name, m.kind, p.hardware, p.mesh_device, m.weights_ref)"
+# Python 3.12 resolution of runtime.packages (what the image does) -- must keep numpy<2 and torch 2.11.0+cpu
+uv venv --python 3.12 $PUB/depcheck-n15 -q
+uv pip install --python $PUB/depcheck-n15/bin/python --dry-run torch==2.11.0 fastapi uvicorn "pydantic>=2" pillow "numpy>=1.24.4,<2" \
+   safetensors huggingface_hub "transformers==5.12.1" "opencv-python-headless==4.8.1.78" "torchvision==0.26.0" pytest \
+   --extra-index-url https://download.pytorch.org/whl/cpu --index-strategy unsafe-best-match
+#   -> torch 2.11.0+cpu, torchvision 0.26.0+cpu, numpy 1.26.4, transformers 5.12.1, opencv-python-headless 4.8.1.78 (cp312 wheel), pytest 9.1.1 (logs/uv-dryrun-n15.log)
+```
+
+## 3. Package, serve, push (Docker)
+
+Rootless Docker on this box needs `source $ROOT/bin/docker-env.sh` first (PATH + `DOCKER_HOST`; the bare `docker`
+on PATH is podman). **Run every `tt-model` command from this directory**: `source.tt_metal` and
+`extra_code[].root: code` resolve against the process CWD. `--out` points outside any git checkout because `stage()`
+deletes `<out>/gr00t-n1.5-3b-p150` before rebuilding. The manifest `name` is the slug `gr00t-n1.5-3b-p150`, so that is
+the build directory, the cache directory (`~/.cache/tt-model/gr00t-n1.5-3b-p150/`) and the generated card's title.
+
+```bash
+source $ROOT/bin/docker-env.sh
+cd $PUB/GR00T-N1.5-3B-p150
+
+$ROOT/.venv/bin/tt-model package --container tt-model.yaml --out $PUB/build     # tt-metal C++ build (ccache) + venv + verify.sh
+# device: serve -> smoke -> bench -> stop inside ONE lock hold (`serve` returns after READY and leaves the container
+# running on the card; a second with-device.sh caller would probe a busy card and reset it)
+DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
+  source '$ROOT'/bin/docker-env.sh
+  '$ROOT'/.venv/bin/tt-model serve '$PUB'/build/gr00t-n1.5-3b-p150/tt_kernel_manifest.json 2>&1 | tee '$PUB'/logs/serve-n15.log || exit 1
+  PORT=$(grep -o "127.0.0.1:[0-9]*" '$PUB'/logs/serve-n15.log | head -1 | cut -d: -f2)
+  python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:$PORT --out '$PUB'/logs/smoke-n15.json; RC=$?
+  python3 code/scripts/bench_http.py --url http://127.0.0.1:$PORT --reps 50 --warmup 10 --out '$PUB'/logs/bench-http-n15.json
+  '$ROOT'/.venv/bin/tt-model logs changh95/GR00T-N1.5-3B-p150 > '$PUB'/logs/container-n15.log 2>&1
+  '$ROOT'/.venv/bin/tt-model stop changh95/GR00T-N1.5-3B-p150; exit $RC'
+$ROOT/.venv/bin/tt-model push $PUB/build/gr00t-n1.5-3b-p150 --publish
+```
+
+`serve` pre-downloads the pinned files (a cache hit on this host), then `docker run --user 0:0 --device
+/dev/tenstorrent --ipc host` with `/dev/hugepages-1G`, `~/.cache/huggingface` at `/hf` (rw, `HF_HOME=/hf`) and
+`~/.cache/tt-model/gr00t-n1.5-3b-p150/{cache,weights,tensors}` at `/cache` (`TT_METAL_CACHE`, JIT kernels),
+`/weight-cache` (`TT_DIT_CACHE_DIR`; the port's weight tiers go to `/weight-cache/gr00t-tt`, ~6.4 GB) and
+`/tensor-cache`; it exports exactly `HF_MODEL=nvidia/GR00T-N1.5-3B`, `MESH_DEVICE=P150`, `TT_MESH_SHAPE=1x1` plus
+`serve.env` from `tt-model.yaml`, publishes the first free port from 20000 and waits ≤ 1800 s for `Application startup
+complete`. tt-cli users: `tt serve changh95/GR00T-N1.5-3B-p150` / `tt model stop changh95/GR00T-N1.5-3B-p150`
+(after the push).
+
+`push` makes `code/` and `image/` on the Hub exactly the staged trees and replaces `README.md` with the generated card
+(everything worth keeping lives in `card.description` / `card.quickstart` of `tt-model.yaml`); `media/`, `SERVING.md`,
+`GPU_COMPARISON.md`, `LICENSE`, `LICENSE-NOTICE.md`, `tt-model.yaml` and `.gitattributes` at the repo root survive; the
+`license` / `pipeline_tag` / `base_model` front matter is restored afterwards with `huggingface_hub.metadata_update`.
+
+## 4. Container validation (the image built from this manifest)
+
+Image `tt-model/gr00t-n1.5-3b-p150:443501bd3b7a` (`sha256:443501bd3b7afce579bf743eeedd64809886653c29a9d7125614e8451cd61a7b`,
+3.2 GB as 46 OCI blobs), built 2026-09-14 15:17–15:30 UTC from the manifest above (`tt-model package`: tt-metal C++ build
+with a warm ccache ≈ 7 min, Python 3.12 venv, all 17 `verify:` lines + the launcher's own passed inside the image;
+13 min 17 s in total). Validated on the p150a on 2026-09-15 00:56–01:02 KST: three `tt-model serve` boots of this exact
+image, each followed by `smoke_test.py`, a 50-request `bench_http.py` and `tt-model stop`, all inside one device-lock hold
+per run (`publish/logs/validate-n15.run2.log` = boots 1–2, `validate-n15.run3.log` = boot 3; per-boot files
+`{serve,smoke,bench-http,container,info}-n15.{2,3}.*`; boot 1's per-file artefacts were overwritten by boot 3 and survive
+only inside `validate-n15.run2.log`).
+
+| boot | caches (`~/.cache/tt-model/gr00t-n1.5-3b-p150/`) | host load | `tt-model serve` → READY | container boot (lifespan) | smoke | `bench_http.py` (10 warm-up + 50): device / server-side median (p90) | `tt-model stop` |
+|---|---|---|---|---|---|---|---|
+| 1 (**cold**) | `weights/` and `cache/` empty before; 6.4 GB + 411 MB after | two concurrent tt-metal image builds + CPU tests (load avg 16–110) | **1 min 43 s** (103.6 s wall) | 100.9 s: device open ≈ 4 s → `Model built in 8.9 s (… cache path cold)` (checkpoint → host plan → `.tensorbin`) → trace capture **87.3 s** incl. the JIT of the four traces' kernels (`riscv-tt-elf-g++` lines in the container log) → 3 warm-ups 50.9 / 45.8 / 101.5 ms | **PASS** — PCC(actions) 0.999982, max\|d\| 0.0289; PCC(`action_pred_valid`) 0.999976; device 46.96 / 42.21 ms, total 54.42 / 48.21 ms | 44.27 (55.35) / 53.59 (91.12) ms; encode median 3.2, p90 41 ms; client wall 58.1 (94.5) | clean SIGTERM shutdown 4.9 s |
+| 2 (warm, immediately after) | reused | same loaded host | 20.2 s (21.3 s wall) | 15.1 s: `Model built in 0.6 s (… cache path warm)`, capture 6.6 s, warm-ups 47.3 / 119.4 / 65.3 ms (encode spikes = host contention) | **PASS** — same fidelity; device 50.88 / 42.59 ms, total 67.44 / 45.45 ms | 44.51 (55.76) / 58.14 (96.86) ms; encode median 5.8, p90 38 ms; client wall 64.4 (104.2) | clean 3.1 s |
+| 3 (warm, quiet host) | reused | load avg ≈ 9, no other device or build work | **9.7 s** (10.0 s wall) | 7.6 s: model 0.6 s, capture 3.2 s, warm-ups 44.5 → 44.3 ms | **PASS** — same fidelity; device 42.99 / 42.19 ms, total 47.31 / 44.95 ms, client wall 50 ms | **42.43 (43.18) / 45.40 (49.20) ms**, min 41.66 / 44.46; decode 1.06, encode 1.63, decode_actions 0.14; client wall 47.6 (51.8) | clean 2.0 s |
+
+Fidelity is identical in every boot and identical to the host runs of §2 — the served `actions` JSON of the container is
+byte-for-byte the host run's (`smoke-n15.3.json` vs `smoke_n15_snapshot_20260914-235713.json`) — and equal to the
+port's device test `tests/tt/results/test_e2e_predict_actions_vs_golden_n15_20260914-230743.json` to the printed precision:
+`left_arm` 0.99995 / 0.0214, `right_arm` 0.99997 / 0.0172, `left_hand` 0.94335 / 0.0179 (abs-gated), `right_hand`
+0.99999 / 0.0289. The boot-3 benchmark is the card's "served" row; the device median 42.4 ms is 0.7 ms above the port's
+own `bench_e2e` device figure (41.68 ms: upload + traces + readback) and the loaded-host runs show that the extra
+latency and the p90 tails sit in the host-side stages (`encode`), not on the device.
+
+`GET /info` inside the container reports `weights.snapshot_dir = /hf/hub/models--nvidia--GR00T-N1.5-3B/snapshots/869830fc…`
+(the mounted HF cache), `source.tt_metal = {commit 668c290…, ttnn_dist_version 0.65.2.dev10011}`,
+`source.port_snapshot_commit = 5dc0517ee3b…`, `device_facts` grid 11×10 / DRAM 8×1 (`info-n15.3.json`).
+
+**Device-sharing note for this host.** The first validation attempt (00:32–00:40 KST) ran while another session's
+container was still holding the card after that session's lock hold had ended; `bin/with-device.sh`'s probe hung and
+reset the card twice, then refused to run. Since then every device entry goes through
+`publish/scripts/run_validation_n15_when_free.sh`, which waits until `docker ps` shows no `tt-model-*` container and the
+lock is free. Use the manifest path with `tt-model stop` / `logs` for an un-pushed package (`tt-model stop
+changh95/GR00T-N1.5-3B-p150` only works after `pull`).
+
+## 5. Request / response contract
+
+| route | returns |
+|---|---|
+| `GET /health` | `{"status": "ok" \| "starting", "model": "GR00T-N1.5-3B-p150", "device": "blackhole:0"}` — 200 always; `ok` only after the warm-up |
+| `GET /info` | model / version / task / hardware / `stage` (`stage1-ttnn-traces` + megakernel note) / `weights` (repo, revision, snapshot dir actually loaded) / `source` (port snapshot commit, server version, tt-metal commit, ttnn dist version) / `inputs` (the full observation contract) / `outputs` / `noise.default_seed` / `limits` / `device_facts` (11×10 grid, 8 DRAM banks) / `warmup_latency_ms` / `warmup_fidelity_vs_golden` / `license` |
+| `GET /v1/models` | OpenAI-shaped stub `{"object": "list", "data": [{"id": "nvidia/GR00T-N1.5-3B", "object": "model", "owned_by": "changh95"}]}` so the tt-model ready card / `tt-model curl` do not 404; this is not a chat API |
+| `GET /demo` | `{"request": <ready-to-post demo body>, "expected": <fp32 reference actions + gates>}` |
+| `POST /predict` | one observation → one 16-step action chunk (below) |
+
+`POST /predict` request (JSON; unknown fields → 422):
+
+| field | type | meaning |
+|---|---|---|
+| `images` | `{"ego_view": <b64 PNG/JPEG>}` (or a one-element list per camera) | exactly the GR1 camera; any resolution with sides in [64, 4096] px; the server applies the checkpoint's eval chain (x/255 → 0.95 centre crop → bilinear antialiased 224×224 → Eagle2.5 normalisation), bit-exact vs the reference preprocessing |
+| `state` | `{"left_arm": [[7]], "right_arm": [[7]], "left_hand": [[6]], "right_hand": [[6]]}` | raw physical joint state, one time step (`(D,)` or `(1, D)`), finite floats; sin/cos-encoded and padded to 64 dims on the host |
+| `instruction` | `str` (aliases `language`, `prompt`) | task text, 1–2000 chars, ≤ 102 BPE tokens after the port's prompt build (`repr([instruction])` inside the Eagle2 chat prompt); longer → 400, never truncated |
+| `embodiment` | `"gr1"`, optional | must equal the server's; anything else → 400 |
+| `seed` | int ≥ 0, optional | CPU-generator seed of the `[1, 16, 32]` flow-matching noise; default 0 (the deployed policy's) |
+| `noise` | `[[32 floats] × 16]`, optional | explicit initial noise; mutually exclusive with `seed` |
+| `state_dtype` | `"float64"` (default) \| `"float32"` | dtype of the state before normalisation (the GR1 dataset stores float64) |
+| `return_normalized` | bool, default false | also return the model-space chunk `action_pred_normalized` (`[16][32]`) and `action_pred_valid_hd` (`[16, 26]`) |
+
+Response (200):
+
+```json
+{"actions": {"left_arm": [[7 floats] x 16], "right_arm": [...], "left_hand": [...], "right_hand": [...]},
+ "action_horizon": 16, "action_keys": ["left_arm", "right_arm", "left_hand", "right_hand"], "action_dims": {"left_arm": 7, "right_arm": 7, "left_hand": 6, "right_hand": 6},
+ "normalized": false, "embodiment": "gr1", "embodiment_id": 24, "version": "n15", "model": "GR00T-N1.5-3B-p150", "layout": "gr1",
+ "images": {"ego_view": {"frames": 1, "received_hw": [256, 256]}}, "seq_len": 296, "prompt_tokens": 14, "state_dtype": "float64",
+ "noise_source": "seed" | "client", "seed": 0 | null,
+ "timing_ms": {"decode": 1.5, "encode": 2.6, "device": 43.0, "decode_actions": 0.1, "total": 47.2}}
+```
+
+`actions` are physical joint targets (radians) — `(y + 1) / 2 · (max − min) + min` with the checkpoint's
+`experiment_cfg/metadata.json` GR1 statistics — exactly what `Gr00tPolicy.get_action` returns. `timing_ms`: `decode` =
+base64 + image decode + validation; `encode` = the port's host preprocessing; `device` = `predict_normalized` (static-shape
+asserts, input writes, 4 × `execute_trace`, one blocking readback); `decode_actions` = un-normalisation; `total` = the
+whole handler. Errors: **400** structural problems (`{"detail": {"errors": [...], "inputs": {...}}}`), undecodable image,
+bad state / instruction / noise, other embodiment; **503** while starting; **500** `Type: text` on a device failure.
+Handlers are synchronous and serialised on one lock; batch 1.
+
+## 6. Environment (read in the lifespan, never at import)
+
+| variable | container value (`serve.env`) | meaning |
+|---|---|---|
+| `HF_MODEL` / `TT_WEIGHTS_REVISION` | `nvidia/GR00T-N1.5-3B` (launcher) / `869830fc…` | must be the version's pinned repo and sha (both are asserted against the port's own `configs.HF_REPOS` / `HF_SNAPSHOT_SHAS`) |
+| `GR00T_VERSION` | `n15` | the server code is shared by both GR00T packages and never guesses |
+| `GR00T_EMBODIMENT` / `GR00T_LAYOUT` | `gr1` / `gr1` | the only device-validated layout of this release |
+| `GR00T_POLICY` / `GR00T_TRACE_LAYOUT` | `mixed_dit` / `per_stage` | `TTPolicy` knobs; `bf16` / `mixed` and `two` exist in the port but are not the benchmarked defaults |
+| `GR00T_TRACE_REGION_SIZE` / `GR00T_L1_SMALL_SIZE` | default 67108864 / 32768 | the port's validated `open_gr00t_device` values |
+| `GR00T_TOKENIZER_DIR_N15` | `/opt/tt-metal/gr00t_p150/assets/tokenizer/n15` | vendored Qwen2 tokenizer (not in the weights repo); `GR00T_TOKENIZER_DIR` overrides |
+| `GR00T_TT_CACHE` | `/weight-cache/gr00t-tt` | host `.pt` plan tier + `.tensorbin` device tier (persisted under `~/.cache/tt-model/gr00t-n1.5-3b-p150/weights`) |
+| `GR00T_PROJECT_ROOT` / `GR00T_GOLDEN_ROOT` | `/nonexistent` | the port's dev-box defaults; nothing on the serve path needs them |
+| `GR00T_WARMUP_RUNS` / `GR00T_WARMUP_PCC_MIN` | `3` / default 0.999 | traced predicts before READY; boot fails below the PCC floor vs the shipped golden |
+| `GR00T_DEFAULT_SEED`, `GR00T_TORCH_THREADS`, `GR00T_WEIGHTS_DIR` | unset | request default seed (0), torch intra-op threads, offline snapshot dir |
+| `TT_MESH_SHAPE`, `TT_DEVICE_ID`, `TT_METAL_VISIBLE_DEVICES` | `1x1` (launcher), `0`, `0` | one chip; any other mesh shape is refused |
+
+## 7. Caveats
+
+- **Stage-1 path only.** TTNN ops in Metal traces; the persistent-megakernel denoise (K1 streaming kernel: 464 GB/s bf16 /
+  414 GB/s bfp8, `code/models/experimental/gr00t/tests/tt/results/mk_k1_summary.md`) is not integrated into `Gr00tTT` and not served.
+- **Shape-locked traces.** One embodiment (`gr1`), one camera, one frame, ≤ 102 text tokens (LLM S padded to 384), batch 1.
+  Other embodiments / cameras / longer instructions → 400 before anything reaches the device. Concurrent requests queue on one lock.
+- **Deterministic noise.** Actions are a deterministic function of (observation, seed | noise); the reference noise of the
+  demo is not seed-reproducible, so the smoke test posts it as `noise`. Same observation + same noise → bit-identical actions.
+- **Base checkpoint.** Outputs are the GR1 sim data-config actions of the released N1.5 checkpoint; a real robot needs NVIDIA's post-training.
+- **First container boot is cold**: the port converts the 5.4 GB checkpoint into its `.pt` plan tier and the `.tensorbin`
+  device tier under `/weight-cache/gr00t-tt` and JITs the kernels of the four traces into `/cache`; both persist under
+  `~/.cache/tt-model/gr00t-n1.5-3b-p150/`, so later boots are seconds (measured values in §4).
+- **Python 3.12 image vs 3.10 tree venv.** Host runs are an approximation; the image is authoritative (its `verify.sh`
+  imports the app and the port with no device; the served smoke in §4 is the proof).
+- **Weights licence.** `nvidia/GR00T-N1.5-3B` is under the NVIDIA License (non-commercial: research or evaluation use);
+  the server reports it under `/info -> license`. See `LICENSE-NOTICE.md`.
+- `tt-model curl` and the ready card's `/v1/models` hint are OpenAI-shaped and are not this API; use the routes above.

media/demo_observation.json

@@ -0,0 +1,50 @@
+{
+ "state": {
+  "left_arm": [
+   [
+    0.024056308373237207,
+    0.033822765064097184,
+    -0.016396038448321357,
+    -2.0726931601418075,
+    0.012247859931799656,
+    -0.03578162003720947,
+    -0.012941182543890062
+   ]
+  ],
+  "right_arm": [
+   [
+    0.11583298923348632,
+    -0.10512690694722195,
+    0.2879116792491345,
+    -1.9319273126265488,
+    0.3167383566755674,
+    -0.1142246377868063,
+    1.0186058587273703
+   ]
+  ],
+  "left_hand": [
+   [
+    6.398235076672701e-05,
+    -9.923445381200851e-05,
+    -0.0002996759294460034,
+    -0.00018601688776459478,
+    -0.003114431240670283,
+    1.7421393062719688
+   ]
+  ],
+  "right_hand": [
+   [
+    0.009224068251638124,
+    0.011558206453856013,
+    0.014177547256735298,
+    0.011238545611423637,
+    0.1418649869353279,
+    1.741832493216234
+   ]
+  ]
+ },
+ "instruction": "pick the pear from the counter and place it in the plate",
+ "embodiment": "gr1",
+ "state_dtype": "float64",
+ "_note": "GR1 sim robot_sim.PickNPlace demo step (the port's canonical golden observation); add images.ego_view = base64 of media/demo_ego_view.png and POST to /predict"
+}

tt-model.yaml

@@ -0,0 +1,204 @@
+# SPDX-License-Identifier: Apache-2.0
+# tt-model-manager container manifest (schema 5.1) for NVIDIA Isaac GR00T N1.5-3B (Stage-1 TTNN path) on Blackhole.
+#
+# Run EVERY tt-model command from this directory: `source.tt_metal` and `extra_code[].root: code`
+# resolve against the process CWD, not against this file.  Rootless Docker on this box needs
+# `source /home/deepgadget/experiments/tt-models/bin/docker-env.sh` first (bare `docker` is podman).
+#
+#   tt-model package --container tt-model.yaml --out /home/deepgadget/experiments/gr00t/publish/build
+#   DEVICE_LOCK_TIMEOUT=14400 /home/deepgadget/experiments/gr00t/bin/with-device.sh bash -c '
+#       tt-model serve /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.5-3b-p150/tt_kernel_manifest.json && \
+#       python3 code/gr00t_p150/server/smoke_test.py --url http://127.0.0.1:20000; tt-model stop changh95/GR00T-N1.5-3B-p150'
+#   tt-model push /home/deepgadget/experiments/gr00t/publish/build/gr00t-n1.5-3b-p150 --publish
+schema: "5.1"
+
+repo: changh95/GR00T-N1.5-3B-p150
+name: gr00t-n1.5-3b-p150          # slug ([a-z0-9][a-z0-9._-]*): cache dir, container name, generated H1
+
+# A POINTER, pinned.  Weights are never baked into the image: `tt-model serve` pre-downloads exactly these files
+# at this sha into the host HF cache (mounted at /hf, HF_HOME=/hf) and the server resolves the same snapshot with
+# huggingface_hub.snapshot_download(revision=TT_WEIGHTS_REVISION, local_files_only=True) and the port's own
+# common.configs.snapshot_dir() (both must agree, the server asserts it).
+weights:
+  repo: nvidia/GR00T-N1.5-3B
+  revision: 869830fc749c35f34771aa5209f923ac57e4564e
+  allow_patterns:
+    - "model-*.safetensors"            # 3 bf16 shards, 5,448,327,040 bytes
+    - model.safetensors.index.json     # required by the port's LazyCheckpoint (common/checkpoint.py)
+    - config.json
+    - experiment_cfg/metadata.json     # GR1 normalisation statistics (common/normalization.py)
+    - LICENSE                          # NVIDIA License -- keep it next to the weights it covers
+
+kind: tt-dit-server
+arch: blackhole
+
+source:
+  # Clean checkout of tt-metal main 668c2907575 (v0.79.0-dev20260914-1): a local clone of the validated tree with its
+  # three submodules populated and `git status --porcelain` empty, so the image is built from exactly the commit the
+  # card links to (the working clone has untracked dirs and would be recorded as a dirty tree).
+  tt_metal: /home/deepgadget/experiments/gr00t/publish/tt-metal-668c2907575
+  # tt-metal-relative files the port imports from the tree OTHER than itself, with their import closure
+  # (the image's tt-metal copy excludes models/, so everything under models/ must be listed here or in extra_code).
+  code:
+    - models/common/utility_functions.py                         # models.tt_dit.utils.matmul imports is_blackhole (needs pytest!)
+    - models/tt_dit/utils/matmul.py                              # tt/layers.py: the `minimal` matmul rows (lazy)
+    - models/tt_dit/utils/agmm_rules.py                          # lazy from matmul.py
+    - models/tt_dit/utils/mmrs_rules.py                          # lazy from matmul.py
+    - models/tt_dit/utils/tensor.py                              # layers/linear.py, layers/module.py, lora.py, parallel/manager.py
+    - models/tt_dit/utils/progress.py                            # layers/module.py
+    - models/tt_dit/utils/walltime.py                            # utils/progress.py
+    - models/tt_dit/utils/substate.py                            # layers/module.py
+    - models/tt_dit/layers/linear.py                             # encoders/qwen3vl/vision_qwen3vl.py (N1.7 tower; lazy, unused by N1.5)
+    - models/tt_dit/layers/module.py
+    - models/tt_dit/layers/normalization.py
+    - models/tt_dit/layers/lora.py                               # imported at the bottom of layers/linear.py
+    - models/tt_dit/parallel/config.py
+    - models/tt_dit/parallel/manager.py
+    - models/tt_dit/encoders/qwen3vl/vision_qwen3vl.py           # tt/qwen3vl_vision.py (lazy; not on the N1.5 serve path)
+    - models/demos/deepseek_v3_b1/unified_kernel_descriptor.py   # tt/megakernel/descriptors.py (lazy; Stage-2 work, not served)
+  # This repo's own code (lands at /opt/tt-metal/<path>; PYTHONPATH=/opt/tt-metal).
+  extra_code:
+    - root: code
+      paths:
+        - models          # code/models/experimental/gr00t: the frozen snapshot of the port (tt-metal ref gr00t-p150-snapshot, 5dc0517ee3b)
+        - gr00t_p150      # the policy server, its smoke test, the demo observation and the vendored Qwen2 tokenizer files
+        - scripts         # download_weights.sh, bench_http.py
+  ubuntu: "22.04"
+  python: "3.12"
+
+runtime:
+  app: gr00t_p150.server.app:app
+  mesh_shape_env: TT_MESH_SHAPE
+  # On top of the kind defaults (fastapi, uvicorn, pydantic>=2, pillow), the auto-pinned torch==2.11.0+cpu (the tree's
+  # requirements-dev.txt pin) and ttnn's own deps (numpy<2, loguru, ...).  Every third-party module on the serve path,
+  # pinned where the tree pins it.  Resolution on Python 3.12 proven with a uv dry-run (SERVING.md).
+  packages:
+    - "numpy>=1.24.4,<2"
+    - safetensors                        # common/checkpoint.py
+    - huggingface_hub                    # server: snapshot_download(local_files_only=True) guard
+    - transformers==5.12.1               # common/prompts.get_tokenizer (Qwen2 tokenizer, lazy); tree pin
+    - opencv-python-headless==4.8.1.78   # common/preprocessing.py imports cv2 at module level; tree pin (cp312 wheel exists)
+    - torchvision==0.26.0                # preprocessing.n15_crop_resize (center_crop / resize antialias); tree pin for torch 2.11.0
+    - pytest                             # models/common/utility_functions.py imports pytest at module level
+
+serve:
+  port: 20000
+  hardware: p150
+  mesh_device: P150
+  env:
+    TT_WEIGHTS_REVISION: "869830fc749c35f34771aa5209f923ac57e4564e"   # == weights.revision == configs.HF_SNAPSHOT_SHAS["n15"]
+    TT_METAL_VISIBLE_DEVICES: "0"
+    TT_DEVICE_ID: "0"
+    GR00T_VERSION: "n15"
+    GR00T_EMBODIMENT: "gr1"              # the only embodiment with a device-validated layout in this release
+    GR00T_LAYOUT: "gr1"                  # static layout: 1 image x 256 tokens, L_max 102 text tokens, LLM sequence padded to 384
+    GR00T_POLICY: "mixed_dit"            # TTPolicy.dtype_policy: DiT block matmul weights bfp8_b, everything else bf16 (validated default)
+    GR00T_TRACE_LAYOUT: "per_stage"      # four Metal traces vision / llm / adapter / denoise (the benchmarked layout)
+    GR00T_TOKENIZER_DIR_N15: "/opt/tt-metal/gr00t_p150/assets/tokenizer/n15"
+    GR00T_PROJECT_ROOT: "/nonexistent"   # the port's dev-box default for project-relative paths; nothing on the serve path needs it
+    GR00T_GOLDEN_ROOT: "/nonexistent"    # tests only (defensive)
+    GR00T_TT_CACHE: "/weight-cache/gr00t-tt"   # host plan (.pt) + device-layout (.tensorbin) weight tiers -> ~/.cache/tt-model/gr00t-n1.5-3b-p150/weights
+    GR00T_WARMUP_RUNS: "3"               # traced predict() calls after trace capture, before READY
+
+# Build-time assertions, run INSIDE the finished image as uid 1000, no device, no weights, no HF_MODEL.
+verify:
+  - "import gr00t_p150.server.app as a; assert a.app; import sys; assert 'ttnn' not in sys.modules, 'server import must be side-effect free'"
+  - "import models.experimental.gr00t.tt.model as m; assert m.Gr00tTT"
+  - "from models.experimental.gr00t.common import preprocessing, normalization, prompts, checkpoint, weights, configs; assert configs.HF_SNAPSHOT_SHAS['n15'] == '869830fc749c35f34771aa5209f923ac57e4564e'"
+  - "import pytest, models.common.utility_functions; import models.tt_dit.utils.matmul as mm; assert mm.get_matmul_config"
+  - "import models.demos.deepseek_v3_b1.unified_kernel_descriptor as u; assert u.UnifiedKernelDescriptor"
+  - "import models.tt_dit.encoders.qwen3vl.vision_qwen3vl"
+  - "import models.experimental.gr00t.tt.layers, models.experimental.gr00t.tt.qwen3vl_vision, models.experimental.gr00t.tt.megakernel.descriptors"
+  - "import safetensors, huggingface_hub, cv2, numpy, PIL; assert int(numpy.__version__.split('.')[0]) < 2, numpy.__version__"
+  - "import transformers; assert transformers.__version__ == '5.12.1', transformers.__version__"
+  - "import torchvision; assert torchvision.__version__.startswith('0.26.0'), torchvision.__version__"
+  - "from pathlib import Path; d = Path('/opt/tt-metal/gr00t_p150/assets/tokenizer/n15'); assert all((d / f).is_file() for f in ('vocab.json', 'merges.txt', 'tokenizer_config.json', 'special_tokens_map.json', 'added_tokens.json')), sorted(p.name for p in d.iterdir())"
+  - "import os; os.environ['GR00T_TOKENIZER_DIR_N15'] = '/opt/tt-metal/gr00t_p150/assets/tokenizer/n15'; from models.experimental.gr00t.common.prompts import get_tokenizer; t = get_tokenizer('n15'); assert t.convert_tokens_to_ids('<IMG_CONTEXT>') == 151669"
+  - "from pathlib import Path; assert Path('/opt/tt-metal/models/experimental/gr00t/common/golden_tap_map.json').is_file()"
+  - "from pathlib import Path; assert Path('/opt/tt-metal/gr00t_p150/server/smoke_test.py').is_file() and Path('/opt/tt-metal/scripts/bench_http.py').is_file()"
+  - "from pathlib import Path; d = Path('/opt/tt-metal/gr00t_p150/demo/n15'); assert all((d / f).is_file() for f in ('observation.json', 'noise.json', 'expected.json', 'frames/ego_view_t0.png')), sorted(str(p) for p in d.rglob('*'))"
+  - "import ttnn; assert all(hasattr(ttnn, a) for a in ('begin_trace_capture', 'end_trace_capture', 'execute_trace', 'release_trace', 'copy_host_to_device_tensor', 'MinimalMatmulConfig'))"
+  - "from gr00t_p150.server.app import parse_mesh_shape as p; assert p('1x1') == p('(1, 1)') == p('1,1') == (1, 1)"
+
+card:
+  description: >
+    NVIDIA Isaac GR00T N1.5 (3B vision-language-action policy: SigLIP ViT (224², 256 tokens/image) → linear projector →
+    12-layer Qwen3-1.7B backbone → 4-block VL self-attention → 16-block cross/self AdaLN DiT, 4 Euler flow-matching
+    steps) running on one Tenstorrent Blackhole p150a via tt-nn: one camera frame + proprioceptive state + a task
+    instruction in, a 16-step chunk of GR1 arm/hand joint targets out. Stage-1 release: every op is a TTNN op replayed
+    from four Metal traces (DiT block matmul weights bfp8, everything else bf16); the persistent-megakernel denoise is in
+    progress and not in this image.
+
+    Weights: [nvidia/GR00T-N1.5-3B](https://huggingface.co/nvidia/GR00T-N1.5-3B) (NVIDIA License, non-commercial) ·
+    Paper: [arXiv:2503.14734](https://arxiv.org/abs/2503.14734) (GR00T N1 report; the N1.5 changes are described in the
+    [N1.5 model card](https://huggingface.co/nvidia/GR00T-N1.5-3B)) ·
+    Upstream code: [NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) (tag `n1.5-release`) ·
+    Port: `code/models/experimental/gr00t` in this repo (tt-metal-tree layout, frozen snapshot `5dc0517ee3b`) + the
+    policy server `code/gr00t_p150`
+  quickstart: |
+    ### Run with tt-cli
+
+    ```bash
+    tt serve changh95/GR00T-N1.5-3B-p150
+    python3 - <<'EOF'
+    import base64, json
+    body = json.load(open("media/demo_observation.json"))          # GR1 PickNPlace demo step: raw joint state + instruction
+    body.pop("_note", None)
+    body["images"] = {"ego_view": base64.b64encode(open("media/demo_ego_view.png", "rb").read()).decode()}
+    json.dump(body, open("req.json", "w"))
+    EOF
+    curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
+    tt model stop changh95/GR00T-N1.5-3B-p150
+    ```
+
+    - `POST /predict`: `images` (`{"ego_view": <base64 PNG/JPEG>}` — the one GR1 camera; any size with sides in 64–4096 px, the server applies the checkpoint's own eval chain: 0.95 centre crop → bilinear 224×224 → Eagle2.5 normalisation), `state` (`{group: [[floats]]}`: `left_arm` 7, `right_arm` 7, `left_hand` 6, `right_hand` 6 — raw joint values, one time step), `instruction` (task text, ≤ 102 BPE tokens); optional `embodiment` (`gr1`, the only layout in this image), `seed` (initial flow-matching noise; default 0 = the deployed policy's seed → deterministic) or `noise` (an explicit `[16][32]` initial noise), `state_dtype` (`float64` default, as the GR1 dataset), `return_normalized` (also return the model-space chunk).
+    - `GET /health`, `GET /info` (the full input/output contract, weights/tree provenance, warm-up latency and fidelity), `GET /demo` (the shipped demo request and its fp32 reference actions).
+
+    ### Response
+
+    ```json
+    {"actions": {"left_arm":   [[0.0069, 0.2107, 0.0483, -1.8136, -0.1264, -0.0548, 0.0147], "... 16 steps"],
+                 "right_arm":  [[-0.0177, -0.1731, 0.3702, -1.7429, 0.4968, -0.0567, 0.9849], "..."],
+                 "left_hand":  [[0.0081, 0.0155, 0.0033, 0.0064, 0.0169, 0.0438], "..."],
+                 "right_hand": [[-1.4880, -1.4915, -1.4781, -1.4810, -2.9609, 2.9904], "..."]},
+     "action_horizon": 16, "action_keys": ["left_arm", "right_arm", "left_hand", "right_hand"], "action_dims": {"left_arm": 7, "right_arm": 7, "left_hand": 6, "right_hand": 6},
+     "normalized": false, "embodiment": "gr1", "embodiment_id": 24, "version": "n15", "model": "GR00T-N1.5-3B-p150", "layout": "gr1",
+     "images": {"ego_view": {"frames": 1, "received_hw": [256, 256]}}, "seq_len": 296, "prompt_tokens": 14, "state_dtype": "float64",
+     "noise_source": "client", "seed": null,
+     "timing_ms": {"decode": 1.5, "encode": 2.6, "device": 43.0, "decode_actions": 0.1, "total": 47.2}}
+    ```
+
+    - `actions` are **physical, un-normalised** joint targets (radians) in the GR1 arms-and-hands action space of the checkpoint's `experiment_cfg/metadata.json` statistics, 16 steps × [7, 7, 6, 6] dims — what `Gr00tPolicy.get_action` returns for this checkpoint; the values above are abridged from the shipped demo request (`GET /demo`) with its golden initial noise. `seq_len` is the LLM prompt length the request produced (296 for the demo; padded to 384 on device). `timing_ms.device` covers input upload, the four trace replays and the action readback; `total` is the whole handler.
+
+    ### Demo
+
+    | Input (`media/demo_ego_view.png`, GR1 sim `robot_sim.PickNPlace` traj 0 / step 100, 256×256) | Instruction | p150a actions vs the fp32 reference (`media/demo_actions.png`) |
+    |:---:|:---|:---:|
+    | ![](media/demo_ego_view.png) | `pick the pear from the counter and place it in the plate` | ![](media/demo_actions.png) |
+
+    ### Accuracy and speed
+
+    | Metric | Value |
+    |---|---:|
+    | Action chunk vs the fp32 reference (`Gr00tPolicy`, same observation, same initial noise) | PCC `left_arm` 0.99995 · `right_arm` 0.99997 · `right_hand` 0.99999 (gates 0.9999; max\|d\| 0.021 / 0.017 / 0.029 rad) · `left_hand` max\|d\| 0.018 rad ≤ 0.046 (PCC 0.943 reported only: the golden chunk is near-constant there, spread 0.1 rad — NVIDIA's own bf16 GPU path scores 0.952) · normalised chunk `action_pred_normalized` 0.99998 (max\|d\| 0.014 ≤ 0.027) |
+    | Intermediate taps vs the fp32 reference (untraced path from pixels, 57 gated taps + determinism row) | **58/58 rows pass**; tightest: `vit_block_first` 0.99994 (gate 0.9999), `backbone_features` 0.99586 (0.995), `vl_self_attention_out` 0.99119 (0.991), `vit_post_ln` 0.99506 (0.993) |
+    | Traced == untraced, determinism, stale state | traced `action_pred` / `backbone_features` / hoisted K,V bit-equal to the untraced run; 5 trace replays bit-identical; mutated inputs (`n_text` > L_max, other shapes) refused before any device write; A→B→A→B alternating observations on one captured model pass |
+    | Inference on p150a (port benchmark, warm, batch 1, one 224² image, LLM S 296→384, median of 50; `benchmarks/results/e2e_stage1_n15.json`) | **43.4 ms end-to-end** (p90 44.1) = host encode 1.5 + upload 3.8 + 4 traces 37.8 (vision 8.7 · LLM 8.7 · adapter 4.5 · denoise 16.0) + readback 0.1 + decode 0.2 |
+    | Served over HTTP by this image (`tt-model serve`, warm, batch 1, the demo request, 10 warm-ups + 50 timed; `scripts/bench_http.py`) | **42.4 ms device · 45.4 ms server-side** (medians; p90 43.2 / 49.2; min 41.7 / 44.5) = image decode 1.1 + host encode 1.6 + device 42.4 (upload, 4 trace replays, readback) + un-normalise 0.1; 47.6 ms client wall on the same host (p90 51.8). Under a heavily loaded host (two concurrent tt-metal builds, load average 16–110) the same loop read 44.3 / 44.5 ms device and 53.6 / 58.1 ms server-side medians with p90s of 55 / 91–97 ms — the host-side stages, not the device, absorb the contention. The served actions are bit-identical to the port's device test and to the host run |
+    | Same policy on an RTX 5090 (same host, NVIDIA's unmodified `Gr00tPolicy`, eager PyTorch bf16 autocast + flash-attn 2, batch 1) | `get_action` e2e **58.0 ms** (preprocessing 2.2, model 55.4 = backbone 20.8 + action head 34.6) → the p150a is 1.34× faster end-to-end (43.4 vs 58.0) and 1.33× on the model (41.7 device vs 55.4); NVIDIA's N1.5 README quotes 47.9 ms on an H100 (backbone 23.2 + head 4 × 6.2) |
+
+    ### Caveats
+
+    - **Stage-1 path.** Vision / LLM / adapter / denoise are TTNN ops captured in four Metal traces; DiT block matmul weights are bfp8_b, everything else bf16 (`mixed_dit`). The persistent-megakernel denoise (Stage 2) is at rung K1 — its weight-streaming kernel measures 464 GB/s bf16 / 414 GB/s bfp8 (90.6 % of DRAM peak) on the full DiT block set — and is **not** in this image.
+    - **One embodiment, one layout.** The image is built for GR1 arms-and-hands (`gr1`, embodiment slot 24, one `ego_view` camera, ≤ 102 text tokens → LLM sequence padded to 384); other embodiments, cameras or longer instructions are refused with 400, nothing is truncated. Batch 1; requests are serialised on one lock.
+    - **Noise is seeded** (seed 0, as the deployed policy); pass `seed` or `noise` for another draw. The reference noise of the shipped demo is not seed-reproducible, so the smoke test sends it as `noise`. Outputs are the base checkpoint's actions for the GR1 sim data config — a real robot needs NVIDIA's post-training.
+    - Not an OpenAI-compatible API; `GET /v1/models` is a stub so the tt-model ready card does not 404.
+    - Validated on tt-metal main `668c2907575` (`v0.79.0-dev20260914-1`), single p150a only, with this exact image (`tt-model serve` → smoke test → 50-request benchmark → `tt-model stop`, three boots). **First boot is cold**: the checkpoint is converted into the port's device layout under `~/.cache/tt-model/gr00t-n1.5-3b-p150/weights` (6.4 GB, 9 s) and the kernels of the four traces are JIT-compiled into `…/cache` (411 MB, ~87 s incl. trace capture) — `tt-model serve` reported READY after 1 min 43 s; the next boots reused both caches and were ready in 20 s (loaded host) and 9.7 s (quiet host). Every boot re-checks the warm-up actions against the shipped fp32 reference (PCC 0.999982 here) and refuses to serve below 0.999.
+    - GPU comparison: the RTX 5090 row is NVIDIA's unmodified `Gr00tPolicy` path (`Isaac-GR00T` `n1.5-release`, torch 2.7.1+cu128, transformers 4.51.3, flash-attn 2.8.3), best median of 3 × (20 warm-ups + 20 timed calls), GPU idle, no `torch.compile` / TensorRT; the p150a rows are the Stage-1 trace path incl. upload and readback. p150a power was not measured, so no efficiency comparison is made. Full table: [`GPU_COMPARISON.md`](GPU_COMPARISON.md).
+
+    ### Licensing
+
+    - Weights: [nvidia/GR00T-N1.5-3B](https://huggingface.co/nvidia/GR00T-N1.5-3B), **NVIDIA License** — use limited to research or evaluation (non-commercial), redistribution only under the same licence (copy in [`LICENSE`](LICENSE)). Not redistributed here; `tt-model serve` fetches the pinned snapshot into your HF cache.
+    - Tokenizer files under `code/gr00t_p150/assets/tokenizer/n15/` (Qwen2 BPE + Eagle2.5 special tokens) are copied from [NVIDIA/Isaac-GR00T `n1.5-release`](https://github.com/NVIDIA/Isaac-GR00T/tree/n1.5-release) (Apache-2.0).
+    - Port and serving code (`code/models/experimental/gr00t`, `code/gr00t_p150`, `code/scripts`): Apache-2.0 (SPDX headers), © 2026 Tenstorrent; as a derivative work of the weights it is distributed here under the same non-commercial use limitation (NVIDIA License §3.2). tt-metal / tt-nn: Apache-2.0.
+    - Demo frame and state (`media/`): one step of the GR1 `robot_sim.PickNPlace` demo data shipped with Isaac-GR00T.