---
tags:
- blackhole
- p300x2
- tt-model-cache
- tt-model-catalog
- tt-model-container
- vllm-plugin
---

# nemotron-3.5-lightning-30b-a3b-nvfp4-tt-p300x2-vllm026

UNPUBLISHED TEST VARIANT on stock vLLM 0.26.0 + the standalone vllm-tt-plugin, with NVIDIA's recommended nemotron_v3 reasoning parser and qwen3_coder tool parser. NVIDIA Nemotron 3.5 Lightning 30B-A3B (NVFP4 checkpoint): a hybrid Mamba-Transformer mixture-of-experts reasoning model with a 1,048,576-token context, ported to Tenstorrent Blackhole and served through vLLM on a QuietBox2 (2x p300c, 4 chips).

Runs on **p300x2** (mesh `P300x2`) — 1,048,576-token context, up to 32 concurrent sequences.

Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1).

## Quickstart

```bash
tt-model pull  ndaly/Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026 --with-weights
tt-model serve ndaly/Nemotron-3.5-Lightning-30B-A3B-NVFP4-tt-p300x2-vllm026
```

`pull --with-weights` downloads the Docker image and the [`nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4`](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4) weights at `6dbbd757ea75a8ece6e0702872e3ae53f9987728` (into your HF cache; they are not in the image). `serve` starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`.

### What was validated
Full 1M-token context on a QuietBox2 (P300x2 mesh, 4 Blackhole chips). Release validation
(2026-08-19, kernel 7.0.0-28): TPOT 15.4 ms (~65 tok/s/user) single-stream, TTFT 109 ms,
22/22 spec-test conformance. This container was re-verified on the same box on 2026-09-09
(kernel 7.0.0-30): identity, deterministic generation and the separate `reasoning` field
all pass; TTFT 110 ms; single-stream TPOT measured 42 ms (~24 tok/s/user) — identical to a
host-native run of the bring-up environment on that day, so the gap is the host, not the
container. Reasoning is returned in a separate `reasoning` field (`deepseek_r1` parser); with
`chat_template_kwargs: {"enable_thinking": false}` the parser has no `</think>` to split on
and the answer lands in `reasoning` with empty `content`.

### Try it
```bash
curl -s localhost:20000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4",
  "messages": [{"role": "user", "content": "Merge two sorted lists in Python."}],
  "max_tokens": 2048, "temperature": 0}'
```
The first boot converts the NVFP4 weights to device layout and JIT-compiles kernels (~12 min
on a QuietBox2); both are cached under `~/.cache/tt-model/<name>/` so later boots take ~4 min.
After any non-clean stop, run `tt-smi -r all` before serving again.

## Provenance

The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image:

| component | built from |
| --- | --- |
| tt-metal | a local checkout — commit not published *(dirty tree — the image includes uncommitted changes)* |
| vLLM | [`v0.26.0`](https://github.com/vllm-project/vllm/releases/tag/v0.26.0) |
| vllm-tt-plugin | a local checkout — commit not published |
| `code/` digest | `5787b8dfce9d7e38` (sha256, first 16 hex digits) |
| built | 2026-09-11T17:47:54+00:00 by tt-model 0.1.0 |