---
tags:
- blackhole
- p300x2
- tt-model-cache
- tt-model-container
- vllm-plugin
---

# qwen3.8-27b-dflash2-vision-p300x2-q4kv

Unified [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) serving on a Tenstorrent P300x2 (4 Blackhole chips, 4-way tensor parallel): Q4 KV cache, up to eight mixed text/image requests, up to four images per request, automatic lossless DFlash2 acceleration for compatible greedy requests, and an ordinary-decode fallback for sampling, logprobs, penalties, constraints, and structured output. The immutable `vision-baseline-v1` tag preserves the pre-unification implementation for bisecting.

Runs on **p300x2** (mesh `P150x4`) — 262,144-token context, up to 8 concurrent sequences.

Packaged and published with [tt-model-manager](https://github.com/tenstorrent/tt-model-manager) 0.1.0 (manifest schema 5.1).

## Quickstart

```bash
tt-model pull  mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv --with-weights
tt-model serve mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
```

`pull --with-weights` downloads the Docker image and automatically downloads both required checkpoints: [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B) at `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`, [`incoai/Qwen3.8-27B-DFlash2`](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) at `dedf8df68adfb1afeaf7b7480c0a0243108177b4` (into your HF cache; they are not in the image). `serve` starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs `Application startup complete`.

### Query

OpenAI-compatible API on port 20000, model id `Qwen/Qwen3.8-27B`:

```bash
curl -s http://localhost:20000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model": "Qwen/Qwen3.8-27B", "messages": [{"role": "user", "content": "Write a Python function that merges two sorted lists."}], "max_tokens": 512}'
```

### Profile

`unified` is the only public profile and the default. It accepts up to eight concurrent text/image requests and up to four images per request. Eligible greedy requests use DFlash automatically; requests with supported sampling or output controls use the plain lane without silently dropping parameters.

```bash
tt-model serve mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
tt-model profiles mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
```

The endpoint is synchronous and prefix caching is disabled. Features unsupported by both execution lanes return a descriptive 4xx response.

## Benchmarks

Final `unified` validation on four Blackhole p300c chips, firmware 19.15.0.0:

| Gate | Result |
|---|---|
| Functional / tools / reasoning | 10/10; 4/4; xhigh/medium/low 18/18 |
| Decode / prefill | 54.48 tok/s short; 80.59 tok/s long; 6,679.3 prefill tok/s |
| Context / concurrency | 260,340 tokens; 4 agents in 8.4 s; 8 in 15.9 s |
| Vision | DFlash 49.472 tok/s; unified plain 31.503 tok/s (1.570x); plain is +2.56% vs the 30.717 tok/s standalone reference |
| Exact greedy parity | 371 image tokens, 384 text tokens, and all 4/8 concurrent outputs |
| Q4 shared pool | 8 x 129,298-token prompts passed at 98.4% occupancy |

Historical pre-unification measurements from the immutable `vision-baseline-v1` implementation follow. `t/s` = aggregate output tokens per second, `(t/s/u)` = per user; plain = the same box without speculation ([changh95/qwen3.8-27b-p300x2](https://huggingface.co/changh95/qwen3.8-27b-p300x2)).

### Historical `batch8-dflash2`, t/s (t/s/u), 1, 2, 4, 8 concurrent users

| ISL | OSL | 1 user | 2 users | 4 users | 8 users | plain 1 user |
|---|---|---|---|---|---|---|
| 128 | 128 | 56.2 (56.2) | 100.7 (53.1) | 182.2 (46.2) | 271.5 (34.9) | 32.3 |
| 1,024 | 128 | 53.3 (53.3) | 98.4 (49.9) | 158.8 (42.3) | 243.6 (31.3) | 31.3 |
| 2,048 | 128 | 51.6 (51.7) | 86.5 (44.4) | 131.8 (34.8) | 194.3 (24.9) | 30.7 |
| 4,096 | 128 | 45.8 (45.8) | 76.6 (38.8) | 103.5 (26.9) | 135.0 (17.1) | 28.8 |
| 8,192 | 128 | 35.1 (35.1) | 52.8 (26.5) | 71.8 (18.1) | 80.4 (10.4) | 25.9 |
| 16,384 | 128 | 26.0 (26.0) | 33.8 (17.1) | 40.9 (10.3) | 45.2 (5.7) | 21.2 |
| 32,768 | 128 | 16.1 (16.1) | 19.8 (10.2) | 22.0 (5.6) | 23.3 (2.9) | 15.1 |
| 65,536 | 128 | 9.7 (9.7) | 10.3 (5.2) | 10.7 (2.7) | 9.8 (1.2) | 9.2 |
| 131,072 | 128 | 4.4 (4.4) | 4.6 (2.3) | 4.7 (1.2) | – | 4.6 |
| 128 | 1024 | 51.4 (51.4) | 86.7 (43.9) | 168.1 (49.1) | 297.3 (41.8) | 32.6 |
| 8,192 | 1024 | 57.6 (57.6) | 63.3 (33.3) | 125.5 (36.3) | 198.5 (30.1) | 31.1 |
| 10,000 | 1024 | 58.5 (58.5) | 64.5 (33.9) | 102.1 (33.5) | 185.8 (26.3) | 30.9 |

* A dash = the KV pool does not seat that many users at that length (the workflow skips the point). At 1 to 4 users the server drafts 7 tokens per step; at 5 to 8, 3 per step.

### Historical `single-user-dflash2`, one user

| ISL | OSL | TTFT ms | TPOT ms | t/s/u | plain t/s/u |
|---|---|---|---|---|---|
| 128 | 128 | 134 | 15.0 | 62.7 | 32.3 |
| 1,024 | 128 | 183 | 14.2 | 64.3 | 31.3 |
| 2,048 | 128 | 304 | 16.0 | 54.9 | 30.7 |
| 4,096 | 128 | 594 | 17.5 | 45.4 | 28.8 |
| 8,192 | 128 | 1,187 | 16.1 | 39.6 | 25.9 |
| 16,384 | 128 | 2,460 | 17.0 | 27.7 | 21.2 |
| 32,768 | 128 | 5,073 | 21.2 | 16.5 | 15.1 |
| 65,536 | 128 | 11,163 | 20.8 | 9.3 | 9.2 |
| 131,072 | 128 | 26,055 | 12.1 | 4.6 | 4.6 |
| 128 | 1024 | 135 | 12.1 | 82.1 | 32.6 |
| 8,192 | 1024 | 1,197 | 17.9 | 52.5 | 31.1 |
| 10,000 | 1024 | 1,515 | 14.3 | 63.6 | 30.9 |

### Code (SPEED-Bench `coding`, 80 prompts; greedy; answers end at EOS), t/s (tok/s per user); speed-up per user

| thinking | users | `batch8-dflash2` | `single-user-dflash2` | plain | speed-up |
|---|---|---|---|---|---|
| off | 1 | 84 (111) | 95 (119) | 35 (36) | 3.3× |
| off | 2 | 152 (107) | – | – | – |
| off | 4 | 247 (102) | – | – | – |
| off | 8 | 233 (60) | – | 111 (23) | 2.6× |
| on | 1 | 73 (77) | 80 (83) | – | – |
| on | 2 | 143 (76) | – | – | – |
| on | 4 | 268 (73) | – | – | – |
| on | 8 | 397 (55) | – | 218 (29) | 1.9× |

* Per-user speed = 1000 / mean time per output token (decode only); `t/s` = aggregate output tokens per second over the run, which includes prefill and the gaps between a user's requests. `thinking off` passes `"chat_template_kwargs": {"enable_thinking": false}`; `thinking on` is the served default (the harness's prompts), with answers capped at 4,096 tokens. Speed-up = per-user speed of the faster speculative profile over plain decode at the same user count.

## Provenance

The exact sources the image was built from — `code/` in this repo is byte-identical to the model code inside the image:

| component | built from |
| --- | --- |
| tt-metal | a local checkout — commit not published |
| vLLM | [`v0.26.0`](https://github.com/vllm-project/vllm/releases/tag/v0.26.0) |
| vllm-tt-plugin | a local checkout — commit not published |
| `code/` digest | `ce4a1982fe594327` (sha256, first 16 hex digits) |
| built | 2026-09-18T16:52:38+00:00 by tt-model 0.1.0 |