qwen3.8-27b-dflash2-vision-p300x2-q4kv

Unified Qwen/Qwen3.8-27B serving on a Tenstorrent P300x2 (4 Blackhole chips, 4-way tensor parallel): Q4 KV cache, up to eight mixed text/image requests, up to four images per request, automatic lossless DFlash2 acceleration for compatible greedy requests, and an ordinary-decode fallback for sampling, logprobs, penalties, constraints, and structured output. The immutable vision-baseline-v1 tag preserves the pre-unification implementation for bisecting.

Runs on p300x2 (mesh P150x4) — 262,144-token context, up to 8 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv --with-weights
tt-model serve mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv

pull --with-weights downloads the Docker image and automatically downloads both required checkpoints: Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, incoai/Qwen3.8-27B-DFlash2 at dedf8df68adfb1afeaf7b7480c0a0243108177b4 (into your HF cache; they are not in the image). serve starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Query

OpenAI-compatible API on port 20000, model id Qwen/Qwen3.8-27B:

curl -s http://localhost:20000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model": "Qwen/Qwen3.8-27B", "messages": [{"role": "user", "content": "Write a Python function that merges two sorted lists."}], "max_tokens": 512}'

Profile

unified is the only public profile and the default. It accepts up to eight concurrent text/image requests and up to four images per request. Eligible greedy requests use DFlash automatically; requests with supported sampling or output controls use the plain lane without silently dropping parameters.

tt-model serve mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv
tt-model profiles mando2222/qwen3.8-27b-dflash2-vision-p300x2-q4kv

The endpoint is synchronous and prefix caching is disabled. Features unsupported by both execution lanes return a descriptive 4xx response.

Benchmarks

Final unified validation on four Blackhole p300c chips, firmware 19.15.0.0:

Gate	Result
Functional / tools / reasoning	10/10; 4/4; xhigh/medium/low 18/18
Decode / prefill	54.48 tok/s short; 80.59 tok/s long; 6,679.3 prefill tok/s
Context / concurrency	260,340 tokens; 4 agents in 8.4 s; 8 in 15.9 s
Vision	DFlash 49.472 tok/s; unified plain 31.503 tok/s (1.570x); plain is +2.56% vs the 30.717 tok/s standalone reference
Exact greedy parity	371 image tokens, 384 text tokens, and all 4/8 concurrent outputs
Q4 shared pool	8 x 129,298-token prompts passed at 98.4% occupancy

Historical pre-unification measurements from the immutable vision-baseline-v1 implementation follow. t/s = aggregate output tokens per second, (t/s/u) = per user; plain = the same box without speculation (changh95/qwen3.8-27b-p300x2).

Historical batch8-dflash2, t/s (t/s/u), 1, 2, 4, 8 concurrent users

ISL	OSL	1 user	2 users	4 users	8 users	plain 1 user
128	128	56.2 (56.2)	100.7 (53.1)	182.2 (46.2)	271.5 (34.9)	32.3
1,024	128	53.3 (53.3)	98.4 (49.9)	158.8 (42.3)	243.6 (31.3)	31.3
2,048	128	51.6 (51.7)	86.5 (44.4)	131.8 (34.8)	194.3 (24.9)	30.7
4,096	128	45.8 (45.8)	76.6 (38.8)	103.5 (26.9)	135.0 (17.1)	28.8
8,192	128	35.1 (35.1)	52.8 (26.5)	71.8 (18.1)	80.4 (10.4)	25.9
16,384	128	26.0 (26.0)	33.8 (17.1)	40.9 (10.3)	45.2 (5.7)	21.2
32,768	128	16.1 (16.1)	19.8 (10.2)	22.0 (5.6)	23.3 (2.9)	15.1
65,536	128	9.7 (9.7)	10.3 (5.2)	10.7 (2.7)	9.8 (1.2)	9.2
131,072	128	4.4 (4.4)	4.6 (2.3)	4.7 (1.2)	–	4.6
128	1024	51.4 (51.4)	86.7 (43.9)	168.1 (49.1)	297.3 (41.8)	32.6
8,192	1024	57.6 (57.6)	63.3 (33.3)	125.5 (36.3)	198.5 (30.1)	31.1
10,000	1024	58.5 (58.5)	64.5 (33.9)	102.1 (33.5)	185.8 (26.3)	30.9

• A dash = the KV pool does not seat that many users at that length (the workflow skips the point). At 1 to 4 users the server drafts 7 tokens per step; at 5 to 8, 3 per step.

Historical single-user-dflash2, one user

ISL	OSL	TTFT ms	TPOT ms	t/s/u	plain t/s/u
128	128	134	15.0	62.7	32.3
1,024	128	183	14.2	64.3	31.3
2,048	128	304	16.0	54.9	30.7
4,096	128	594	17.5	45.4	28.8
8,192	128	1,187	16.1	39.6	25.9
16,384	128	2,460	17.0	27.7	21.2
32,768	128	5,073	21.2	16.5	15.1
65,536	128	11,163	20.8	9.3	9.2
131,072	128	26,055	12.1	4.6	4.6
128	1024	135	12.1	82.1	32.6
8,192	1024	1,197	17.9	52.5	31.1
10,000	1024	1,515	14.3	63.6	30.9

Code (SPEED-Bench coding, 80 prompts; greedy; answers end at EOS), t/s (tok/s per user); speed-up per user

thinking	users	batch8-dflash2	single-user-dflash2	plain	speed-up
off	1	84 (111)	95 (119)	35 (36)	3.3×
off	2	152 (107)	–	–	–
off	4	247 (102)	–	–	–
off	8	233 (60)	–	111 (23)	2.6×
on	1	73 (77)	80 (83)	–	–
on	2	143 (76)	–	–	–
on	4	268 (73)	–	–	–
on	8	397 (55)	–	218 (29)	1.9×

• Per-user speed = 1000 / mean time per output token (decode only); t/s = aggregate output tokens per second over the run, which includes prefill and the gaps between a user's requests. thinking off passes "chat_template_kwargs": {"enable_thinking": false}; thinking on is the served default (the harness's prompts), with answers capped at 4,096 tokens. Speed-up = per-user speed of the faster speculative profile over plain decode at the same user count.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published
vLLM	v0.26.0
vllm-tt-plugin	a local checkout — commit not published
code/ digest	ce4a1982fe594327 (sha256, first 16 hex digits)
built	2026-09-18T16:52:38+00:00 by tt-model 0.1.0