qwen3-vl-8b-gguf (Enclave model volume)

Curated model volume for Enclave confidential inference: Qwen3-VL-8B-Instruct at Q8_0 (8.7 GB) with its F16 vision projector (1.2 GB), bundled with the official tokenizer.json so the volume is self-contained.

This is the first Enclave volume that carries an mmproj, and it carries one on purpose. The text-only volumes in this org drop the projector files — qwen3.5-9b-gguf says so explicitly — because llm-chat's ggml path used to preload a single LLM gguf and had no way to use a vision tower. It does now: the host pairs the weights with the projector it finds beside them, and a deployment that attaches this volume can be shown a picture. The projector is loaded lazily, on the first image a deployment is actually sent, so a chat that never attaches one never pays its VRAM.

Two ggufs in one volume stay unambiguous the same way a multi-quant tree does: the projector is identified by its mmproj name and taken out of the model pick before "exactly one gguf" is decided, and the volume table names Qwen3VL-8B-Instruct-Q8_0.gguf as the preload anyway.

Q8_0 rather than Q4_K_M is deliberate. Serving cost here is dominated by the KV cache — Qwen3-VL is DENSE, so all 36 layers hold KV, 144 KiB per token at f16 (72 KiB at q8_0) — which puts the whole build at ~25 GB against a 25% H200 share's ~35 GB, where Q4 would save only ~3.7 GB of that. Vision work is also where quantization damage hurts most: reading a value off a chart or small text in a screenshot is exactly the task where a weaker quant returns a confident wrong answer rather than an obviously bad one. The F16 projector follows the same logic, and costs 0.4 GB over Q8_0 for an encoder that runs once per image.

Provenance

File	Upstream	Revision	sha256
Qwen3VL-8B-Instruct-Q8_0.gguf	Qwen/Qwen3-VL-8B-Instruct-GGUF	f982a07559d4a2f6c8744d840bf6fccab30eea96	0d264b3941185d00a74f75c4245521dae088ff1efc90ab8d1754e83f5844adb0
mmproj-Qwen3VL-8B-Instruct-F16.gguf	Qwen/Qwen3-VL-8B-Instruct-GGUF	f982a07559d4a2f6c8744d840bf6fccab30eea96	ca524100ebf825c9a870db1c580d03879e0da0ab2541697e2458e64891cf9d38
tokenizer.json	Qwen/Qwen3-VL-8B-Instruct	0c351dd01ed87e9c1b53cbc748cba10e6187ff3b	a5d85b6dcc535e6b93115a9ef287e6132fdbf30270da6218194ba742261173c7

Upstream ships F16 and Q4_K_M weights and a Q8_0 projector as well (~32 GB for the repo); this volume carries only the served pair plus the tokenizer, the same rationale as the other volumes here.

The tokenizer is the Qwen3-VL one: vocab 151936, <|im_end|> 151645 / <|endoftext|> 151643, with the vision tokens (<|vision_start|> 151652, <|image_pad|> 151655) the projector's marker tokens use. It is NOT byte-identical to the Qwen3.5 family tokenizer in qwen3.5-9b-gguf (different vocab), so this model does not share a draft model with them.

LICENSE is the Apache-2.0 text; the upstream is Apache-2.0.

Geometry

36 layers, 8 KV heads, head_dim 128, vocab 151936, chatml template, dense attention (no kv_layers reduction). Vision: SigLIP-family encoder, 16px patches with 2x2 spatial merge, M-RoPE image positions — all handled inside llama.cpp's mtmd, so nothing about it appears in a deployment's config beyond "vision": true.

Usage on Enclave

Wrapped as a read-only attested volume (dm-verity; the root hash is bound into the enclave's attestation) and attached by name; the guest reads it at /models/<name>. The deployment's llm-chat config marks the catalog entry "vision": true, which is what makes the playground offer the attach button and the API accept OpenAI content-parts messages carrying images.

Vision needs a node whose llama.cpp toolchain includes libmtmd. On an older node this volume still serves as a text model and the deployment reports that images are unavailable, rather than silently ignoring them.