bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

FP8 quantization of bottlecapai/ThinkingCap-Qwen3.6-27B — capability of Qwen3.6-27B with 50% less thinking tokens on average, achieved by finetuning Qwen3.6-27B (Qwen Team, 2026) with online reinforcement learning while preserving the original answer quality and style.

➡️ Full model description, evaluation results (multi-seed, statistically tested), recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.6-27B.

About this quantization

Native block-wise FP8 (E4M3): weights are quantized to FP8 with 128×128 block scales (weight_scale_inv) and activations are quantized dynamically per token — quant_method: "fp8", weight_block_size: [128, 128], the same DeepSeek-V3-style format that Qwen/Qwen3.6-27B-FP8 ships. vLLM and SGLang load it natively, and the model's MTP (multi-token-prediction) head is preserved, so self-speculative decoding works (≈3.35 accepted tokens/step). ≈30 GB instead of ≈55 GB bf16 — near-lossless quality at half the memory, with FP8 tensor-core throughput on Hopper / Blackwell (Ada / Ampere serve it via weight-only FP8-Marlin kernels).

Kept in bf16: lm_head, the MTP head, the vision tower, and the Gated-DeltaNet input gates (linear_attn.in_proj_a/b, whose 48-wide output doesn't tile a 128 block).

Usage

# vLLM
vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

# SGLang
python -m sglang.launch_server --model-path bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 --trust-remote-code

For local llama.cpp / Ollama / LM Studio use, see the GGUF quantizations at bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF.

Expected performance

From our internal serving-validation harness (vLLM, single-stream, temperature 0) on a fast N=100/dataset subset of MMLU-Pro (reasoning) and RealWorldQA (vision) — a quick quant-parity + decode-speed check, not the headline accuracy evals (for the multi-seed, statistically-tested results see the main model card).

Two things to read off it: FP8 matches the bf16 finetune's accuracy within subset noise while decoding ≈34% faster (85 vs 64 tok/s), and MTP self-speculative decoding (≈3.35 accepted tokens per verify step) adds a further ≈2.4× — stacking with the finetune's token savings to ≈7× faster per task than the unquantized base.

median tokens = median completion length (the finetune's lever); task s = median tokens ÷ single-stream tok/s (real per-request time); speedup is vs the base model in standard decoding. NVIDIA NVFP4 is NVIDIA's ModelOpt NVFP4 quant of the base model, shown for comparison.

MMLU-Pro (reasoning)

config	acc	median tokens	tok/s	task s	speedup	accept_len
Qwen3.6-27B base · standard	0.86	1936	62.0	31.2	1.00×	—
Qwen3.6-27B base · MTP	0.83	2042	160.6	12.7	2.46×	3.32
ThinkingCap bf16 · standard	0.86	894	63.6	14.1	2.21×	—
ThinkingCap bf16 · MTP	0.87	884	164.2	5.4	5.78×	3.34
ThinkingCap-FP8 · standard	0.91	876	85.4	10.3	3.03×	—
ThinkingCap-FP8 · MTP	0.89	893	208.1	4.3	7.26×	3.35
NVIDIA NVFP4 (base) · standard	0.84	1842	98.4	18.7	1.67×	—
NVIDIA NVFP4 (base) · MTP	0.82	1980	239.9	8.3	3.76×	3.30

RealWorldQA (vision)

config	acc	median tokens	tok/s	task s	speedup	accept_len
Qwen3.6-27B base · standard	0.71	601	62.0	9.7	1.00×	—
Qwen3.6-27B base · MTP	0.69	545	160.6	3.4	2.85×	3.32
ThinkingCap bf16 · standard	0.76	272	63.6	4.3	2.26×	—
ThinkingCap bf16 · MTP	0.80	277	164.2	1.7	5.71×	3.34
ThinkingCap-FP8 · standard	0.80	265	85.4	3.1	3.13×	—
ThinkingCap-FP8 · MTP	0.79	290	208.1	1.4	6.93×	3.35
NVIDIA NVFP4 (base) · standard	0.67	535	98.4	5.4	1.80×	—
NVIDIA NVFP4 (base) · MTP	0.67	537	239.9	2.2	4.41×	3.30