---
license: apache-2.0
base_model:
- bottlecapai/ThinkingCap-Qwen3.6-27B
tags:
- gguf
- llama.cpp
- magicquant
- hybrid-quantization
- mtp
- multi-token-prediction
- speculative-decoding
- qwen3_6
- efficient-thinking
- conversational
library_name: llama.cpp
---

# ThinkingCap-Qwen3.6-27B — MagicQuant hybrid GGUFs (Q4 / Q5 / Q6)

Per-tensor-group hybrid quantizations of
[bottlecapai/ThinkingCap-Qwen3.6-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B)
(a Qwen3.6-27B finetune tuned for efficient thinking — comparable capability with
~50% fewer thinking tokens on average) found by **MagicQuant** evolutionary search:
a measured Predict→Measure→Learn loop with real perplexity + KL-divergence guard on
every surviving candidate, imatrix-weighted encoding, and stream-aware sampling. Each
tier's per-group precision layout was selected from 16 fully-measured candidate hybrids
rather than applied uniformly.

> **Vision-capable.** The base is a Qwen3.5-VL vision-language model. These GGUFs are the
> quantized **text** model; pair any of them with the included
> `mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf` (the f16 vision projector, 0.86 GiB) to run
> image input. Verified end-to-end (image description) with the Q4 quant + this mmproj.

The embedded **MTP (multi-token-prediction) head** is preserved, so these work with
llama.cpp speculative decoding using the *same file* as its own draft model.

## Vision (image input)

Serve any tier with the vision projector:

```bash
llama-server -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf \
  --mmproj mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf \
  -c 8192 -ngl 99 -fa on
```

Then send images via the OpenAI-compatible `image_url` content field. (MTP speculative
decoding and vision can each be enabled; combine flags as needed.)

## Files

| File | Size | PPL (wikitext-2) | vs BF16 baseline (6.7803) | Layout |
|---|---|---|---|---|
| `ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf` | 14.65 GiB | **6.8931** | +0.113 | attn/FFN/embed Q4_K_M, head Q6_K |
| `ThinkingCap-Qwen3.6-27B-Q5_K_M.gguf` | 20.89 GiB | **6.8270** | +0.047 | uniform Q6_K (search winner for the tier) |
| `ThinkingCap-Qwen3.6-27B-Q6_K.gguf` | 23.00 GiB | **6.8304** | +0.050 | mixed: E/K/O BF16, U Q6_K, D IQ4_NL, attn-Q MXFP4 |

All measured identically: 100 chunks, ctx 512, wikitext-2, on an AMD Strix Halo APU
(gfx1151). The search converged over 3 rounds (mean abs residual 2.20 → 1.56). Q5 and Q6
land within measurement noise of each other (~0.05 above the BF16 baseline); Q5's lean
uniform-Q6_K config is 2 GiB smaller for essentially the same quality.

## MTP speculative decoding

The embedded nextn/MTP head is preserved in every file. Serve with the model as its own draft:

```bash
llama-server -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf \
  -md ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf --spec-type draft-mtp \
  -c 8192 -ngl 99 -fa on -ctk q8_0 -ctv q8_0
```

> Requires a llama.cpp build with `qwen35`/qwen3.6 arch + MTP (`--spec-type draft-mtp`) support.

## Notes

- Chat template is embedded (no patching needed); architecture `qwen35`.
- Group legend: E=embeddings, H=lm head, Q/K/O=attention, U/D=FFN up/down, S=SSM (mamba) ops.
- `mmproj-*-f16.gguf` (0.86 GiB) is the vision projector — required only for image input.
- This is a hybrid (linear-attention + attention) architecture; SSM conv/1D tensors are
  kept at F32 where the block size requires it.
- Sibling repo with AMD-native (ROCmFPX fork-only) builds of the same layouts:
  [lmcoleman/ThinkingCap-Qwen3.6-27B-ROCmFPX-GGUF](https://huggingface.co/lmcoleman/ThinkingCap-Qwen3.6-27B-ROCmFPX-GGUF).

Built with Foundry + MagicQuant (evolutionary per-tensor-group hybrid quantization;
measured search with imatrix + KL guard).