README.md

---
base_model: bottlecapai/ThinkingCap-Qwen3.6-27B
base_model_relation: quantized
library_name: mlx
license: apache-2.0
pipeline_tag: image-text-to-text
tags:
- mlx
- quantized
- mixed-precision
- 4bit
- 8bit
- optiq
- apple-silicon
- image-text-to-text
- vision-language
- qwen3_6
- token-efficient
- efficient-thinking
---

# mlx-community/ThinkingCap-Qwen3.6-27B-OptiQ-4bit

> **Built with [mlx-optiq](https://mlx-optiq.com)**, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. [Try the Lab](https://mlx-optiq.com/docs/lab/) · [All OptiQ quants](https://mlx-optiq.com/models) · [Docs](https://mlx-optiq.com/docs/)

A 4-bit mixed-precision MLX quant of [bottlecapai/ThinkingCap-Qwen3.6-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B), a token-efficient reasoning fine-tune of Qwen3.6-27B. Sensitive layers are kept at 8-bit and robust ones at 4-bit, so quality holds up far better than a uniform 4-bit quant at nearly the same size.

**Image input works.** The vision tower is kept at bf16 in a sidecar, so this quant takes images as well as text.

51.8 GB of bf16 weights become **19 GB**, which fits a 24 GB Mac.

## Quantization details

| Property | Value |
|---|---|
| Predominant precision | 4-bit |
| Layers at 8-bit (sensitive) | 220 |
| Layers at 4-bit (robust) | 276 |
| Total quantized layers | 496 |
| Achieved bits per weight | 4.769 |
| Group size | 64 |
| Vision tower | bf16, 333 tensors, in `optiq/optiq_vision.safetensors` |
| Bundled MTP head | `optiq/mtp.safetensors` (4-bit projections, BF16 norms) |
| Size on disk | 19 GB (language 18 GB, sidecars 1.2 GB), from a 51.8 GB bf16 base |

We follow the same naming convention `llama.cpp` uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is the predominant precision, not the weighted average.

### How the bit-widths were chosen

The per-layer allocation is transferred from [mlx-community/Qwen3.6-27B-OptiQ-4bit](https://huggingface.co/mlx-community/Qwen3.6-27B-OptiQ-4bit), where it was derived by a KL-divergence sensitivity sweep against the bf16 reference on a [six-domain calibration mix](https://mlx-optiq.com/blog/calibration-mix).

ThinkingCap is a fine-tune of `Qwen/Qwen3.6-27B` and its architecture is unchanged, so all 496 quantizable layers map across exactly, and the allocation lands at the same 4.769 bits per weight when recomputed against ThinkingCap's own tensors.

To be precise about what that means: these are **measured** bit-widths, not a static rule-of-thumb recipe. But they were measured on the base model, not on this fine-tune. Fine-tuning shifts weights, so ThinkingCap's own per-layer sensitivities could differ somewhat from the base's. Which layers are fragile is mostly a property of the architecture, so the transfer is sound, but it is a transfer and you should know that.

Only the language tower is quantized. The vision tower stays at bf16, which is how every OptiQ VLM ships: it is a small fraction of the weights, so quantizing it costs quality for very little disk.

## Usage

### Text

Load it with `mlx-lm` and use it as usual. The sidecars live in an `optiq/` subfolder, so a stock `*.safetensors` glob ignores them and `mlx-lm` sees a clean language model.

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/ThinkingCap-Qwen3.6-27B-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Explain quantum computing in simple terms.",
    max_tokens=512,
)
```

This is a reasoning model: it thinks inside `<think>...</think>` before answering, so give it enough `max_tokens` to finish.

### Images

Image input needs [`mlx-optiq`](https://mlx-optiq.com/), which loads the bf16 vision sidecar and feeds the merged embeddings to the quantized language tower:

```bash
pip install mlx-optiq
```

```python
from PIL import Image
from optiq.runtime.engine import OptiqEngine

engine = OptiqEngine("mlx-community/ThinkingCap-Qwen3.6-27B-OptiQ-4bit")
answer = engine.generate(
    "What is in this image?",
    images=[Image.open("photo.jpg")],
    max_tokens=512,
)
print(answer.text)
```

Or serve it over an OpenAI-compatible endpoint that accepts image content parts:

```bash
optiq serve --model mlx-community/ThinkingCap-Qwen3.6-27B-OptiQ-4bit
```

### Speculative decoding (MTP)

The base ships a Multi-Token Prediction head, bundled here as `optiq/mtp.safetensors`:

```bash
optiq serve --model mlx-community/ThinkingCap-Qwen3.6-27B-OptiQ-4bit --mtp
```

## Verification

Text and image generation were both checked on the finished artifact before release. No task benchmarks were run on this quant; for measured quality numbers on the base architecture, see the [Qwen3.6-27B OptiQ card](https://huggingface.co/mlx-community/Qwen3.6-27B-OptiQ-4bit).

Quantization does not change the behaviour or alignment of the base model. Use it under the same terms as [the original](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B).