---
license: apache-2.0
base_model: bottlecapai/ThinkingCap-Qwen3.6-27B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
tags:
  - nvfp4
  - fp4
  - nvfp4a16
  - compressed-tensors
  - llm-compressor
language:
  - en
---

# ThinkingCap-Qwen3.6-27B-NVFP4A16

NVFP4A16 quantization of
[bottlecapai/ThinkingCap-Qwen3.6-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B)
— a token-efficient **reasoning** finetune of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) by BottleCap AI (`qwen3_5`: dense hybrid GatedDeltaNet linear-attention + full-attention over 64 text layers in a 3:1 pattern, plus a vision tower and an MTP head; multimodal image-text-to-text). It matches Qwen3.6-27B answer quality while emitting ~50% fewer thinking tokens on average.

**Variant**: NVFP4 **A16** — 4-bit NVFP4 (FP4 E2M1, group size 16) weights, activations BF16. Native on NVIDIA Blackwell.
**Quantized by**: [sahilchachra](https://huggingface.co/sahilchachra)
**Tooling**: `llm-compressor` `model_free_ptq` (data-free, RTN) -> `compressed-tensors`

> This is a quantized derivative. Weights, behavior, and license follow the base
> model — see the
> [original card](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B) for full details, benchmarks, and citation.

## What is quantized

Quantized to 4-bit:

- full-attention `self_attn.{q,k,v,o}_proj`
- `mlp.{gate,up,down}_proj` (all text layers)

Kept in **BF16**: GatedDeltaNet `linear_attn` (mamba) layers, vision tower (`model.visual.*`, 27 blocks), MTP head, token embeddings, lm_head, all norms (incl. q_norm / k_norm).

### Note on what's 4-bit vs BF16 (quality/speed tradeoff)

This is a **hybrid** architecture: 48 of the 64 layers are GatedDeltaNet **linear-attention ("Mamba") layers**. Only the full-attention `self_attn.{q,k,v,o}` projections and the per-layer `mlp.{gate,up,down}` projections are quantized to 4-bit; the **Mamba `linear_attn` layers are deliberately kept in BF16** (along with the vision tower, MTP head, embeddings, lm_head and norms), because they are quantization-sensitive and keeping them full-precision preserves the reasoning quality and the concise `<think>` behavior.

As a result, roughly **two-thirds of the weight bytes read per token stay BF16** (~18 GB BF16 vs ~9 GB of 4-bit weights), so this variant's memory footprint is close to an 8-bit build and it is tuned for **quality rather than peak throughput** — on Blackwell it can run a little slower than a full W8A8/FP8 build of the base model.

The Mamba layers **can also be quantized** (to shrink the model further and speed up memory-bound decoding), but **accuracy may take a hit** — this build intentionally trades that extra speed for output quality.

## Calibration

Data-free — weight-only (`model_free_ptq`, round-to-nearest); no calibration data. Weights are quantized by streaming the safetensors from disk.

## Prompt template & sampling

This is a **reasoning ("thinking") model**. Use the Qwen3.6 chat template — ChatML (`<|im_start|>role … <|im_end|>`) with a `<think>…</think>` reasoning trace, thinking enabled by default. Apply it via `tokenizer.apply_chat_template(messages, add_generation_prompt=True)` (or the processor for image inputs); do not hand-format prompts. See the base [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) for full usage details.

**Recommended sampling**: `temperature=1.0, top_p=0.95, top_k=20, min_p=0.0` with thinking on (the base model's recommended sampling, per the original card).


## Usage (vLLM)

```python
from vllm import LLM, SamplingParams

# This is a multimodal checkpoint: the vision tower is kept in BF16
# (only the text / MoE weights are 4-bit). vLLM builds the full model.
llm = LLM(
    model="sahilchachra/ThinkingCap-Qwen3.6-27B-NVFP4A16",
    trust_remote_code=True,
)
out = llm.chat(
    [{"role": "user", "content": "Hello!"}],
    SamplingParams(temperature=0.6, top_p=0.95, max_tokens=512),
)
print(out[0].outputs[0].text)
```

Serving via the CLI, pass the flag directly:

```bash
vllm serve sahilchachra/ThinkingCap-Qwen3.6-27B-NVFP4A16 \
    --trust-remote-code \
    --max-model-len 262144 --reasoning-parser qwen3
```