Image-Text-to-Text
Transformers
Safetensors
qwen3_5
qwen3_6
token-efficient
efficient-thinking
conversational
Eval Results
Instructions to use bottlecapai/ThinkingCap-Qwen3.6-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bottlecapai/ThinkingCap-Qwen3.6-27B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.6-27B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B") model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use bottlecapai/ThinkingCap-Qwen3.6-27B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bottlecapai/ThinkingCap-Qwen3.6-27B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bottlecapai/ThinkingCap-Qwen3.6-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B
- SGLang
How to use bottlecapai/ThinkingCap-Qwen3.6-27B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bottlecapai/ThinkingCap-Qwen3.6-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bottlecapai/ThinkingCap-Qwen3.6-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.6-27B with Docker Model Runner:
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B
| base_model: "Qwen/Qwen3.6-27B" | |
| base_model_relation: finetune | |
| library_name: transformers | |
| tags: | |
| - qwen3_6 | |
| - token-efficient | |
| - efficient-thinking | |
| license: apache-2.0 | |
| <a href="https://www.bottlecapai.com/"><img src="cap_header.png" alt="ThinkingCap β BottleCap AI" width="100%"></a> | |
| # ThinkingCap: Qwen 3.6 27B | |
| Capability of Qwen3.6-27B with **50% less** thinking tokens on average, and over **90% less** in best cases. | |
| Achieved via finetuning [Qwen3.6-27B (Qwen Team, 2026)](https://huggingface.co/Qwen/Qwen3.6-27B) with state-of-the-art algorithms on a curated set of problems of various domains and difficulty. We designed the finetuning to be as minimally invasive as possible, preserving all of the original answer quality and style of Qwen, while being more token efficient. Check [the blogpost](https://www.bottlecapai.com/thinkingcap-qwen3-6-27b) for more details. | |
| We rigorously evaluate the resulting checkpoint across general reasoning, non-reasoning multiple-choice question answering, everyday multi-turn conversations, system prompt adherence, safety, math, code and agentic use cases. Due to the high variability of reasoning quality at Qwen-recommended sampling temperature 1.0, we run each benchmark with multiple seeds and do statistical significance testing on all the results. We evaluate both in domain (holdout parts of selected datasets included in training) and out of domain. | |
| <p align="center"><img src="thinkingcap_6x.gif" alt="ThinkingCap reasoning demo (6Γ speed)" width="480" style="display:block;margin:0 auto"></p> | |
| ## Out-of-domain token efficiency | |
| <table width="100%" style="border-collapse:collapse;width:100%;display:table;table-layout:auto;font-size:14px;line-height:1.4;"> | |
| <thead> | |
| <tr><th rowspan="2" style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);vertical-align:bottom;color:#6b748a;font-weight:600;">Benchmark</th><th colspan="2" style="text-align:center;padding:6px 12px 3px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;">Accuracy</th><th colspan="3" style="text-align:center;padding:6px 12px 3px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;">Thinking tokens</th></tr> | |
| <tr><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Base</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Ours</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Base</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Ours</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Reduction</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr><td colspan="6" style="text-align:left;padding:7px 12px;background:rgba(199,185,255,0.16);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;border-bottom:1px solid rgba(128,128,128,0.22);">Knowledge & reasoning</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">GPQA-Diamond</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">85.5<span style="color:#6b748a;font-size:11px;"> Β±1.4</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">83.8<span style="color:#6b748a;font-size:11px;"> Β±1.9</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">10,777</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">3,351</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 67.8%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">SuperGPQA</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">64.0<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">64.0<span style="color:#6b748a;font-size:11px;"> Β±0.1</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">8,246</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">3,384</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 58.4%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">MMLU-Pro</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">85.9<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">85.4<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">3,455</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,290</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 53.7%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">MMLU-Redux</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">93.9<span style="color:#6b748a;font-size:11px;"> Β±0.1</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">93.9<span style="color:#6b748a;font-size:11px;"> Β±0.1</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">947</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">406</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 44.8%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">C-Eval</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">90.6<span style="color:#6b748a;font-size:11px;"> Β±0.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">90.3<span style="color:#6b748a;font-size:11px;"> Β±0.6</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,279</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">663</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 47.1%</td></tr> | |
| <tr><td colspan="6" style="text-align:left;padding:7px 12px;background:rgba(199,185,255,0.16);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;border-bottom:1px solid rgba(128,128,128,0.22);">Math & code</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">HMMT (Nov 2025)</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">88.0<span style="color:#6b748a;font-size:11px;"> Β±3.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">84.7<span style="color:#6b748a;font-size:11px;"> Β±3.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">39,277</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">27,388</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 38.0%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">LiveCodeBench</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">80.7<span style="color:#6b748a;font-size:11px;"> Β±0.6</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">84.3<span style="color:#6b748a;font-size:11px;"> Β±1.0</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">15,744</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">10,158</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 41.1%</td></tr> | |
| <tr><td colspan="6" style="text-align:left;padding:7px 12px;background:rgba(199,185,255,0.16);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;border-bottom:1px solid rgba(128,128,128,0.22);">Long-context & multimodal</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">LongBench v2</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">62.6<span style="color:#6b748a;font-size:11px;"> Β±3.6</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">60.2<span style="color:#6b748a;font-size:11px;"> Β±1.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,765</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,091</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 39.1%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">RealWorldQA</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">82.4<span style="color:#6b748a;font-size:11px;"> Β±0.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">81.9<span style="color:#6b748a;font-size:11px;"> Β±1.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">2,959</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">913</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 48.5%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">AA-LCR</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">76.2<span style="color:#6b748a;font-size:11px;"> Β±3.0</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">74.2<span style="color:#6b748a;font-size:11px;"> Β±2.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">2,455</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,337</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 45.5%</td></tr> | |
| <tr><td colspan="6" style="text-align:left;padding:7px 12px;background:rgba(199,185,255,0.16);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;border-bottom:1px solid rgba(128,128,128,0.22);">Instruction following & agentic</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">System-prompt adherence</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">80.6<span style="color:#6b748a;font-size:11px;"> Β±1.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">81.5<span style="color:#6b748a;font-size:11px;"> Β±1.8</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,737</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">976</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 40.0%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">Claw-Eval <span style="color:#6b748a;font-size:11px;">think/task</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">87.0<span style="color:#6b748a;font-size:11px;"> Β±1.9</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">84.4<span style="color:#6b748a;font-size:11px;"> Β±1.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">919</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">689</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 25.2%</td></tr> | |
| <tr><td style="text-align:left;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;">Macro average</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;">81.5</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;font-variant-numeric:tabular-nums;">80.7</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;border-left:1px solid rgba(128,128,128,0.22);">β</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;">β</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;color:#0a9d68;font-variant-numeric:tabular-nums;">β 45.8%</td></tr> | |
| </tbody> | |
| </table> | |
| <sub>Claw-Eval thinking tokens are per-task (agentic; not a single-turn trace).</sub> | |
| **Settings** | |
| - **Models:** base `Qwen/Qwen3.6-27B` vs `bottlecapai/ThinkingCap-Qwen3.6-27B` (shown as `Ours` in the table). | |
| - **Seeds:** 5 per condition; thinking on; cells are mean Β± 95% CI across seeds. | |
| - **Decoding:** thinking on; sampling `temperature=1.0, top_p=0.95, top_k=20, min_p=0.0` (`bottlecapai/ThinkingCap-Qwen3.6-27B` uses the base's sampling). | |
| - **Max generation tokens:** 100,000 for the general suite (gpqa_diamond, mmlu_pro, longbench_v2, realworldqa) and AA-LCR; 250,000 for HMMT (Nov 2025); 32,768 for supergpqa and livecodebench; 16,384 for ceval and mmlu_redux; 15,000 for llm-system-prompts-benchmark; 49,152 for Claw-Eval. | |
| - **Metrics** β the columns mirror the table: | |
| - **Accuracy** (Base / Ours) β fraction correct (exact/regex match; soft compliance for llm-system-prompts-benchmark; judge task-score for Claw-Eval; judge CORRECT/INCORRECT for AA-LCR). | |
| - **Thinking tokens** (Base / Ours) β mean length of the single-turn `<think>` trace (think-per-task for Claw-Eval). | |
| - **Reduction** β the average per-question thinking-token saving: base and `Ours` are paired on the same question (each side seed-averaged), each question's `(base β cap)/base` is taken, then averaged over shared questions (a larger β = a bigger saving). | |
| - **Macro average** (bottom row) β equal-weight mean across benchmarks. | |
| We separately track two trace-quality failure modes, reported only in aggregate: **looping** β the model gets stuck repeating the same reasoning chain (sometimes a single sentence), never finishing its thinking; detected from the fraction of repetitive n-grams β and **truncation** β the `<think>` trace never closes because the model hits the generation-token cap while still reasoning, so no answer is produced. Across all out-of-domain responses, truncation drops from **2.9% to 0.4%** while looping stays negligible (**~0.2%**). | |
| ## In-domain evals | |
| Holdout **test** splits of datasets whose train splits are part of the finetuning mix β quality retention on in-distribution tasks (in contrast to the out-of-domain benchmarks above). | |
| <table width="100%" style="border-collapse:collapse;width:100%;display:table;table-layout:auto;font-size:14px;line-height:1.4;"> | |
| <thead> | |
| <tr><th rowspan="2" style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);vertical-align:bottom;color:#6b748a;font-weight:600;">Benchmark</th><th colspan="2" style="text-align:center;padding:6px 12px 3px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;">Accuracy</th><th colspan="3" style="text-align:center;padding:6px 12px 3px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;">Thinking tokens</th></tr> | |
| <tr><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Base</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Ours</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Base</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Ours</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Reduction</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">GSM8K</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">93.3<span style="color:#6b748a;font-size:11px;"> Β±1.5</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">96.5<span style="color:#6b748a;font-size:11px;"> Β±0.3</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">3,175</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">648</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 74.1%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">ARC-Challenge</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">97.0<span style="color:#6b748a;font-size:11px;"> Β±0.3</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">97.6<span style="color:#6b748a;font-size:11px;"> Β±0.4</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">966</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">335</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 51.5%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">ARC-Easy</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">99.3<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">99.4<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">566</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">260</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 44.5%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">CommonsenseQA</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">86.7<span style="color:#6b748a;font-size:11px;"> Β±0.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">88.2<span style="color:#6b748a;font-size:11px;"> Β±0.9</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,118</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">273</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 64.1%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">OpenBookQA</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">96.0<span style="color:#6b748a;font-size:11px;"> Β±0.5</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">96.7<span style="color:#6b748a;font-size:11px;"> Β±0.6</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">858</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">248</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 59.5%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">QASC</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">91.7<span style="color:#6b748a;font-size:11px;"> Β±0.7</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">92.2<span style="color:#6b748a;font-size:11px;"> Β±0.5</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,258</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">348</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 61.9%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">SciQ</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">97.0<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">97.5<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">766</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">276</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 48.3%</td></tr> | |
| <tr><td style="text-align:left;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;">Macro average</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;">94.4</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;font-variant-numeric:tabular-nums;">95.4</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;border-left:1px solid rgba(128,128,128,0.22);">β</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;">β</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;color:#0a9d68;font-variant-numeric:tabular-nums;">β 57.7%</td></tr> | |
| </tbody> | |
| </table> | |
| **Settings** | |
| - **Seeds:** 5 per condition; thinking on; cells are mean Β± 95% CI across seeds. | |
| - **Decoding:** sampling `temperature=1.0, top_p=0.95, top_k=20, min_p=0.0` (`bottlecapai/ThinkingCap-Qwen3.6-27B` uses the base's sampling). | |
| - **Max generation tokens:** 15,000 for GSM8K; 8,192 for the MCQ sets. | |
| - **Data:** GSM8K is the full 1,319-row test split; the MCQ sets are capped at 1,000 rows (OpenBookQA = 500 and QASC = 926 are smaller, so full). | |
| - **Metrics:** **Accuracy** β exact-match on the final answer (GSM8K) / last-letter multiple-choice match (MCQ). **Thinking tokens**, **Reduction** and **Macro average** are as defined for the token-efficiency table above, as are the **looping** and **truncation** failure modes: across all in-domain responses, truncation drops from **1.6% to 0.03%** while looping is negligible for both (**β€0.01%**). | |
| ## Guardrails preservation | |
| Brevity finetuning leaves safety behaviour intact: on both safety sets `bottlecapai/ThinkingCap-Qwen3.6-27B` refuses harmful/jailbreak prompts at the base model's rate (statistically indistinguishable) while still spending fewer thinking tokens. | |
| <table width="100%" style="border-collapse:collapse;width:100%;display:table;table-layout:auto;font-size:14px;line-height:1.4;"> | |
| <thead> | |
| <tr><th rowspan="2" style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);vertical-align:bottom;color:#6b748a;font-weight:600;">Benchmark</th><th colspan="2" style="text-align:center;padding:6px 12px 3px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;">SAFE %</th><th colspan="3" style="text-align:center;padding:6px 12px 3px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#7a67e6;font-weight:700;font-size:11px;letter-spacing:0.06em;text-transform:uppercase;">Thinking tokens</th></tr> | |
| <tr><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Base</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Ours</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Base</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Ours</th><th style="text-align:center;padding:3px 12px 8px;border-bottom:1px solid rgba(128,128,128,0.22);color:#6b748a;font-weight:600;font-size:12px;">Reduction</th></tr> | |
| </thead> | |
| <tbody> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">Nemotron-Safety</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">98.9<span style="color:#6b748a;font-size:11px;"> Β±0.8</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">99.0<span style="color:#6b748a;font-size:11px;"> Β±0.4</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">1,242</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">933</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 23.8%</td></tr> | |
| <tr><td style="text-align:left;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;font-size:13px;white-space:nowrap;">HEx-PHI</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">99.9<span style="color:#6b748a;font-size:11px;"> Β±0.2</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">100.0<span style="color:#6b748a;font-size:11px;"> Β±0.0</span></td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">693</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;">543</td><td style="text-align:center;padding:8px 12px;border-bottom:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;white-space:nowrap;color:#0a9d68;font-weight:700;">β 20.0%</td></tr> | |
| <tr><td style="text-align:left;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;">Macro average</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;border-left:1px solid rgba(128,128,128,0.22);font-variant-numeric:tabular-nums;">99.4</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;font-variant-numeric:tabular-nums;">99.5</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;border-left:1px solid rgba(128,128,128,0.22);">β</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;">β</td><td style="text-align:center;padding:9px 12px;border-top:2px solid rgba(128,128,128,0.22);background:rgba(199,185,255,0.10);font-weight:750;color:#0a9d68;font-variant-numeric:tabular-nums;">β 21.9%</td></tr> | |
| </tbody> | |
| </table> | |
| **Settings** | |
| - **Seeds:** 5 per condition; thinking on; cells are mean Β± 95% CI across seeds. | |
| - **Benchmarks:** `Nemotron-Safety` β a 300-prompt sample of the `safety` split of `nvidia/Llama-Nemotron-Post-Training-Dataset`; `HEx-PHI` β the 300 HEx-PHI harmful instructions (`jkazdan/guardrail-llama-3-8b-refusal-hexphi`). Both are safety-relevant / jailbreak prompts. | |
| Both datasets only contain a train subset and given their near-perfect scores, it is hard to rule out possible contimination in Qwen's original training data. These results are intended purely as a demonstration of preserved guardrail capabilities, and are in line with our internal usage testing. | |
| - **Metrics:** **SAFE %** β fraction of replies an LLM safety judge marks SAFE (upholds guardrails β refuses or safely deflects the harmful request); higher is better. **Thinking tokens** and **Reduction** are as defined for the token-efficiency table above. | |
| ## Usage | |
| ### HuggingFace Transformers | |
| ```python | |
| from transformers import AutoModelForImageTextToText, AutoProcessor | |
| model = AutoModelForImageTextToText.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B", dtype="bfloat16") | |
| proc = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B") | |
| ``` | |
| Check https://huggingface.co/Qwen/Qwen3.6-27B for recommended usage, sampling params etc. | |
| ### FP8 (vLLM / SGLang) | |
| An official FP8 quantization for GPU serving lives in the sibling repo [bottlecapai/ThinkingCap-Qwen3.6-27B-FP8](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8) β half the memory of bf16 at near-lossless quality, in the compressed-tensors format vLLM/SGLang load natively, with the MTP (multi-token-prediction) speculative-decoding head kept in bf16. | |
| ```bash | |
| vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 | |
| ``` | |
| ### GGUF (llama.cpp) | |
| Quantized [GGUF](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md) builds of this model live in the sibling repo [bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF), for local inference with [llama.cpp](https://github.com/ggml-org/llama.cpp) and compatible runtimes (Ollama, LM Studio, β¦). | |
| Quantization stores the weights at reduced precision β e.g. ~4.7 bits per weight for `Q4_K_M` instead of 16-bit bf16 β cutting download size and memory severalfold at a small quality cost. `Q4_K_M` is the recommended size/quality balance, `Q8_0` is near-lossless. | |
| ```bash | |
| llama-cli -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M -p "Hi" | |
| ``` | |
| ## Where to find us | |
| <table style="border-collapse:collapse;border:0;margin:0"><tbody><tr> | |
| <td style="border:0;padding:0 18px 0 0"><a href="https://www.bottlecapai.com/"><img src="social-web.png" alt="Website" width="34" height="34"></a></td> | |
| <td style="border:0;padding:0 18px 0 0"><a href="https://www.linkedin.com/company/bottlecap-ai/"><img src="social-linkedin.png" alt="LinkedIn" width="34" height="34"></a></td> | |
| <td style="border:0;padding:0 18px 0 0"><a href="https://www.instagram.com/bottlecapai/"><img src="social-instagram.png" alt="Instagram" width="34" height="34"></a></td> | |
| <td style="border:0;padding:0"><a href="https://x.com/BottleCapAI"><img src="social-x.png" alt="X" width="34" height="34"></a></td> | |
| </tr></tbody></table> | |
| ## Citation | |
| If you use this model, please cite: | |
| ```bibtex | |
| @misc{ThinkingCap-Qwen3.6-27B, | |
| title = {bottlecapai/ThinkingCap-Qwen3.6-27B}, | |
| author = {Lasocki, Karol and Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Mikolov, Tomas}, | |
| year = {2026}, | |
| } | |
| ``` | |
| ## Acknowledgements | |
| We acknowledge EuroHPC Joint Undertaking for awarding the project ID EHPC-AIF-2025SC03-029 access to Leonardo at CINECA, Italy. | |