Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,74 @@
+---
+license: apache-2.0
+base_model: bottlecapai/ThinkingCap-Qwen3.6-27B
+base_model_relation: quantized
+pipeline_tag: text-generation
+tags:
+  - awq
+  - int4
+  - w4a16
+  - compressed-tensors
+  - llm-compressor
+language:
+  - en
+---
+
+# ThinkingCap-Qwen3.6-27B-AWQ
+
+AWQ (W4A16) quantization of
+[bottlecapai/ThinkingCap-Qwen3.6-27B](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B)
+— a token-efficient **reasoning** finetune of [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) by BottleCap AI (`qwen3_5`: dense hybrid GatedDeltaNet linear-attention + full-attention over 64 text layers in a 3:1 pattern, plus a vision tower and an MTP head; multimodal image-text-to-text). It matches Qwen3.6-27B answer quality while emitting ~50% fewer thinking tokens on average.
+
+**Variant**: AWQ **W4A16** — 4-bit symmetric integer weights, group size 128, with activation-aware scaling. Activations stay BF16.
+**Quantized by**: [sahilchachra](https://huggingface.co/sahilchachra)
+**Tooling**: `llm-compressor` (`AWQModifier` + `QuantizationModifier`) -> `compressed-tensors` `pack-quantized`
+
+> This is a quantized derivative. Weights, behavior, and license follow the base
+> model — see the
+> [original card](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B) for full details, benchmarks, and citation.
+
+## What is quantized
+
+Quantized to 4-bit:
+
+- full-attention `self_attn.{q,k,v,o}_proj`
+- `mlp.{gate,up,down}_proj` (all text layers)
+
+Kept in **BF16**: GatedDeltaNet `linear_attn` (mamba) layers, vision tower (`model.visual.*`, 27 blocks), MTP head, token embeddings, lm_head, all norms (incl. q_norm / k_norm).
+
+## Calibration
+
+AWQ: 128 sequences x 512 tokens of **GSM8K** (`openai/gsm8k`, config `main`, train split) — the model's headline in-domain reasoning dataset — rendered through the model's own chat template with the `<think>…</think>` reasoning format, so calibration matches the model's real (thinking) inference distribution. NVFP4 is data-free (no calibration).
+
+## Prompt template & sampling
+
+This is a **reasoning ("thinking") model**. Use the Qwen3.6 chat template — ChatML (`<|im_start|>role … <|im_end|>`) with a `<think>…</think>` reasoning trace, thinking enabled by default. Apply it via `tokenizer.apply_chat_template(messages, add_generation_prompt=True)` (or the processor for image inputs); do not hand-format prompts. See the base [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) for full usage details.
+
+**Recommended sampling**: `temperature=1.0, top_p=0.95, top_k=20, min_p=0.0` with thinking on (the base model's recommended sampling, per the original card).
+
+
+## Usage (vLLM)
+
+```python
+from vllm import LLM, SamplingParams
+
+# This is a multimodal checkpoint: the vision tower is kept in BF16
+# (only the text / MoE weights are 4-bit). vLLM builds the full model.
+llm = LLM(
+    model="sahilchachra/ThinkingCap-Qwen3.6-27B-AWQ",
+    trust_remote_code=True,
+)
+out = llm.chat(
+    [{"role": "user", "content": "Hello!"}],
+    SamplingParams(temperature=0.6, top_p=0.95, max_tokens=512),
+)
+print(out[0].outputs[0].text)
+```
+
+Serving via the CLI, pass the flag directly:
+
+```bash
+vllm serve sahilchachra/ThinkingCap-Qwen3.6-27B-AWQ \
+    --trust-remote-code \
+    --max-model-len 262144 --reasoning-parser qwen3
+```