knoopx
/

ThinkingCap-Qwen3.6-27B-NInfer

@@ -4,8 +4,6 @@ language:
 - en
 pipeline_tag: text-generation
 library_name: ninfer
-base_model:
-- Qwen/Qwen3.6-27B
 tags:
 - qwen3.6
 - qwen3_5
@@ -24,150 +22,92 @@ tags:
 - llm
 - ninfer
 - single-file
 ---
 # ThinkingCap-Qwen3.6-27B
-A single-file [NInfer](https://github.com/ninfer) `.ninfer` artifact set for **ThinkingCap-Qwen3.6-27B** — a token-efficient / efficient-thinking multimodal (image+video→text) 27B finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B). This card documents two derived NInfer artifacts, each a self-contained single binary that the NInfer engine serves directly:
-- **`thinkingcap_qwen3_6_27b_nvfp4.ninfer`** — NVFP4 (W4A4 mixed-precision) artifact, 16.55 GiB
-- **`thinkingcap_qwen3_6_27b.ninfer`** — groupwise-int4 (on-the-fly Q4/Q5/Q6/W8) artifact, 16.07 GiB
-Both are byte-level containers (`raw-bytes-v1` encoding) produced by the NInfer converter toolchain, not Hugging Face `pytorch_model.bin` / `safetensors` weights. They are the serving artifacts for the model on the NInfer runtime.
 ## Model architecture
-- **Architecture:** `Qwen3_5ForConditionalGeneration` (`model_type: qwen3_5`), a 27B dense multimodal (image+video→text) finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B). It is a fully dense feed-forward transformer: every decoder layer uses a dense SwiGLU MLP with no routing.
-- **Text:** 64 decoder layers on a hybrid attention schedule — 16 full-attention layers interleaved with 48 recurrent layers (short convolution, width 4) on a full-attention interval of 4; hidden size 5120, intermediate 17408, output matrix 248,320 rows (tokenizer domain 248,077), 24 query heads / 4 KV heads (head dim 256), 1 MTP draft layer, 262,144 max position embeddings
-- **Vision:** 27-layer ViT (hidden 1152, intermediate 4304, 16 heads), 16×16 spatial patch, 2-frame temporal patch, 2×2 spatial merge
-- **Positional:** RoPE θ = 10,000,000 with mRoPE section `[11, 11, 10]`
-- **Upstream base:** [`bottlecapai/ThinkingCap-Qwen3.6-27B`](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B) (itself a finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B))
 ## Artifacts
-### 1. NVFP4 — `thinkingcap_qwen3_6_27b_nvfp4.ninfer`
-**Quantization scheme:** NVFP4 **W4A4 mixed-precision**. The compressed-tensors NVFP4 recipe (`nvfp4-pack-quantized`) stores packed 4-bit FP4 (E2M1) weights together with per-tensor group scales, with `input_activations: tensor_group` so activations are also 4-bit-quantized at inference. This is a true **W4A4** path — unlike a **W4A16** build, where only the weights are 4-bit (Q4) but activations and GEMM inputs stay at 16-bit, keeping the matmul precision at 16-bit. W4A4 trades some quality headroom for lower memory and (on FP4-capable hardware) lower compute, and is the configuration that mirrors the upstream vLLM-quantized NVFP4 checkpoints.
-**Dual-source conversion.** This artifact is built from **two fixed source roles** in one pass:
-1. **Base BF16** checkpoint — the full-precision source (`bottlecapai/ThinkingCap-Qwen3.6-27B`, rev `6a9e13bd`), preflight of 1,118 recipes / 1,199 BF16 tensors across 3 shards.
-2. **vLLM-quantized NVFP4** checkpoint — the NVFP4 source (`sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4`, rev `9b5389d4`), preflight of 379 `linear_nvfp4` + 117 `linear_bf16` lines with dtypes U8×379 / F8_E4M3×379 / F32×758.
-The converter fuses the two: NVFP4-packed lines take their 4-bit weight bytes + scales from the vLLM NVFP4 source, while the remaining 16-bit and resource tensors are sourced from the base BF16 tree. The result is a single NInfer container carrying both the NVFP4 linear layers and the BF16/FP32 remainder.
-**Converter module + recipe:** `tools.convert.qwen3_6_27b.convert_nvfp4`, `RECIPE_ID = qwen3_6_27b_nvfp4-v1` (converter revision `19e80bb`).
-**Conversion command:**
 ```bash
 python3 -m tools.convert.qwen3_6_27b.convert_nvfp4 \
-  --model <bf16-source-dir> \
-  --nvfp4-model <nvfp4-source-dir> \
-  --out out/thinkingcap_qwen3_6_27b_nvfp4.ninfer \
-  --device cpu
 ```
-CLI arguments (from the module's argparse): `--model` (base BF16 dir, required), `--nvfp4-model` (NVFP4 dir, required), `--out` (required), `--device` (default `cuda`; the conversion run used `cpu`), plus report-only `--out-basename`, `--base-repo`, and `--nvfp4-repo`.
-**Artifact stats** (from `thinkingcap_qwen3_6_27b_nvfp4.ninfer.conversion.json`):
-| field | value |
-|---|---|
-| identity | `qwen3.6-27b` / weights_id `nvfp4` |
-| target_key | `qwen3_6_27b` |
-| recipe_id | `qwen3_6_27b_nvfp4-v1` |
-| objects | 1307 (1301 tensors + 6 resources) |
-| artifact bytes | 18,324,064,000 (16.55 GiB) |
-| tensor_bytes | 18,310,883,160 |
-| resource_bytes | 12,833,441 |
-| alignment_bytes | 138,503 |
-| md5 | `24a3d90b8aae9ea4f5296061439bd625` |
-| elapsed | 69.54 s |
-**Format histogram** (per-tensor format counts, summing to 1301 tensors):
-| format | count |
-|---|---|
-| BF16 | 591 |
-| FP32 | 343 |
-| NVFP4 | 247 |
-| Q4G64_F16S | 55 |
-| Q5G64_F16S | 54 |
-| W8G32_F16S | 9 |
-| Q6G64_F16S | 1 |
-| I32 | 1 |
-| **total tensors** | **1301** |
-Plus **6 resources** (`raw-bytes-v1` encoding). Layouts: `blockscale-k16-m128x4-v1` ×247 (the NVFP4 blockscale line), `contiguous-le-v1` ×935, `row-split-k128-v1` ×119.
-**NVFP4 source:** [`sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4`](https://huggingface.co/sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4) (the `--nvfp4-model` input to the conversion; rev `9b5389d4`)
----
-### 2. Groupwise-int4 — `thinkingcap_qwen3_6_27b.ninfer`
-**Quantization scheme:** **on-the-fly groupwise quantization from a single BF16 source**. Rather than consuming a pre-quantized checkpoint, the converter reads the base BF16 checkpoint and quantizes each linear layer at conversion time into a per-group 4/5/6-bit + W8 format selected by tensor size/role — `Q4G64_F16S` (4-bit, group-64), `Q5G64_F16S` (5-bit, group-64), `Q6G64_F16S` (6-bit, group-64), and `W8G32_F16S` (8-bit, group-32). Larger/more-sensitive linears get higher-bit groupwise formats; smaller ones get 4-bit, trading memory for quality. Everything else stays BF16/FP32. This is a **W4A16-class** build: weights are groupwise-quantized but activations remain 16-bit, in contrast to the W4A4 NVFP4 artifact above.
-**Converter module + recipe:** `tools.convert.qwen3_6_27b.convert`, `RECIPE_ID = qwen3_6_27b-v2` (converter revision `843284b`).
-**Conversion command:**
 ```bash
-python -m tools.convert.qwen3_6_27b.convert \
-  --model <bf16-source-dir> \
-  --out out/thinkingcap_qwen3_6_27b.ninfer \
-  --device cuda
 ```
-CLI arguments (from the module's argparse): `--model` (BF16 dir, required), `--out` (required), `--device` (default `cuda`; the conversion run resolved to `cpu`).
-**Artifact stats** (from `thinkingcap_qwen3_6_27b.ninfer.conversion.json`):
-| field | value |
-|---|---|
-| identity | `qwen3.6-27b` / weights_id `groupwise-int` |
-| target_key | `qwen3_6_27b` |
-| recipe_id | `qwen3_6_27b-v2` |
-| objects | 1124 (1118 tensors + 6 resources) |
-| artifact bytes | 17,495,365,888 (16.07 GiB) |
-| tensor_bytes | 17,482,342,304 |
-| resource_bytes | 12,833,441 |
-| alignment_bytes | 14,015 |
-| md5 | `8c08f2f50ca93cab2e13508365d65147` |
-| elapsed | 188.55 s |
-**Format histogram** (per-tensor format counts, summing to 1118 tensors):
-| format | count |
-|---|---|
-| BF16 | 582 |
-| Q5G64_F16S | 246 |
-| Q4G64_F16S | 183 |
-| FP32 | 96 |
-| W8G32_F16S | 7 |
-| Q6G64_F16S | 3 |
-| I32 | 1 |
-| **total tensors** | **1118** |
-Plus **6 resources** (`raw-bytes-v1` encoding). Layouts: `contiguous-le-v1` ×679, `row-split-k128-v1` ×439.
-**Source:** [`bottlecapai/ThinkingCap-Qwen3.6-27B`](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B) (BF16 base, rev `6a9e13bd`; preflight 1,118 recipes / 1,199 BF16 tensors / 3 shards).
 ## Serving
-Both artifacts are served by the **NInfer** engine:
-```bash
---model-id thinkingcap-27b
-```
-## Known issues
-- **No KL-divergence or perplexity benchmark has been run on either artifact.** Quality of the W4A4 NVFP4 and the groupwise-int4 builds has **not** been measured against the BF16 reference on a held-out corpus; the only validation performed is structural (format histogram, object/tensor/resource counts, and byte/alignment integrity from the converter's own accounting). Do not treat these artifacts as numerically validated for downstream quality claims.
 ## References
-- Base checkpoint: [`bottlecapai/ThinkingCap-Qwen3.6-27B`](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B)
-- NVFP4 source: [`sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4`](https://huggingface.co/sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4) (the `--nvfp4-model` input to the conversion; rev `9b5389d4`)
-- Converter toolchain: `tools.convert.qwen3_6_27b` (modules `convert` / `convert_nvfp4`), NInfer source tree.

 - en
 pipeline_tag: text-generation
 library_name: ninfer
 tags:
 - qwen3.6
 - qwen3_5
 - llm
 - ninfer
 - single-file
+base_model:
+- Qwen/Qwen3.6-27B
 ---
 # ThinkingCap-Qwen3.6-27B
+Three single-file NInfer (`*.ninfer`) artifacts for **ThinkingCap-Qwen3.6-27B**, a token-efficient, thinking-oriented, multimodal (image+video→text) 27B finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B). Each artifact is a self-contained binary the NInfer engine serves directly. All three trace back to the same finetune, but each is produced by a distinct converter module and carries a distinct recipe and weight profile — so none is a re-label of another.
 ## Model architecture
+- `Qwen3_5ForConditionalGeneration` (`model_type: qwen3_5`), dense multimodal (image+video→text) finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B). Fully dense transformer: every decoder layer uses a dense SwiGLU MLP with no routing.
+- **Text:** 64 decoder layers; hybrid attention — 16 full-attention layers interleaved with 48 recurrent layers (short convolution, width 4), full-attention interval 4; hidden 5120, intermediate 17408, output matrix 248320 rows (tokenizer domain 248077); 24 query / 4 KV heads (head dim 256); 1 MTP draft layer; 262144 max position embeddings.
+- **Vision:** 27-layer ViT (hidden 1152, intermediate 4304, 16 heads), 16×16 spatial patch, 2-frame temporal patch, 2×2 spatial merge.
+- **Positional:** RoPE θ=10,000,000 with mRoPE section `[11, 11, 10]`.
+- Upstream base: [`bottlecapai/ThinkingCap-Qwen3.6-27B`](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B), itself a finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B).
 ## Artifacts
+### 1. Groupwise-int4 (W4A16) — `thinkingcap_qwen3_6_27b.ninfer`
+On-the-fly groupwise quantization from a single BF16 source; each linear layer is quantized at conversion time into a per-group 4/5/6-bit + W8 format selected by tensor size/role (`Q4G64_F16S`, `Q5G64_F16S`, `Q6G64_F16S`, `W8G32_F16S`). Weights are groupwise-quantized while activations stay 16-bit (W4A16-class build).
+Single-source provenance: base BF16 checkpoint `bottlecapai/ThinkingCap-Qwen3.6-27B` (rev `6a9e13bd`). Converter module `tools.convert.qwen3_6_27b.convert` (recipe `qwen3_6_27b-v2`, weights_id `groupwise-int`):
+```bash
+python -m tools.convert.qwen3_6_27b.convert \
+  --model <base-bf16-checkpoint-dir> --out out/<name>.ninfer --device cuda
+```
+Stats: objects 1124 (1118 tensors + 6 resources); tensor_bytes 17482342304; resource_bytes 12833441; alignment_bytes 14015; md5 `8c08f2f50ca93cab2e13508365d65147`.
+Histogram (sums to 1118): `BF16 582, Q5G64_F16S 246, Q4G64_F16S 183, FP32 96, W8G32_F16S 7, Q6G64_F16S 3, I32 1`. Layouts: `contiguous-le-v1 ×679, row-split-k128-v1 ×439`; 6 resources encoded `raw-bytes-v1`.
+### 2. NVFP4 (W4A4) — `thinkingcap_qwen3_6_27b_nvfp4.ninfer`
+NVFP4 W4A4 mixed-precision: packed 4-bit FP4 (E2M1) weights with per-tensor group scales and `input_activations: tensor_group`, so activations are also 4-bit at inference. True W4A4 path (vs W4A16, where activations stay 16-bit).
+Dual-source: base BF16 `bottlecapai/ThinkingCap-Qwen3.6-27B` (rev `6a9e13bd`) fused with vLLM-quantized NVFP4 partner [`sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4`](https://huggingface.co/sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4) (rev `9b5389d4`). NFp4-packed lines take 4-bit weight bytes + scales from the partner; remaining 16-bit/resource tensors come from the base BF16 tree. Converter `tools.convert.qwen3_6_27b.convert_nvfp4` (recipe `qwen3_6_27b_nvfp4-v1`, weights_id `nvfp4`, model_id `qwen3.6-27b`):
 ```bash
 python3 -m tools.convert.qwen3_6_27b.convert_nvfp4 \
+  --model <base-bf16-checkpoint-dir> \
+  --nvfp4-model <nvfp4-checkpoint-dir> \
+  --out out/<name>.ninfer --device cpu
 ```
+Stats: objects 1307 (1301 tensors + 6 resources); tensor_bytes 18310883160; resource_bytes 12833441; alignment_bytes 138503; md5 `24a3d90b8aae9ea4f5296061439bd625`.
+Histogram (sums to 1301): `BF16 591, FP32 343, NVFP4 247, Q4G64_F16S 55, Q5G64_F16S 54, W8G32_F16S 9, Q6G64_F16S 1, I32 1`. Layouts: `blockscale-k16-m128x4-v1 ×247, contiguous-le-v1 ×935, row-split-k128-v1 ×119`; 6 resources `raw-bytes-v1`.
+### 3. NVFP4 (W4A4) w8g32 — `thinkingcap_qwen3_6_27b_nvfp4_w8g32.ninfer`
+Shares the same W4A4 FP4 recipe family as Artifact 2 — packs the same inventory including one `Q6G64_F16S` object — but is produced by a distinct converter generation (weights_id `nvfp4_w8g32`) and carries a larger payload (+3840 bytes vs Artifact 2). Separate build, not a rename of Artifact 2. Named on the same compact scheme as the other two artifacts, distinguished by the `w8g32` weight-generation suffix.
+Dual-source: same partners as Artifact 2. Converter `tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4` (weights_id `nvfp4_w8g32`):
 ```bash
+python -m tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4 \
+  --model <base-bf16-checkpoint-dir> \
+  --nvfp4-model <nvfp4-checkpoint-dir> \
+  --out out/<name>.ninfer --device cuda
 ```
+Stats: objects 1307 (1301 tensors + 6 resources); byte total 18324067840; layouts `blockscale-k16-m128x4-v1 ×247, contiguous-le-v1 ×935, row-split-k128-v1 ×119`; raw-bytes-v1 encodings. Identity embedded in the artifact: model_id `qwen3.8-27b`, weights_id `nvfp4_w8g32`.
+## Files and checksums
+Byte sizes below are authoritative LFS sizes.
+| File | Size (bytes) | SHA256 | Scheme |
+|---|---|---|---|
+| `thinkingcap_qwen3_6_27b.ninfer` | 17495365888 | `406d7ca3c8248ec304518563b9dbb00cf48d9cfc19d126e6df66d354e6d5b892` | Groupwise-int4 W4A16 |
+| `thinkingcap_qwen3_6_27b_nvfp4.ninfer` | 18324064000 | `ec149b15d29acc0bfdb6f5720e12a39f15746b688ac4e290a6ebf6661ac2c8e7` | NVFP4 W4A4 |
+| `thinkingcap_qwen3_6_27b_nvfp4_w8g32.ninfer` | 18324067840 | `00c3412b764c24bdf37058dea5209b677462fa68c517c8e21c89aec9f38dcbd1` | NVFP4 W4A4 (w8g32) |
 ## Serving
+All artifacts are served directly by the **NInfer** engine as self-contained single binaries — no separate weights download. Capability flags relevant to this model: multimodal image input (`--vision`), MTP decoding (1 MTP draft layer), and int8 KV cache.
+## Known limitations
+No KL-divergence or perplexity benchmark has been run on any artifact; quantization fidelity is unmeasured against the BF16 reference. Validation performed is structural only — format histogram, object/tensor/resource counts, and byte/alignment integrity from the converter's accounting. The artifacts are not numerically validated for downstream quality claims.
 ## References
+- Base checkpoint: [`bottlecapai/ThinkingCap-Qwen3.6-27B`](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.6-27B) (finetune of [`Qwen/Qwen3.6-27B`](https://huggingface.co/Qwen/Qwen3.6-27B))
+- NVFP4 partner: [`sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4`](https://huggingface.co/sakamakismile/ThinkingCap-Qwen3.6-27B-NVFP4)
+- NInfer engine: https://github.com/ninfer
+- Converter toolchain: `tools.convert.qwen3_6_27b.{convert,convert_nvfp4}` and `tools.convert.qwen3_8_27b.w8g32_nvfp4.convert_nvfp4`