---
license: apache-2.0
library_name: gguf
base_model: Qwen/Qwen3.6-35B-A3B
base_model_relation: quantized
model_name: Qwen3.6-35B-A3B-Cerebellum-GGUF
model_creator: Qwen
model_type: qwen3
quantized_by: deucebucket
pipeline_tag: text-generation
tags:
  - GGUF
  - qwen3
  - qwen
  - quantized
  - cerebellum
  - imatrix
  - moe
  - mixed-precision
  - 3-bit
  - conversational
model-index:
- name: Qwen3.6-35B-A3B-Cerebellum-GGUF
  results:
  - task:
      name: Text Generation
      type: text-generation
    dataset:
      name: AI2 Reasoning Challenge
      type: ai2_arc
      config: ARC-Challenge
      split: test
    metrics:
    - name: normalized accuracy
      type: acc_norm
      value: 0.958
    source:
      name: Local benchmark run (RTX 3090, llama.cpp)
      url: https://huggingface.co/deucebucket/Qwen3.6-35B-A3B-Cerebellum-GGUF/tree/main/benchmark_results
  - task:
      name: Text Generation
      type: text-generation
    dataset:
      name: HellaSwag
      type: hellaswag
      split: validation
    metrics:
    - name: accuracy
      type: acc
      value: 0.923
    source:
      name: Local benchmark run (RTX 3090, llama.cpp)
      url: https://huggingface.co/deucebucket/Qwen3.6-35B-A3B-Cerebellum-GGUF/tree/main/benchmark_results
  - task:
      name: Text Generation
      type: text-generation
    dataset:
      name: MMLU-Redux
      type: cais/mmlu
      config: all
      split: test
    metrics:
    - name: accuracy
      type: acc
      value: 0.75
    source:
      name: Local benchmark run (RTX 3090, llama.cpp)
      url: https://huggingface.co/deucebucket/Qwen3.6-35B-A3B-Cerebellum-GGUF/tree/main/benchmark_results
  - task:
      name: Text Generation
      type: text-generation
    dataset:
      name: HumanEval+ (pass@1)
      type: openai_humaneval
      split: test
    metrics:
    - name: pass@1
      type: pass@1
      value: 0.902
    source:
      name: Local benchmark run (RTX 3090, llama.cpp)
      url: https://huggingface.co/deucebucket/Qwen3.6-35B-A3B-Cerebellum-GGUF/tree/main/benchmark_results
---

<p align="center">
  <img src="cerebellum_banner.png" alt="Cerebellum" width="640">
</p>

# Qwen 3.6 35B-A3B — Cerebellum GGUF

Sensitivity-guided mixed-precision quantization of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B). Cerebellum measures which weight groups survive extreme compression and which don't, then writes a single GGUF with per-tensor precision assignments — a standard GGUF that runs on stock `llama.cpp`, no fork.

| Variant | File | Size | BPW | Best for |
|---------|------|------|-----|----------|
| **14 GB** (recommended) | `Qwen3.6-35B-A3B-Cerebellum-14GB.gguf` | **14.0 GB** | 3.34 | best coding, 160K+ context |
| v3 (smallest) | `Qwen3.6-35B-A3B-Cerebellum-v3-Q3_K_M.gguf` | 11 GB | 2.76 | tightest VRAM, vision |

## Evaluations

Coding — upstream EvalPlus (`evalplus.codegen` against `llama-server`, greedy / temp 0, n=164), same protocol across the size ladder:

| build | size | HumanEval | HumanEval+ |
|-------|------|:---:|:---:|
| micro | 11.96 GB | 90.9 | 87.2 |
| **14 GB (recommended)** | **14.0 GB** | **93.3** | **90.2** |
| uniform Q3_K_M | 16.0 GB | 91.5 | 89.0 |
| Base | 17.3 GB | 92.7 | 89.0 |

Long-context: needle recall passes to 90K+ (`verify-stress`). Throughput: ~168 tok/s decode (3B-active MoE); fits 160K+ context at ~19 GB on a 24 GB card. Per-question artifacts in `benchmark_results/14gb/`.

## Why the 14 GB over v3

v3 (11 GB) is the tightest-VRAM build. The 14 GB spends ~3 GB more to promote the routed `ffn_down_exps` to Q4_K — the group the ablation identifies as where the quality lives — and that gives it the family's **best coding** plus 160K+ context headroom. It posts above the 16 GB uniform Q3_K_M (−2 GB) and matches the 17.3 GB Base (−3.3 GB): the Base's extra promotions buy ~0 coding, so 14 GB is the efficient point. Pick v3 only when VRAM is tight or you need the vision projector.

## Usage

```bash
# 14 GB (recommended)
llama-server -m Qwen3.6-35B-A3B-Cerebellum-14GB.gguf -ngl 99 -fa on --reasoning off

# v3 (smallest, with vision)
llama-server -m Qwen3.6-35B-A3B-Cerebellum-v3-Q3_K_M.gguf --mmproj mmproj-F16.gguf -ngl 99 -c 8192
```

## Files

| File | Size | Notes |
|------|------|-------|
| `Qwen3.6-35B-A3B-Cerebellum-14GB.gguf` | 14 GB | recommended — best coding, 160K+ ctx |
| `Qwen3.6-35B-A3B-Cerebellum-v3-Q3_K_M.gguf` | 11 GB | smallest; vision (with mmproj) |
| `mmproj-F16.gguf` | 858 MB | vision projector (F16) |
| `benchmark_results/` | — | per-question evaluation artifacts |
| `ablation/` | — | ablation logs + tensor override maps |

## Methodology

Built with [Cerebellum](https://github.com/deucebucket/cerebellum) — sensitivity-guided mixed-precision quantization: crush each tensor group, measure the impact, allocate precision under a size budget, output a plain GGUF. imatrix-calibrated. Quantized by [@deucebucket](https://huggingface.co/deucebucket).

## Independent records

This line has a recorded data point in [club-3090's BENCHMARKS](https://github.com/noonghunna/club-3090/blob/master/BENCHMARKS.md) (author-rig numbers from a full `report.sh --full` chain). The same report corrected their engine-support table for this model ([issue #390](https://github.com/noonghunna/club-3090/issues/390), [PR #393](https://github.com/noonghunna/club-3090/pull/393)). Numbers there are author-reported, not club-validated.