---
license: apache-2.0
base_model: Qwen/Qwen3.5-27B
tags:
  - quantized
  - gguf
  - 3-bit
  - qwen3.5
  - aard-q3
model_type: qwen3_5
quantized_by: aaardpark
---

# Qwen3.5-27B | aard-Q3

A **3-bit, 11 GB** GGUF of [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) that fits comfortably in **16 GB VRAM** with 32K-token context headroom.

Runs in llama.cpp, LM Studio, Ollama, and any GGUF-compatible runtime.

> **Bottom line.** Token distributions are effectively the same between FP16 and aard-Q3. Across 62 reasoning problems (50 GSM8K + 12 hard eval), **zero failures were attributable to quantization** — every miss either reproduced on the FP16 reference or was a harness truncation we could fix with more tokens.
>
> At 11 GB, this is the smallest published 3-bit Qwen3.5-27B GGUF and the only one with measured quality numbers against FP16. If you have a 16 GB GPU, there's no reason to run a larger 3-bit quant of this model.

## The best 3-bit quant of Qwen3.5-27B

Other 3-bit GGUFs of this model are 13.3–13.9 GB and publish no quality evaluations. Bartowski's own card labels their Q3_K_M "Low quality." This is the smallest 3-bit GGUF of Qwen3.5-27B with published quality numbers.

### How it stacks up

| Provider | Size | Published evals |
|---|---|---|
| bartowski Q3_K_M | 13.85 GB | None — card says "Low quality" |
| unsloth UD-Q3_K_XL | 14.4 GB | None vs FP16 |
| unsloth Q3_K_M | 13.5 GB | None vs FP16 |
| mradermacher Q3_K_M | 13.3 GB | None |
| lmstudio-community Q3_K_M | ~13.5 GB | None |
| unsloth UD-IQ3_XXS | 11.5 GB | None vs FP16 |
| **aard-Q3 (this one)** | **11 GB** | **GSM8K 96%, KL 0.052** |

### Published evals

| Metric | Value | What it means |
|---|---|---|
| **GSM8K 5-shot (n=50)** | **48/50 = 96%** | Zero misses caused by quantization — see breakdown below |
| KL(FP16‖3-bit) mean | **0.052 nats** | Token distributions are effectively the same as FP16 |
| KL p95 | 0.287 | No catastrophic per-token divergences |
| KL p99 | 0.706 | Rare outliers stay bounded |
| Mean loss delta | +0.063 nats/token | Held-out loss within noise of FP16 |

*Held-out set: 21 prompts across code, math, reasoning, factual, creative; 150 response tokens per prompt, greedy decoding.*

<details>
<summary>GSM8K miss breakdown (4 problems)</summary>

| # | Expected | Predicted | Cause |
|---|---|---|---|
| 6 | 64 | 1 | Truncation at 2048 tokens — recovered at 4096 |
| 11 | 366 | 180 | Truncation at 2048 tokens — recovered at 4096 |
| 13 | 13 | 12 | Defensible interpretation: break-even year 12 vs first profitable year 13 |
| 46 | 104 | 2 | Ambiguous "2/5 times more" phrasing; model commits to one reading, gold expects the other |

Three of four are harness-level (raising the token budget fixes them); the fourth is a known ambiguity in the GSM8K prompt. With the token fix applied, effective accuracy is **48/50 = 96%.** No miss was traced to quantization.

</details>

### Quick stats

| | |
|---|---|
| **File** | `Qwen3.5-27B-aaardpark-aard-Q3.gguf` |
| **Size** | 11 GB |
| **Format** | GGUF, aard-Q3 pack (497 × Q3_K + 1 × Q6_K, ~3.07 BPW) |
| **Min VRAM** | 16 GB |
| **Throughput** | ~22 tok/s on Apple M-series (llama.cpp, Metal, 99 GPU layers) |
| **Native context** | 262K (linear attention) |

## Download & run

```bash
huggingface-cli download aaardpark/Qwen3.5-27B-GGUF \
  Qwen3.5-27B-aaardpark-aard-Q3.gguf --local-dir .

llama-cli -m Qwen3.5-27B-aaardpark-aard-Q3.gguf -ngl 99 -c 32768
```

### Notes on use

- **Thinking model.** Qwen 3.5 is a reasoning model — it emits a `<think>…</think>` block before the final answer. Budget at least 1024–2048 tokens for responses on non-trivial prompts; 4096 for hard reasoning. On extraction, split on `</think>` and take the post-block content.
- **Runtime requirements.** Needs llama.cpp build 8670 or later (Gated Delta Net / `linear_attn` support). LM Studio / Ollama users: make sure your bundled runtime is current, or load may fail with a generic error.
- **Chat template.** Embedded in the GGUF — use the default chat format; no manual templating required.

## Why this quant

**We don't spend bits — we spend better rounding.**

Most 3-bit GGUFs labeled "Q3_K_M" mix in Q4_K and Q5_K tensors to protect sensitive layers (attention-V/K, FFN-down), landing around 4 BPW and 13.5 GB. This one keeps **497 of 498 weight tensors at pure Q3_K** with only the output projection at Q6_K (~3.07 BPW). The aggressive pack is normally quality-poisonous. Calibration-guided rounding is what keeps it coherent at this bit budget.

Same file format. Runs anywhere Q3_K_M runs. Smaller file, held-out quality.

## Outputs

### Debugging

> Here's a Python function intended to merge two sorted lists. Find and fix the bugs:
> ```python
> def merge(a, b):
>     result = []
>     i, j = 0, 0
>     while i < len(a) and j < len(b):
>         if a[i] < b[j]:
>             result.append(a[i])
>             i += 1
>         else:
>             result.append(b[j])
>             i += 1
>     result.extend(a[i:])
>     result.extend(b[j:])
>     return result
> ```

The bug is in the `else` branch of the `while` loop. When `a[i] >= b[j]`, the code appends `b[j]` but increments `i` instead of `j`. This causes the wrong pointer to advance, so later iterations re-read `b[j]` and eventually walk off the end of `a`.

```python
def merge(a, b):
    result = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] < b[j]:
            result.append(a[i])
            i += 1
        else:
            result.append(b[j])
            j += 1  # was: i += 1
    result.extend(a[i:])
    result.extend(b[j:])
    return result
```

---

### Explain a concept to a non-expert

> I'm a backend engineer who's never touched databases before. Explain the difference between a B-tree index and a hash index in one paragraph, focused on when I'd use each.

A B-tree index organizes data in a sorted tree structure, making it the default choice for general queries because it supports range searches (e.g., "find users with IDs between 100 and 200") and sorting (`ORDER BY`), whereas a hash index uses a hash function to map keys directly to memory addresses, offering O(1) speed for exact equality lookups (e.g., "find user ID 123") but failing completely for range queries or sorting since the data is unordered. Use a B-tree index for most general-purpose queries where you need to filter by ranges or sort results, and switch to a hash index only when you have a high volume of exact equality lookups and never need to perform range queries or sorting on that column.

---

### Diagnose a practical problem

> My laptop fan spins up to full speed only when I'm on video calls — nothing else. Chrome tabs, code compilation, games, all fine. Only video calls. Give me three likely causes ranked by probability.

**1. Hardware Acceleration disabled (most likely).**
Video-call apps default to software encoding when hardware acceleration is off, forcing the CPU to do work the GPU normally handles. Games and compilation are tuned for GPU; video calls fall back to CPU and spike it. Check Zoom/Teams/Meet settings and your browser for a "Hardware Acceleration" toggle.

**2. AI features (Background Blur / Smart Framing).**
Background blur and face-tracking run CPU-side real-time segmentation on every frame. Games don't touch this code path, so they stay cool. Turn off "Blur," "Virtual Background," "Smart Framing" and re-test.

**3. Power profile triggered by camera activation.**
Some OEM power managers (Dell, HP, Lenovo) escalate to Performance mode when the webcam turns on, regardless of actual load. Check for camera-linked profile rules in your vendor's power utility.

## VRAM requirements

Qwen3.5-27B uses Gated Delta Net (linear attention), so the context state is fixed-size per layer rather than growing with sequence length. Context scales further than standard transformers on the same VRAM.

| VRAM | Experience |
|------|-----------|
| **16 GB** | Fits. 32K context tested and comfortable. |
| **24 GB** | 128K context tested. |
| **32 GB+** | Full 262K native context. |

## Details

- **Base model:** [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B)
- **Method:** Calibration-guided per-tensor 3-bit quantization, aard-Q3 pack via modified llama.cpp writer
- **Output projection:** Q6_K (all other 497 weight tensors at Q3_K)
- **Calibration:** Chat-formatted prompts across code, math, reasoning, factual, creative

## More from aaardpark

- [gemma-4-31B-it](https://huggingface.co/aaardpark/gemma-4-31B-it-GGUF) — 15.3 GB, 96% GSM8K
- [Qwen 2.5 72B Instruct GGUF](https://huggingface.co/aaardpark/Qwen2.5-72B-Instruct-GGUF) — 35 GB, 88% GSM8K
- [Qwen 2.5 32B Instruct GGUF](https://huggingface.co/aaardpark/Qwen2.5-32B-Instruct-GGUF) — 15 GB