aaardpark/Qwen3.5-27B-GGUF

Qwen3.5-27B | aard-Q3

A 3-bit, 11 GB GGUF of Qwen/Qwen3.5-27B that fits comfortably in 16 GB VRAM with 32K-token context headroom.

Runs in llama.cpp, LM Studio, Ollama, and any GGUF-compatible runtime.

Bottom line. Token distributions are effectively the same between FP16 and aard-Q3. Across 62 reasoning problems (50 GSM8K + 12 hard eval), zero failures were attributable to quantization — every miss either reproduced on the FP16 reference or was a harness truncation we could fix with more tokens.

At 11 GB, this is the smallest published 3-bit Qwen3.5-27B GGUF and the only one with measured quality numbers against FP16. If you have a 16 GB GPU, there's no reason to run a larger 3-bit quant of this model.

The best 3-bit quant of Qwen3.5-27B

Other 3-bit GGUFs of this model are 13.3–13.9 GB and publish no quality evaluations. Bartowski's own card labels their Q3_K_M "Low quality." This is the smallest 3-bit GGUF of Qwen3.5-27B with published quality numbers.

How it stacks up

Provider	Size	Published evals
bartowski Q3_K_M	13.85 GB	None — card says "Low quality"
unsloth UD-Q3_K_XL	14.4 GB	None vs FP16
unsloth Q3_K_M	13.5 GB	None vs FP16
mradermacher Q3_K_M	13.3 GB	None
lmstudio-community Q3_K_M	~13.5 GB	None
unsloth UD-IQ3_XXS	11.5 GB	None vs FP16
aard-Q3 (this one)	11 GB	GSM8K 96%, KL 0.052

Published evals

Metric	Value	What it means
GSM8K 5-shot (n=50)	48/50 = 96%	Zero misses caused by quantization — see breakdown below
KL(FP16‖3-bit) mean	0.052 nats	Token distributions are effectively the same as FP16
KL p95	0.287	No catastrophic per-token divergences
KL p99	0.706	Rare outliers stay bounded
Mean loss delta	+0.063 nats/token	Held-out loss within noise of FP16

Held-out set: 21 prompts across code, math, reasoning, factual, creative; 150 response tokens per prompt, greedy decoding.

GSM8K miss breakdown (4 problems)

#	Expected	Predicted	Cause
6	64	1	Truncation at 2048 tokens — recovered at 4096
11	366	180	Truncation at 2048 tokens — recovered at 4096
13	13	12	Defensible interpretation: break-even year 12 vs first profitable year 13
46	104	2	Ambiguous "2/5 times more" phrasing; model commits to one reading, gold expects the other

Three of four are harness-level (raising the token budget fixes them); the fourth is a known ambiguity in the GSM8K prompt. With the token fix applied, effective accuracy is 48/50 = 96%. No miss was traced to quantization.

Quick stats

File	Qwen3.5-27B-aaardpark-aard-Q3.gguf
Size	11 GB
Format	GGUF, aard-Q3 pack (497 × Q3_K + 1 × Q6_K, ~3.07 BPW)
Min VRAM	16 GB
Throughput	~22 tok/s on Apple M-series (llama.cpp, Metal, 99 GPU layers)
Native context	262K (linear attention)

Download & run

huggingface-cli download aaardpark/Qwen3.5-27B-GGUF \
  Qwen3.5-27B-aaardpark-aard-Q3.gguf --local-dir .

llama-cli -m Qwen3.5-27B-aaardpark-aard-Q3.gguf -ngl 99 -c 32768

Notes on use

• Thinking model. Qwen 3.5 is a reasoning model — it emits a <think>…</think> block before the final answer. Budget at least 1024–2048 tokens for responses on non-trivial prompts; 4096 for hard reasoning. On extraction, split on </think> and take the post-block content.
• Runtime requirements. Needs llama.cpp build 8670 or later (Gated Delta Net / linear_attn support). LM Studio / Ollama users: make sure your bundled runtime is current, or load may fail with a generic error.
• Chat template. Embedded in the GGUF — use the default chat format; no manual templating required.

Why this quant

We don't spend bits — we spend better rounding.

Most 3-bit GGUFs labeled "Q3_K_M" mix in Q4_K and Q5_K tensors to protect sensitive layers (attention-V/K, FFN-down), landing around 4 BPW and 13.5 GB. This one keeps 497 of 498 weight tensors at pure Q3_K with only the output projection at Q6_K (~3.07 BPW). The aggressive pack is normally quality-poisonous. Calibration-guided rounding is what keeps it coherent at this bit budget.

Same file format. Runs anywhere Q3_K_M runs. Smaller file, held-out quality.

Outputs

Debugging

Here's a Python function intended to merge two sorted lists. Find and fix the bugs:

def merge(a, b):
    result = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] < b[j]:
            result.append(a[i])
            i += 1
        else:
            result.append(b[j])
            i += 1
    result.extend(a[i:])
    result.extend(b[j:])
    return result

The bug is in the else branch of the while loop. When a[i] >= b[j], the code appends b[j] but increments i instead of j. This causes the wrong pointer to advance, so later iterations re-read b[j] and eventually walk off the end of a.

def merge(a, b):
    result = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] < b[j]:
            result.append(a[i])
            i += 1
        else:
            result.append(b[j])
            j += 1  # was: i += 1
    result.extend(a[i:])
    result.extend(b[j:])
    return result

---

Explain a concept to a non-expert

I'm a backend engineer who's never touched databases before. Explain the difference between a B-tree index and a hash index in one paragraph, focused on when I'd use each.

A B-tree index organizes data in a sorted tree structure, making it the default choice for general queries because it supports range searches (e.g., "find users with IDs between 100 and 200") and sorting (ORDER BY), whereas a hash index uses a hash function to map keys directly to memory addresses, offering O(1) speed for exact equality lookups (e.g., "find user ID 123") but failing completely for range queries or sorting since the data is unordered. Use a B-tree index for most general-purpose queries where you need to filter by ranges or sort results, and switch to a hash index only when you have a high volume of exact equality lookups and never need to perform range queries or sorting on that column.

---

Diagnose a practical problem

My laptop fan spins up to full speed only when I'm on video calls — nothing else. Chrome tabs, code compilation, games, all fine. Only video calls. Give me three likely causes ranked by probability.

1. Hardware Acceleration disabled (most likely). Video-call apps default to software encoding when hardware acceleration is off, forcing the CPU to do work the GPU normally handles. Games and compilation are tuned for GPU; video calls fall back to CPU and spike it. Check Zoom/Teams/Meet settings and your browser for a "Hardware Acceleration" toggle.

2. AI features (Background Blur / Smart Framing). Background blur and face-tracking run CPU-side real-time segmentation on every frame. Games don't touch this code path, so they stay cool. Turn off "Blur," "Virtual Background," "Smart Framing" and re-test.

3. Power profile triggered by camera activation. Some OEM power managers (Dell, HP, Lenovo) escalate to Performance mode when the webcam turns on, regardless of actual load. Check for camera-linked profile rules in your vendor's power utility.

VRAM requirements

Qwen3.5-27B uses Gated Delta Net (linear attention), so the context state is fixed-size per layer rather than growing with sequence length. Context scales further than standard transformers on the same VRAM.

VRAM	Experience
16 GB	Fits. 32K context tested and comfortable.
24 GB	128K context tested.
32 GB+	Full 262K native context.

Details

• Base model: Qwen/Qwen3.5-27B
• Method: Calibration-guided per-tensor 3-bit quantization, aard-Q3 pack via modified llama.cpp writer
• Output projection: Q6_K (all other 497 weight tensors at Q3_K)
• Calibration: Chat-formatted prompts across code, math, reasoning, factual, creative

More from aaardpark

• gemma-4-31B-it — 15.3 GB, 96% GSM8K
• Qwen 2.5 72B Instruct GGUF — 35 GB, 88% GSM8K
• Qwen 2.5 32B Instruct GGUF — 15 GB