Squash history: release snapshot (frees LFS blobs from renamed/replaced files)

.gitattributes

@@ -0,0 +1,48 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+imatrix-mixed-v2.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
+assets/kld-vs-size-linear.png filter=lfs diff=lfs merge=lfs -text
+assets/kld-vs-size.png filter=lfs diff=lfs merge=lfs -text
+assets/top1-vs-size.png filter=lfs diff=lfs merge=lfs -text
+mmproj-BF16.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+assets/gain-vs-unsloth.png filter=lfs diff=lfs merge=lfs -text
+Qwen3.8-27B-AP-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text

Qwen3.8-27B-AP-IQ3_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5c916a8855ec2ef4dfc8295d21e8ebf0d4ad1a8381b6af76c7b44e9d7f1cbcf
+size 12040882080

Qwen3.8-27B-AP-IQ3_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1f018d9d76022f9ecda8d636b1f72c03c4efe40fee817ebfca3352ccf12cc74
+size 10737408160

Qwen3.8-27B-AP-IQ4_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2074fd5c3c7f6540913c2f62ad02c50b3f7dde7880d18b3acb02432f2edcab67
+size 14252844960

Qwen3.8-27B-AP-Q3_K_XL.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5b35d550712f01fcaa0a982f281b7bcf47f6309c541450ca0e064c945c1c8fc
+size 13146392480

Qwen3.8-27B-AP-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4c4b1d393b288205d6303c941c0c954d0ea57ef8e3228bca74187cc858e9d8e
+size 16464439200

Qwen3.8-27B-AP-Q4_K_XL.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5490390fa01335062247d4f43190a6659117e3c388b26260b81285531edc3dfc
+size 17559177120

README.md

@@ -0,0 +1,149 @@
+---
+base_model:
+- Qwen/Qwen3.8-27B
+base_model_relation: quantized
+license: apache-2.0
+library_name: gguf
+pipeline_tag: text-generation
+tags:
+- gguf
+- qwen3.8-27b
+- imatrix
+- agentionai
+---
+
+# Qwen3.8-27B · Agention Precision GGUF
+
+**Same size. Same speed. Closer to BF16.**
+
+![Lower KL at the same GiB means more of Qwen left in the file](assets/kld-vs-size.png)
+
+Agention Precision is a drop-in GGUF pack for [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B): standard llama.cpp types, no fork, no flags. Vision projector and MTP draft head included.
+
+Every 27B quant gives something up. These give up less. We measured the leading public GGUFs against the full BF16 model on one protocol. At every size we ship, ours is the closest to Qwen’s next-token distribution on unseen technical text and on general web text.
+
+Swap cost is zero. VRAM and tokens/sec stay the same. The file behaves more like the weights Qwen released.
+
+| Start here | File | Size | 32k VRAM | Fits | Gain vs leading same-size quant |
+|---|---|---:|---:|---|---|
+| Most headroom | **`AP-Q4_K_XL`** | 16.35 GiB | ~18 GiB | 24 GB | **4% closer** to BF16; hardest 1% of tokens **7% closer** |
+| 24 GB, smaller | **`AP-Q4_K_M`** | 15.33 GiB | ~17 GiB | 24 GB | **5% closer** on unseen technical text; hardest 1% of tokens **9% closer** |
+| Default | **`AP-IQ4_XS`** | 13.27 GiB | ~15 GiB | 16 GB | **8% closer** to BF16; hardest 1% of tokens **9% closer** |
+| Need headroom | **`AP-Q3_K_XL`** | 12.24 GiB | ~14 GiB | 16 GB + longer ctx | **10% closer** on unseen technical text |
+| 12 GB, more quality | **`AP-IQ3_S`** | 11.21 GiB | ~13 GiB | 16 GB at 8–16k | **8% closer** on unseen technical text, **5%** on web; wikitext-2 12% further |
+| 12 GB, balanced | **`AP-IQ3_XS`** | 10.70 GiB | ~12.5 GiB | 12 GB at 8–16k | **6–8% closer** than the best research quant at this size, **19%** on wikitext-2 |
+| 12 GB / multi-model | **`AP-IQ3_XXS`** | 10.00 GiB | ~12 GiB | 12 GB at 8–16k | **10–22% closer** than the best ~10 GiB research quant |
+| Vision | `mmproj-BF16.gguf` | 0.87 GiB | +0.9 GiB | any tier | Qwen’s own encoder at BF16 |
+
+VRAM = weights + q8_0 KV + llama.cpp buffers. Only a quarter of layers use full attention, so 32k context is about 1 GiB at q8_0.
+
+**Take `AP-IQ4_XS` if it fits.** Step down only for memory.
+
+---
+
+## Measured, not claimed
+
+Fidelity = KL divergence of the next-token distribution vs Qwen3.8-27B BF16. Lower is closer to the original.
+
+60 × 2048 tokens, three corpora, same build, same BF16 logits:
+
+- **held-out** — technical prose the build never saw
+- **neutral web** — seeded FineWeb slice, [agentionai/quant-fidelity-corpora](https://huggingface.co/datasets/agentionai/quant-fidelity-corpora)
+- **wikitext-2** — test split
+
+### Head to head with Unsloth, identical bytes
+
+Same tensor types, same file size, same speed, same memory. Only the calibration changes.
+
+| | held-out | neutral web | wikitext-2 | worst 1% tokens (held-out) |
+|---|---|---|---|---|
+| `UD-Q4_K_XL` | 0.0117 | 0.0087 | 0.0122 | 0.088 |
+| **`AP-Q4_K_XL`** | **0.0111 (−4.4%)** | **0.0083 (−4.5%)** | 0.0118 | **0.082 (−7.2%)** |
+| `UD-Q4_K_M` | 0.0153 | 0.0108 | 0.0139 | 0.122 |
+| **`AP-Q4_K_M`** | **0.0146 (−4.9%)** | **0.0104 (−3.7%)** | 0.0150 | **0.111 (−8.8%)** |
+| `UD-IQ4_XS` | 0.0276 | 0.0186 | 0.0252 | 0.233 |
+| **`AP-IQ4_XS`** | **0.0255 (−7.6%)** | **0.0181 (−2.4%)** | **0.0243 (−3.8%)** | **0.211 (−9.3%)** |
+| `UD-Q3_K_XL` | 0.0421 | 0.0270 | 0.0337 | 0.366 |
+| **`AP-Q3_K_XL`** | **0.0380 (−9.9%)** | **0.0250 (−7.6%)** | 0.0359 | **0.327 (−10.6%)** |
+| `UD-IQ3_S` | 0.0617 | 0.0404 | **0.0470** | 0.553 |
+| **`AP-IQ3_S`** | **0.0568 (−7.9%)** | **0.0384 (−5.0%)** | 0.0528 (+12%) | **0.491 (−11.2%)** |
+
+Held-out gains: 3.2σ, 3.2σ, 5.7σ, 8.1σ and 6.7σ (`Q4_K_XL` / `Q4_K_M` / `IQ4_XS` / `Q3_K_XL` / `IQ3_S`). Neutral-web gains: 3.7σ at `Q4_K_XL`, 5.0σ at `Q3_K_XL`, 3.1σ at `IQ3_S`; 2.4σ at `Q4_K_M`. Wikitext-2 is a statistical tie (under 1.5σ) at every tier except `IQ3_S`, where Unsloth's file is 12% closer (3.0σ) — the one column and tier where they lead; read the others as even.
+
+Top-1 match with BF16 on held-out text: **92.8%** · **92.2%** · **90.4%** · **89.2%** · **87.6%** · **86.6%** · **85.0%** (`Q4_K_XL` / `Q4_K_M` / `IQ4_XS` / `Q3_K_XL` / `IQ3_S` / `IQ3_M` / `IQ3_XXS`).
+
+### The field near these sizes
+
+| file | size | held-out | neutral web | wikitext-2 |
+|---|---:|---|---|---|
+| **`AP-Q4_K_XL`** | **16.35 GiB** | **0.0111** | **0.0083** | **0.0118** |
+| unsloth `UD-Q4_K_XL` | 16.35 GiB | 0.0117 | 0.0087 | 0.0122 |
+| **`AP-Q4_K_M`** | **15.33 GiB** | **0.0146** | **0.0104** | 0.0150 |
+| unsloth `UD-Q4_K_M` | 15.33 GiB | 0.0153 | 0.0108 | 0.0139 |
+| AtomicChat `AD-IQ4_XS-IQ3_S` | 13.45 GiB | 0.0335 | 0.0234 | 0.0384 |
+| **`AP-IQ4_XS`** | **13.27 GiB** | **0.0255** | **0.0181** | **0.0243** |
+| unsloth `UD-IQ4_XS` | 13.27 GiB | 0.0276 | 0.0186 | 0.0252 |
+| AtomicChat `AD-IQ3_S` | 12.89 GiB | 0.0441 | 0.0303 | 0.0441 |
+| **`AP-Q3_K_XL`** | **12.24 GiB** | **0.0380** | **0.0250** | 0.0359 |
+| unsloth `UD-Q3_K_XL` | 12.24 GiB | 0.0421 | 0.0270 | 0.0337 |
+| ISTA-DASLab `GSQ-RCO-IQ3_S` | 11.29 GiB | 0.0594 | 0.0432 | 0.0665 |
+| **`AP-IQ3_S`** | **11.21 GiB** | **0.0568** | **0.0384** | 0.0528 |
+| unsloth `UD-IQ3_S` | 11.21 GiB | 0.0617 | 0.0404 | **0.0470** |
+| **`AP-IQ3_XS`** | **10.70 GiB** | **0.0714** | **0.0508** | **0.0642** |
+| AtomicChat `AD-IQ2_S` | 10.38 GiB | 0.1187 | 0.0858 | 0.1062 |
+| **`AP-IQ3_XXS`** | **10.00 GiB** | **0.1012** | **0.0689** | **0.0834** |
+| ISTA-DASLab `GSQ-RCO-IQ3_XXS` | 9.73 GiB | 0.1123 | 0.0824 | 0.1063 |
+
+ISTA rows are their `-mtp` builds (draft head included, same as ours).
+
+KL is fidelity to Qwen’s predictions, not a task leaderboard. Downstream evals are next. Until then the claim is narrow and checkable: at every size we ship, you are closer to the original than the same-size alternative.
+
+---
+
+## Running
+
+Use the sampling settings from the
+[Qwen3.8-27B model card](https://huggingface.co/Qwen/Qwen3.8-27B). Thinking is on by
+default. To turn it off per request, send
+`"chat_template_kwargs": {"enable_thinking": false}`.
+
+```bash
+llama-server -hf agentionai/Qwen3.8-27B-AP-GGUF:IQ4_XS \
+  --jinja -ngl 999 -fa on -c 32768 -ctk q8_0 -ctv q8_0
+```
+
+Keep the KV cache at q8_0 or f16. A 4-bit value cache makes long reasoning traces
+degenerate into repetition on this model family.
+
+**LM Studio:** search for `agentionai/Qwen3.8-27B-AP-GGUF` and pick a tier.
+**Ollama:** `ollama run hf.co/agentionai/Qwen3.8-27B-AP-GGUF:IQ4_XS`
+
+**Vision:** download `mmproj-BF16.gguf` alongside the tier and pass it with
+`--mmproj` (llama-server, llama-mtmd-cli). LM Studio picks it up automatically when
+it sits next to the model. This is Qwen3.8-27B's own vision encoder at BF16, and it
+works with every tier here.
+
+```bash
+llama-server -hf agentionai/Qwen3.8-27B-AP-GGUF:IQ4_XS \
+  --mmproj mmproj-BF16.gguf --jinja -ngl 999 -fa on -c 32768 -ctk q8_0 -ctv q8_0
+```
+
+## Built with
+
+The tiers are built and verified with our own Rust tooling,
+[agention-infer](https://github.com/agentionai). Every tier is measured against BF16
+on all three corpora, and shipped only if it beats the same-size alternative.
+[unsloth](https://huggingface.co/unsloth)'s dynamic type maps underpin `AP-Q4_K_XL`,
+`AP-Q4_K_M`, `AP-IQ4_XS`, `AP-Q3_K_XL` and `AP-IQ3_S`, and we thank them for that work.
+`AP-IQ3_XS` and `AP-IQ3_XXS` sit at sizes no one else ships and use our own per-tensor
+allocation, so they are compared against the nearest published files above and below them. `imatrix-mixed-v2.gguf`, the
+importance matrix from our calibration pass, is included for anyone building their
+own quants.
+
+More tiers follow as they clear the same bar.
+
+### Support AgentionAI
+
+These quants are released freely. If they save you VRAM or make Qwen more useful, you
+can buy me some GPU time and sponsor continued quantization and benchmarking on
+[GitHub](https://github.com/sponsors/agentionai). AgentionAi is a one person team and can use your help.

imatrix-mixed-v2.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:deaddb7d88a991e216bebd3da61e627b0f0ff08912d14843d7bdfb255ac331f0
+size 13642688

mmproj-BF16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83ee4f4f205fa514161778c41df1ea14144faa0f713510893b63c2395f5c2d53
+size 931146432