Upload folder using huggingface_hub

.gitattributes

@@ -49,3 +49,7 @@ Muse-Glimmer-30B-AK-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
 Muse-Glimmer-30B-AK-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 Muse-Glimmer-30B-AK-Q6_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
 Muse-Glimmer-30B-AK-Q8_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
+domain_slices.png filter=lfs diff=lfs merge=lfs -text
+kld_vs_size.png filter=lfs diff=lfs merge=lfs -text
+matched_pairs.png filter=lfs diff=lfs merge=lfs -text
+tail_p999.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,148 @@
+---
+base_model: meta-models/Muse-Glimmer-30B
+base_model_relation: quantized
+license: apache-2.0
+library_name: gguf
+pipeline_tag: image-text-to-text
+tags:
+  - gguf
+  - llama.cpp
+  - quantized
+  - imatrix
+  - muse_glimmer
+  - conversational
+---
+
+# Muse-Glimmer-30B - GGUF (AK line)
+
+> 👋 I built this line solo - the calibration, the per-tensor allocations, and the eval harness behind
+> every number below. I'm looking for **internships in AI agent orchestration and model inference**.
+> If this work looks relevant to your team: [linkedin.com/in/theaaryankapoor](https://www.linkedin.com/in/theaaryankapoor/)
+
+**State-of-the-art GGUF quantizations for
+[`meta-models/Muse-Glimmer-30B`](https://huggingface.co/meta-models/Muse-Glimmer-30B).** Eight builds
+(27.86 B params, 52 dense layers, GQA 32:2), each with a custom per-tensor bit allocation derived for its
+size point - plus the stock BF16 vision encoder.
+
+Benchmarked head-to-head against the **Unsloth, Meta and bartowski** lines, every file scored on the same
+rig against the same BF16 reference: **24 wins, 6 statistical ties, 0 losses across 30 paired comparisons
+on two evaluation sets.**
+
+![KL divergence vs file size](kld_vs_size.png)
+
+One line per publisher. Log y, bits-per-weight on the secondary axis, and the crowded 16 GB class magnified.
+
+![Matched pairs](matched_pairs.png)
+
+Every comparison with its 95 % interval - blue clears zero, grey is a statistical tie, and the right
+column carries the held-out C4 verdict. The full numbers are in the table below.
+
+Comparison set: the three widest-distribution GGUF lines for this model, as published **2026-08-11**;
+the Method section has everything needed to reproduce any number here.
+
+> **File naming.** Every quant in this line carries the `AK-` prefix: these are custom per-tensor
+> allocations, not llama.cpp's stock recipes, so `AK-Q4_K_M` and a stock `Q4_K_M` are different files.
+> `mmproj` keeps its upstream name.
+
+## Which file do I want?
+
+| file | size | bpw | mean KLD ↓ | top-1 ↑ | vs closest rival |
+|---|---|---|---|---|---|
+| `AK-Q2_K_XL` | 12.45 GB | 3.576 | 0.056036 | 90.88 % | **−27 %** KLD |
+| `AK-Q3_K_XL` | 13.51 GB | 3.880 | 0.039079 | 92.30 % | **−27 %** KLD |
+| `AK-Q4_K_M` | 15.86 GB | 4.556 | 0.013897 | 95.38 % | **−6 %** KLD |
+| **`AK-Q4_K_XL`** | 16.26 GB | 4.669 | **0.012286** | 95.65 % | **−14 %** KLD |
+| `AK-Q5_K_M` | 19.19 GB | 5.512 | 0.004974 | 97.26 % | best measured (tie) |
+| `AK-Q6_K_XL` | 26.24 GB | 7.536 | 0.000876 | 98.82 % | **−4 %** KLD |
+| **`AK-Q8_K_L`** | 32.28 GB | 9.272 | **0.000356** | 99.25 % | **−21 %** KLD, smaller file |
+| `AK-Q8_K_XL` | 34.96 GB | 10.040 | 0.000316 | 99.30 % | most faithful build |
+| `mmproj` BF16 | 3.85 GB | - | - | vision encoder | stock, unquantized |
+
+`AK-Q4_K_XL` is the strongest file in the crowded 16 GB class - no published quant of this model at any
+comparable size comes within 13 % of it. At Q8, `AK-Q8_K_L` beats Unsloth's build **while being smaller**.
+
+```bash
+llama-server -m Muse-Glimmer-30B-AK-Q4_K_XL.gguf \
+             --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf -c 8192 -ngl 99
+```
+
+## Full measurement table
+
+| publisher | file | bytes | bpw | PPL ratio | mean KLD | p99.9 KLD | top-1 | Δ vs closest rival |
+|---|---|---|---|---|---|---|---|---|
+| bartowski | Q2_K_L | 12,348,891,936 | 3.547 | 1.113088 | 0.132355 | 4.5183 | 86.289 % | |
+| Unsloth | UD-Q2_K_XL | 12,444,212,256 | 3.574 | 1.065168 | 0.077057 | 2.9476 | 89.240 % | |
+| **AaryanK** | **AK-Q2_K_XL** | 12,451,267,776 | 3.576 | 1.048590 | **0.056036** | 2.4523 | 90.878 % | **−27.3 %** [−28.3, −26.3] |
+| Unsloth | UD-Q3_K_XL | 13,360,983,072 | 3.837 | 1.047022 | 0.053173 | 2.2479 | 91.175 % | |
+| **AaryanK** | **AK-Q3_K_XL** | 13,509,095,872 | 3.880 | 1.033090 | **0.039079** | 1.5369 | 92.303 % | **−26.5 %** [−27.8, −25.2] |
+| bartowski | Q3_K_M | 13,962,519,328 | 4.010 | 1.032023 | 0.039487 | 1.6390 | 92.303 % | |
+| bartowski | IQ4_XS | 15,435,096,096 | 4.433 | 1.010121 | 0.015440 | 0.6246 | 95.128 % | |
+| **AaryanK** | **AK-Q4_K_M** | 15,864,857,280 | 4.556 | 1.010896 | **0.013897** | 0.5592 | 95.378 % | **−5.6 %** [−7.6, −3.3] |
+| Unsloth | UD-Q4_K_XL | 15,878,222,368 | 4.560 | 1.010630 | 0.014714 | 0.5871 | 95.249 % | |
+| **AaryanK** | **AK-Q4_K_XL** | 16,255,873,984 | 4.669 | 1.008957 | **0.012286** | 0.5601 | 95.647 % | **−14.0 %** [−15.4, −12.7] |
+| bartowski | Q4_K_S | 16,320,943,136 | 4.687 | 1.010071 | 0.014293 | 0.5860 | 95.319 % | |
+| Meta | kquant-17gb | 16,756,681,056 | 4.812 | 1.009871 | 0.014146 | 0.5918 | 95.297 % | |
+| **AaryanK** | **AK-Q5_K_M** | 19,191,472,832 | 5.512 | 1.003958 | **0.004974** | 0.1922 | 97.256 % | −2.3 % [−4.8, **+0.6**] tie |
+| Unsloth | UD-Q5_K_M | 19,194,274,848 | 5.513 | 1.004517 | 0.005092 | 0.2027 | 97.157 % | |
+| **AaryanK** | **AK-Q6_K_XL** | 26,238,366,400 | 7.536 | 1.000831 | **0.000876** | 0.0384 | 98.819 % | **−3.8 %** [−5.8, −1.9] |
+| Unsloth | UD-Q6_K_XL | 26,265,362,976 | 7.543 | 1.000885 | 0.000911 | 0.0386 | 98.867 % | |
+| **AaryanK** | **AK-Q8_K_L** | 32,283,878,048 | 9.272 | 1.000606 | **0.000356** | 0.0148 | 99.248 % | **−20.8 %** [−23.8, −17.5] |
+| Unsloth | UD-Q8_K_XL | 32,300,651,040 | 9.277 | 1.000728 | 0.000450 | 0.0197 | 99.126 % | |
+| **AaryanK** | **AK-Q8_K_XL** | 34,958,791,360 | 10.040 | 1.000680 | **0.000316** | 0.0140 | 99.301 % | **−29.7 %** [−32.4, −26.6] |
+
+Intervals are a paired per-token cluster bootstrap over the 60 evaluation chunks. Δ is against the
+**closest-sized non-AaryanK** file.
+
+### Reading the numbers
+
+**Two Q8 builds, two jobs.** `AK-Q8_K_L` is the size-class winner - smaller than Unsloth's Q8 and −20.8 %
+KLD, confirmed on every slice tested. `AK-Q8_K_XL` is the maximum-fidelity build: −29.7 % at +8.2 % bytes
+(10.04 bpw vs 9.28), for when the last 2.7 GB of VRAM is cheaper than the last drop of divergence.
+
+**PPL ratio and KLD disagree at Q3 and Q4_K_M.** `AK-Q3_K_XL` leads bartowski's Q3_K_M on the KLD point
+estimate (a statistical tie - see below) while losing PPL ratio by 0.001. `AK-Q4_K_M` shows the same pattern against Meta's build, where it leads
+the KLD point estimate but the interval makes it a tie. PPL scores only the
+probability of the true next token; KLD scores the whole distribution. Both are in the table.
+
+## Does the margin generalise?
+
+![Domain slices](domain_slices.png)
+
+The same files re-measured on six evaluation sets, four held out and audited at zero fragment overlap with
+any calibration corpus. **11 of 16 held-out margins exceed the same file's wikitext margin**, and every
+interval in the chart excludes zero - including all four held-out domains for `AK-Q8_K_L`, where the
+size-matched Q8 lead spans −17 % to −24 %.
+
+## Tail behaviour
+
+![Tail percentile](tail_p999.png)
+
+## Method
+
+* **Reference:** our own BF16 GGUF, converted with llama.cpp pinned at `62bf73d2`. The conversion was
+  checked against every competitor's file across 21 load-bearing KVs, so the comparison measures
+  quantization rather than a conversion delta.
+* **Eval:** `llama-perplexity --kl-divergence`, ctx 4096 × 60 chunks → 122,820 scored tokens.
+  **ctx 4096 matters for this architecture** - it alternates 3× sliding-window (2048) with 1×
+  full-attention NoPE layers, and only at ctx ≥ 4096 does every scored token sit beyond the window.
+* **Statistics:** paired per-token cluster bootstrap at the 2047-token chunk width for every interval.
+* **Confirmation:** the Q4 result was re-run under 9 independent calibration draws across three corpus
+  families on an untouched slice - beneficial in 9/9, no reversals, every interval excluding zero,
+  against an MDE fixed before any data was collected.
+* **Long context:** re-measured at ctx 8192; the lead holds and slightly grows.
+* **Capability:** a 130-case tool-calling suite scored as paired agreement with BF16 - `AK-Q4_K_M`
+  matches BF16 on 128 of 130 cases with one flip in each direction: statistically
+  indistinguishable (exact McNemar p = 1.000).
+* Ten pre-registered apparatus gates, all passing, including full-vocabulary agreement with HF
+  `transformers` and exact greedy generation agreement (235/235 tokens).
+* **Scope:** the text tower is what is measured and quantized; `mmproj` ships as the stock BF16 encoder.
+  KLD values are model-local (this head applies logit soft-capping) - compare within this table only.
+
+Held-out sets: GitHub source (numpy/redis/django/sqlite/nlohmann), OASST dialogue, GSM8K + arXiv
+abstracts, and Wikipedia in sixteen non-English languages.
+
+---
+
+*Per-tensor bit allocation derived separately at each target bit-width using importance data from a
+diverse in-house calibration set. Base model licence and usage policy unchanged from
+[`meta-models/Muse-Glimmer-30B`](https://huggingface.co/meta-models/Muse-Glimmer-30B) (Apache-2.0).*