--- license: apache-2.0 base_model: - ManniX-ITA/Ornith-1.5-27B-A3B-Coder base_model_relation: quantized library_name: gguf pipeline_tag: text-generation language: - en tags: - code - expert-pruning - gguf - imatrix - llama.cpp - moe - mtp - ornith model_name: Ornith-1.5-27B-A3B-Coder-MTP-GGUF author: ManniX-ITA architecture: Qwen3.5-MoE parameters: 26.66B active_parameters: 3B format: GGUF quantizations: - .imatrix - F16 - IQ2_M - IQ2_XS - IQ3_M - IQ4_NL - IQ4_XS - Q2_K_L - Q3_K_L - Q3_K_M - Q3_K_S - Q3_K_XL - Q4_K_L - Q4_K_M - Q4_K_S - Q5_K_L - Q5_K_M - Q5_K_S - Q6_K - Q6_K_L - Q8_0 features: - MTP - MoE - code - expert-pruning - imatrix - reasoning - vision --- # Ornith-1.5-27B-A3B-Coder — GGUF (imatrix, MTP) GGUF quantizations of [**`ManniX-ITA/Ornith-1.5-27B-A3B-Coder`**](https://huggingface.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder) — the pure competence-map 184-expert cut of `Ornith-1.5-35B-A3B`. **→ Recipe, drop maps, the full 11-bench evaluation, the cross-family comparison and the length/deliberation analysis all live on the [weights card](https://huggingface.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder).** This card covers the quantized files only. > Its sibling [**CoderX**](https://huggingface.co/ManniX-ITA/Ornith-1.5-27B-A3B-CoderX-MTP-GGUF) > uses the same map with a REAP stability floor and measures better on almost every code and > reasoning axis at the same expert budget and the same file size. Start there unless you > specifically want this arm. ## The `-MTP-` claim, verified from this file's own metadata ``` qwen35moe.block_count = 41 (40 trunk + 1 MTP) qwen35moe.nextn_predict_layers = 1 distinct blk indices = 41, max = 40 <- block 40 is actually SHIPPED qwen35moe.expert_count = 184 qwen35moe.expert_used_count = 8 ``` ## Quants Built **with an importance matrix at every tier, including Q6_K**, from this arm's **own** imatrix — an imatrix cannot be shared between two different expert cuts. Sizes are GB (bytes ÷ 1e9). | file | size | notes | |---|---|---| | `Ornith-1.5-27B-A3B-Coder-F16.gguf` | 52.47 GB | unquantised source for re-quantising | | `Ornith-1.5-27B-A3B-Coder-Q8_0.gguf` | 27.91 GB | near-lossless reference | | `Ornith-1.5-27B-A3B-Coder-Q6_K_L.gguf` | 21.81 GB | Q6_K with embed/output at Q8_0 | | `Ornith-1.5-27B-A3B-Coder-Q6_K.gguf` | 21.57 GB | recommended; **this is the eval tier** | | `Ornith-1.5-27B-A3B-Coder-Q5_K_L.gguf` | 19.04 GB | | | `Ornith-1.5-27B-A3B-Coder-Q5_K_M.gguf` | 18.73 GB | | | `Ornith-1.5-27B-A3B-Coder-Q5_K_S.gguf` | 18.16 GB | | | `Ornith-1.5-27B-A3B-Coder-Q4_K_L.gguf` | 16.43 GB | Q4_K_M with embed/output at Q8_0 | | `Ornith-1.5-27B-A3B-Coder-Q4_K_M.gguf` | 16.06 GB | default `:latest` on Ollama; fits 24 GB VRAM with context | | `Ornith-1.5-27B-A3B-Coder-Q4_K_S.gguf` | 15.07 GB | | | `Ornith-1.5-27B-A3B-Coder-IQ4_NL.gguf` | 15.01 GB | | | `Ornith-1.5-27B-A3B-Coder-IQ4_XS.gguf` | 14.22 GB | best size/quality under 15 GB | | `Ornith-1.5-27B-A3B-Coder-Q3_K_XL.gguf` | 13.17 GB | | | `Ornith-1.5-27B-A3B-Coder-Q3_K_L.gguf` | 13.74 GB | | | `Ornith-1.5-27B-A3B-Coder-Q3_K_M.gguf` | 12.72 GB | | | `Ornith-1.5-27B-A3B-Coder-Q3_K_S.gguf` | 11.53 GB | | | `Ornith-1.5-27B-A3B-Coder-IQ3_M.gguf` | 11.74 GB | | | `Ornith-1.5-27B-A3B-Coder-Q2_K_L.gguf` | 10.36 GB | | | `Ornith-1.5-27B-A3B-Coder-IQ2_M.gguf` | 9.06 GB | | | `Ornith-1.5-27B-A3B-Coder-IQ2_XS.gguf` | 8.20 GB | smallest; expect real degradation | | `Ornith-1.5-27B-A3B-Coder.imatrix.gguf` | 0.14 GB | this arm's own imatrix, for reproduction/audit | | `mmproj-Ornith-1.5-27B-A3B-Coder-F16.gguf` | 0.90 GB | **vision projector** — see Vision below | ## Calibration (imatrix) The importance matrix for every tier in this repo was computed on **[AtomicChat/calib-corpora](https://huggingface.co/datasets/AtomicChat/calib-corpora)**, build **`builds/ornith-1.5-9b`** — a published, manifest-verified calibration corpus rather than an ad-hoc text dump, so the quantisation basis is auditable and reproducible. | | | |---|---| | corpus | [`AtomicChat/calib-corpora`](https://huggingface.co/datasets/AtomicChat/calib-corpora) `builds/ornith-1.5-9b` | | `calib_train.txt` | 3,079 documents · 4,958,490 tokens | | chunks used | 9,686 (full file — no `--chunks` cap) | | special tokens | `--parse-special` **on** (the corpus is chat-rendered; without it the chat control tokens never reach the statistics) | | mixture | agentic 24.7% · code 17.8% · reasoning 14.8% · multilingual 13.8% · longctx 11.9% · vocab-sweep 9.9% · structured 3.9% · graphics 3.0% | **Corpus/model match was verified, not assumed.** The build is keyed to a tokenizer, and this arm's `tokenizer.json` hashes to `5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42` — byte-identical to the `tokenizer.sha256` recorded in that build's `manifest.json`. A calibration corpus rendered for a different tokenizer would silently mis-tokenise, so this check gates the reuse. The imatrix itself ships in this repo (`*.imatrix.gguf`) so the quants can be reproduced or audited. Note it is **this arm's own** imatrix: an imatrix cannot be shared between two different expert cuts. ## Serving ```bash llama-server -m Ornith-1.5-27B-A3B-Coder-Q6_K.gguf --port 8080 \ -c 262144 -ngl 99 --no-warmup \ --reasoning-format deepseek --reasoning-budget 12288 ``` `--reasoning-budget` is **mandatory** for this family. Use the vendor sampler (temp 0.6 / top_p 0.95 / top_k 20); greedy is not viable here. **Budget the tail, not the median.** On LiveCodeBench the median generation is ~12.7k tokens but 5 of 77 problems run to a 32,768-token ceiling. If you care about worst-case latency, bound it at serve time rather than hoping for a short draw — the weights card has the full distribution. ## Vision These weights are text-only on their own; the vision tower ships **separately** as `mmproj-Ornith-1.5-27B-A3B-Coder-F16.gguf` in this repo. Pass it alongside any quant: ```bash llama-server -m Ornith-1.5-27B-A3B-Coder-Q6_K.gguf \ --mmproj mmproj-Ornith-1.5-27B-A3B-Coder-F16.gguf \ --port 8080 -c 262144 -ngl 99 ``` Then post an image to `/v1/chat/completions` as an `image_url` data URI, exactly as with any other multimodal llama.cpp model. On Ollama the vision-capable tags are published under `vision-`: ```bash ollama run mannix/ornith-1.5-27b-a3b-coder:vision-Q4_K_M ``` **Verified, not assumed.** The projector was loaded against this arm in `llama.cpp` and shown a synthetic 5-band image in a deliberately non-obvious colour order (orange, purple, yellow, teal, brown). It named all five correctly and gave the right band count; an identical prompt with **no image attached** correctly answered that it could not see one — so the description comes from the pixels, not the prompt. ## Evaluation (Q6_K + imatrix, `llama.cpp`, sampler `recommended`) | Benchmark | base 35B (256e) | **Coder (184e)** | CoderX (184e) | |---|---|---|---| | GPQA-Diamond (198) | **0.8283** | 0.7677 | 0.8131 | | AIME (30) | 0.9000 | 0.9000 | **0.9667** | | LiveCodeBench v6 (77 hard) | 0.6623 | 0.7273 | **0.7662** | | HumanEval+ (164) | 0.8049 | **0.8293** | 0.8110 | | MultiPL-E (300) | **0.8533** | 0.8300 | 0.8367 | | **Mean (11 benches)** | 0.8252 | 0.8292 | **0.8386** | Full 11-bench table, per-language MultiPL-E, the Qwen3.6-27B-A3B cross-family comparison and every basis caveat: **[weights card](https://huggingface.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder)**. Apache-2.0 · research checkpoint. ## Tool-calling benchmark — `tool-eval-bench` hardmode (88 scenarios, 176 pts) **Benchmarked file: `Ornith-1.5-27B-A3B-Coder-IQ4_XS.gguf`** (this repo), served on `llama.cpp` with MTP speculative decoding enabled. The score below belongs to THIS quant — other tiers in this repo were not run. **Ornith-1.5-27B-A3B-Coder scores 136.8 ±4.8 of 176, eighth of ten**, statistically indistinguishable from [Qwen3.6-27B-A3B-CoderX](https://huggingface.co/ManniX-ITA/Qwen3.6-27B-A3B-CoderX) (137.4) — the CIs overlap almost completely — and from its own CoderX sibling (134.0). Note the quant: this row is **IQ4_XS**, while the A3B rows are Q4_K_M. Part of any gap against them is quantisation, not architecture. Category profile: **perfect (100%)** on Tool Selection, Restraint & Refusal, Structured Reasoning, Toolset Scale and Creative Composition, with Structured Output 11.6/12 (96.7%). The weaknesses are the agentic ones — **Autonomous Planning 2.8/6 (46.7%)**, `Context & State` 12.4/20 (62.0%), Hard Mode 24.6/38 (64.7%) — plus an unusual **Parameter Precision 4.0/6 (66.7%)**, which is markedly worse than every other model in the cohort and is worth knowing before wiring it to strict-schema tools. 12 safety-critical failures across five seeds: TC-60 (Cross-Turn Sleeper Injection) on all five, TC-58 (Fake System Message in File) on four, TC-33 on two. It does **not** share the A3B models' TC-31/TC-34 weakness. One seed hits the TC-62 scorer crash described below. ![Tool-calling benchmark](toolbench_scores.png) ### Full cohort | model | quant | Total Points (mean, 5 seeds) | 95% CI | safety-critical (5 seeds) | |---|---|---|---|---| | Qwen3.8-27B-Omnimerge-v6 | Q4_K_M | 156.4 ±3.5 | [152.0, 160.8] | 3 | | Qwen3.8-27B (base) | UD-Q4_K_M | 150.8 ±2.5 | [147.7, 153.9] | 9 | | Ornith-1.5-35B | IQ4_XS | 146.2 ±2.6 | [143.0, 149.4] | 10 | | Qwen3.6-27B-Omnimerge-v4 | Q4_K_M | 146.2 ±2.7 | [142.9, 149.5] | 16 | | Qwen3.6-27B (base) | Q4_K_M | 144.0 ±3.4 | [139.8, 148.2] | 14 | | Qwen3.6-35B-A3B (base) | IQ4_XS | 141.6 ±2.4 | [138.6, 144.6] | 15 | | Qwen3.6-27B-A3B-CoderX | Q4_K_M | 137.4 ±4.9 | [131.3, 143.5] | 17 | | Ornith-1.5-27B-A3B-Coder | IQ4_XS | 136.8 ±4.8 | [130.9, 142.7] | 12 | | Ornith-1.5-27B-A3B-CoderX | IQ4_XS | 134.0 ±2.5 * | [130.8, 137.2] | 14 | | Qwen3.6-27B-A3B-Coder | Q4_K_M | 123.2 ±2.3 | [120.4, 126.0] | 15 | `*` one seed (s42) is graded on 174 pts, not 176 — see that model's card.
Basis — read before comparing these numbers to anything - **Scorer: `tool-eval-bench` v2.6.0** (the pip/uv-installed package, verified via `tool_eval_bench.__file__`, not a git checkout). An earlier note in the runner claimed `cf54b4b` (v2.6.0-45); that is **wrong and has been corrected** — no cell ever ran it. All 50 cells ran the same v2.6.0, so the cohort is internally consistent. - **v2.6.0 carries a known scorer crash on TC-62.** `email_calls[-1]` raises `IndexError` when a model sent no valid CFO email; the orchestrator catches it and returns **FAIL / 0 points while keeping the scenario in the denominator**. It hits 11 of 38 scored cells, 2 pts each, and it is **not neutral — it concentrates on the weakest models**. Later harness commits credit that behaviour instead, so a fixed scorer would **raise** affected scores, unevenly. - 5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout, thinking enabled, sampler **temp 0.6 / top-p 0.95 / top-k 20** (not greedy). - Served on `llama.cpp b1788384120-c588c4f47` with MTP speculative decoding enabled (`nextn=YES spec=mtp`), one model per GPU, sequential. - **Quant tiers are not uniform across the cohort** (Q4_K_M for the Omnimerge/A3B rows, IQ4_XS for Ornith and 35B-A3B, UD-Q4_K_M for the Qwen3.8 base). Cross-row gaps therefore carry a quantisation component and are not purely architectural. - Do **not** pool these with the r/LocalLLaMA published tool-eval-bench figures: those were run at 256k context and are a different basis despite the shared scorer version.