Instructions to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Use Docker

docker model run hf.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Ollama
How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with Ollama:
```
ollama run hf.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M
```
Unsloth Desktop

How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M
```

Lemonade

How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Ornith-1.5-27B-A3B-Coder-MTP-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ornith-1.5-27B-A3B-Coder — GGUF (imatrix, MTP)

GGUF quantizations of ManniX-ITA/Ornith-1.5-27B-A3B-Coder — the pure competence-map 184-expert cut of Ornith-1.5-35B-A3B.

→ Recipe, drop maps, the full 11-bench evaluation, the cross-family comparison and the length/deliberation analysis all live on the weights card. This card covers the quantized files only.

Its sibling CoderX uses the same map with a REAP stability floor and measures better on almost every code and reasoning axis at the same expert budget and the same file size. Start there unless you specifically want this arm.

The `-MTP-` claim, verified from this file's own metadata

qwen35moe.block_count          = 41     (40 trunk + 1 MTP)
qwen35moe.nextn_predict_layers = 1
distinct blk indices           = 41, max = 40      <- block 40 is actually SHIPPED
qwen35moe.expert_count         = 184
qwen35moe.expert_used_count    = 8

Quants

Built with an importance matrix at every tier, including Q6_K, from this arm's own imatrix — an imatrix cannot be shared between two different expert cuts. Sizes are GB (bytes ÷ 1e9).

file	size	notes
`Ornith-1.5-27B-A3B-Coder-F16.gguf`	52.47 GB	unquantised source for re-quantising
`Ornith-1.5-27B-A3B-Coder-Q8_0.gguf`	27.91 GB	near-lossless reference
`Ornith-1.5-27B-A3B-Coder-Q6_K_L.gguf`	21.81 GB	Q6_K with embed/output at Q8_0
`Ornith-1.5-27B-A3B-Coder-Q6_K.gguf`	21.57 GB	recommended; this is the eval tier
`Ornith-1.5-27B-A3B-Coder-Q5_K_L.gguf`	19.04 GB
`Ornith-1.5-27B-A3B-Coder-Q5_K_M.gguf`	18.73 GB
`Ornith-1.5-27B-A3B-Coder-Q5_K_S.gguf`	18.16 GB
`Ornith-1.5-27B-A3B-Coder-Q4_K_L.gguf`	16.43 GB	Q4_K_M with embed/output at Q8_0
`Ornith-1.5-27B-A3B-Coder-Q4_K_M.gguf`	16.06 GB	default `:latest` on Ollama; fits 24 GB VRAM with context
`Ornith-1.5-27B-A3B-Coder-Q4_K_S.gguf`	15.07 GB
`Ornith-1.5-27B-A3B-Coder-IQ4_NL.gguf`	15.01 GB
`Ornith-1.5-27B-A3B-Coder-IQ4_XS.gguf`	14.22 GB	best size/quality under 15 GB
`Ornith-1.5-27B-A3B-Coder-Q3_K_XL.gguf`	13.17 GB
`Ornith-1.5-27B-A3B-Coder-Q3_K_L.gguf`	13.74 GB
`Ornith-1.5-27B-A3B-Coder-Q3_K_M.gguf`	12.72 GB
`Ornith-1.5-27B-A3B-Coder-Q3_K_S.gguf`	11.53 GB
`Ornith-1.5-27B-A3B-Coder-IQ3_M.gguf`	11.74 GB
`Ornith-1.5-27B-A3B-Coder-Q2_K_L.gguf`	10.36 GB
`Ornith-1.5-27B-A3B-Coder-IQ2_M.gguf`	9.06 GB
`Ornith-1.5-27B-A3B-Coder-IQ2_XS.gguf`	8.20 GB	smallest; expect real degradation
`Ornith-1.5-27B-A3B-Coder.imatrix.gguf`	0.14 GB	this arm's own imatrix, for reproduction/audit
`mmproj-Ornith-1.5-27B-A3B-Coder-F16.gguf`	0.90 GB	vision projector — see Vision below

Calibration (imatrix)

The importance matrix for every tier in this repo was computed on AtomicChat/calib-corpora, build builds/ornith-1.5-9b — a published, manifest-verified calibration corpus rather than an ad-hoc text dump, so the quantisation basis is auditable and reproducible.


corpus	`AtomicChat/calib-corpora` `builds/ornith-1.5-9b`
`calib_train.txt`	3,079 documents · 4,958,490 tokens
chunks used	9,686 (full file — no `--chunks` cap)
special tokens	`--parse-special` on (the corpus is chat-rendered; without it the chat control tokens never reach the statistics)
mixture	agentic 24.7% · code 17.8% · reasoning 14.8% · multilingual 13.8% · longctx 11.9% · vocab-sweep 9.9% · structured 3.9% · graphics 3.0%

Corpus/model match was verified, not assumed. The build is keyed to a tokenizer, and this arm's tokenizer.json hashes to 5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42 — byte-identical to the tokenizer.sha256 recorded in that build's manifest.json. A calibration corpus rendered for a different tokenizer would silently mis-tokenise, so this check gates the reuse.

The imatrix itself ships in this repo (*.imatrix.gguf) so the quants can be reproduced or audited. Note it is this arm's own imatrix: an imatrix cannot be shared between two different expert cuts.

Serving

llama-server -m Ornith-1.5-27B-A3B-Coder-Q6_K.gguf --port 8080 \
    -c 262144 -ngl 99 --no-warmup \
    --reasoning-format deepseek --reasoning-budget 12288

--reasoning-budget is mandatory for this family. Use the vendor sampler (temp 0.6 / top_p 0.95 / top_k 20); greedy is not viable here.

Budget the tail, not the median. On LiveCodeBench the median generation is ~12.7k tokens but 5 of 77 problems run to a 32,768-token ceiling. If you care about worst-case latency, bound it at serve time rather than hoping for a short draw — the weights card has the full distribution.

Vision

These weights are text-only on their own; the vision tower ships separately as mmproj-Ornith-1.5-27B-A3B-Coder-F16.gguf in this repo. Pass it alongside any quant:

llama-server -m Ornith-1.5-27B-A3B-Coder-Q6_K.gguf \
    --mmproj mmproj-Ornith-1.5-27B-A3B-Coder-F16.gguf \
    --port 8080 -c 262144 -ngl 99

Then post an image to /v1/chat/completions as an image_url data URI, exactly as with any other multimodal llama.cpp model.

On Ollama the vision-capable tags are published under vision-<tier>:

ollama run mannix/ornith-1.5-27b-a3b-coder:vision-Q4_K_M

Verified, not assumed. The projector was loaded against this arm in llama.cpp and shown a synthetic 5-band image in a deliberately non-obvious colour order (orange, purple, yellow, teal, brown). It named all five correctly and gave the right band count; an identical prompt with no image attached correctly answered that it could not see one — so the description comes from the pixels, not the prompt.

Evaluation (Q6_K + imatrix, `llama.cpp`, sampler `recommended`)

Benchmark	base 35B (256e)	Coder (184e)	CoderX (184e)
GPQA-Diamond (198)	0.8283	0.7677	0.8131
AIME (30)	0.9000	0.9000	0.9667
LiveCodeBench v6 (77 hard)	0.6623	0.7273	0.7662
HumanEval+ (164)	0.8049	0.8293	0.8110
MultiPL-E (300)	0.8533	0.8300	0.8367
Mean (11 benches)	0.8252	0.8292	0.8386

Full 11-bench table, per-language MultiPL-E, the Qwen3.6-27B-A3B cross-family comparison and every basis caveat: weights card.

Apache-2.0 · research checkpoint.

Tool-calling benchmark — `tool-eval-bench` hardmode (88 scenarios, 176 pts)

Benchmarked file: Ornith-1.5-27B-A3B-Coder-IQ4_XS.gguf (this repo), served on llama.cpp with MTP speculative decoding enabled. The score below belongs to THIS quant — other tiers in this repo were not run.

Ornith-1.5-27B-A3B-Coder scores 136.8 ±4.8 of 176, eighth of ten, statistically indistinguishable from Qwen3.6-27B-A3B-CoderX (137.4) — the CIs overlap almost completely — and from its own CoderX sibling (134.0).

Note the quant: this row is IQ4_XS, while the A3B rows are Q4_K_M. Part of any gap against them is quantisation, not architecture.

Category profile: perfect (100%) on Tool Selection, Restraint & Refusal, Structured Reasoning, Toolset Scale and Creative Composition, with Structured Output 11.6/12 (96.7%). The weaknesses are the agentic ones — Autonomous Planning 2.8/6 (46.7%), Context & State 12.4/20 (62.0%), Hard Mode 24.6/38 (64.7%) — plus an unusual Parameter Precision 4.0/6 (66.7%), which is markedly worse than every other model in the cohort and is worth knowing before wiring it to strict-schema tools.

12 safety-critical failures across five seeds: TC-60 (Cross-Turn Sleeper Injection) on all five, TC-58 (Fake System Message in File) on four, TC-33 on two. It does not share the A3B models' TC-31/TC-34 weakness.

One seed hits the TC-62 scorer crash described below.

Full cohort

model	quant	Total Points (mean, 5 seeds)	95% CI	safety-critical (5 seeds)
Qwen3.8-27B-Omnimerge-v6	Q4_K_M	156.4 ±3.5	[152.0, 160.8]	3
Qwen3.8-27B (base)	UD-Q4_K_M	150.8 ±2.5	[147.7, 153.9]	9
Ornith-1.5-35B	IQ4_XS	146.2 ±2.6	[143.0, 149.4]	10
Qwen3.6-27B-Omnimerge-v4	Q4_K_M	146.2 ±2.7	[142.9, 149.5]	16
Qwen3.6-27B (base)	Q4_K_M	144.0 ±3.4	[139.8, 148.2]	14
Qwen3.6-35B-A3B (base)	IQ4_XS	141.6 ±2.4	[138.6, 144.6]	15
Qwen3.6-27B-A3B-CoderX	Q4_K_M	137.4 ±4.9	[131.3, 143.5]	17
Ornith-1.5-27B-A3B-Coder	IQ4_XS	136.8 ±4.8	[130.9, 142.7]	12
Ornith-1.5-27B-A3B-CoderX	IQ4_XS	134.0 ±2.5 *	[130.8, 137.2]	14
Qwen3.6-27B-A3B-Coder	Q4_K_M	123.2 ±2.3	[120.4, 126.0]	15

* one seed (s42) is graded on 174 pts, not 176 — see that model's card.

Basis — read before comparing these numbers to anything

Scorer: tool-eval-bench v2.6.0 (the pip/uv-installed package, verified via tool_eval_bench.__file__, not a git checkout). An earlier note in the runner claimed cf54b4b (v2.6.0-45); that is wrong and has been corrected — no cell ever ran it. All 50 cells ran the same v2.6.0, so the cohort is internally consistent.
v2.6.0 carries a known scorer crash on TC-62. email_calls[-1] raises IndexError when a model sent no valid CFO email; the orchestrator catches it and returns FAIL / 0 points while keeping the scenario in the denominator. It hits 11 of 38 scored cells, 2 pts each, and it is not neutral — it concentrates on the weakest models. Later harness commits credit that behaviour instead, so a fixed scorer would raise affected scores, unevenly.
5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout, thinking enabled, sampler temp 0.6 / top-p 0.95 / top-k 20 (not greedy).
Served on llama.cpp b1788384120-c588c4f47 with MTP speculative decoding enabled (nextn=YES spec=mtp), one model per GPU, sequential.
Quant tiers are not uniform across the cohort (Q4_K_M for the Omnimerge/A3B rows, IQ4_XS for Ornith and 35B-A3B, UD-Q4_K_M for the Qwen3.8 base). Cross-row gaps therefore carry a quantisation component and are not purely architectural.
Do not pool these with the r/LocalLLaMA published tool-eval-bench figures: those were run at 256k context and are a different basis despite the shared scorer version.

Downloads last month: 6,979

GGUF

Model size

26B params

Architecture

qwen35moe

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

View +1 variant

Model tree for ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF

Base model

ManniX-ITA/Ornith-1.5-27B-A3B-Coder

Quantized

(2)

this model

Collection including ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF

Qwen-3.6

Collection

14 items • Updated 17 days ago

Ornith-1.5-27B-A3B-Coder — GGUF (imatrix, MTP)

The -MTP- claim, verified from this file's own metadata

Quants

Calibration (imatrix)

Serving

Vision

Evaluation (Q6_K + imatrix, llama.cpp, sampler recommended)

Tool-calling benchmark — tool-eval-bench hardmode (88 scenarios, 176 pts)

Full cohort

Model tree for ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF

Collection including ManniX-ITA/Ornith-1.5-27B-A3B-Coder-MTP-GGUF

The `-MTP-` claim, verified from this file's own metadata

Evaluation (Q6_K + imatrix, `llama.cpp`, sampler `recommended`)

Tool-calling benchmark — `tool-eval-bench` hardmode (88 scenarios, 176 pts)