Instructions to use Myric/KAT-Coder-V2.5-Dev-MTP-head with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Myric/KAT-Coder-V2.5-Dev-MTP-head
# Run inference directly in the terminal:
llama cli -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Myric/KAT-Coder-V2.5-Dev-MTP-head
# Run inference directly in the terminal:
llama cli -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Myric/KAT-Coder-V2.5-Dev-MTP-head
# Run inference directly in the terminal:
./llama-cli -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Myric/KAT-Coder-V2.5-Dev-MTP-head
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Use Docker

docker model run hf.co/Myric/KAT-Coder-V2.5-Dev-MTP-head

LM Studio
Jan
Ollama
How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with Ollama:
```
ollama run hf.co/Myric/KAT-Coder-V2.5-Dev-MTP-head
```

Unsloth Studio

How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Myric/KAT-Coder-V2.5-Dev-MTP-head to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Myric/KAT-Coder-V2.5-Dev-MTP-head to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Myric/KAT-Coder-V2.5-Dev-MTP-head to start chatting

How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Myric/KAT-Coder-V2.5-Dev-MTP-head"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Myric/KAT-Coder-V2.5-Dev-MTP-head" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with Docker Model Runner:
```
docker model run hf.co/Myric/KAT-Coder-V2.5-Dev-MTP-head
```

Lemonade

How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Myric/KAT-Coder-V2.5-Dev-MTP-head

Run and chat with the model

lemonade run user.KAT-Coder-V2.5-Dev-MTP-head-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use Myric/KAT-Coder-V2.5-Dev-MTP-head with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Myric/KAT-Coder-V2.5-Dev-MTP-head

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Myric/KAT-Coder-V2.5-Dev-MTP-head

Run Hermes

hermes

Atomic Chat

KAT-Coder-V2.5-Dev-MTP-head / README.md

Myric

Upload README.md with huggingface_hub

2f91c61 verified 3 days ago

5.26 kB

license: apache-2.0
base_model: Kwaipilot/KAT-Coder-V2.5-Dev
tags:
  - mtp
  - speculative-decoding
  - qwen3_5_moe
  - experiment

KAT-Coder-V2.5-Dev — MTP head (experimental, un-fine-tuned baseline)

Kwaipilot/KAT-Coder-V2.5-Dev ships with mtp_num_hidden_layers: 0 — no MTP/NextN head at all, so no speculative-decoding speedup is possible out of the box. This repo adds one back by transplanting Qwen/Qwen3.6-35B-A3B's own trained MTP head onto KAT-Coder. This is architecturally clean, not a hack: every relevant dimension (hidden_size, expert count, attention/MoE shapes) matches exactly between the two models, since KAT-Coder is itself built on the Qwen3.6-35B-A3B backbone.

This is the naive, un-fine-tuned transplant — the MTP head's weights are copied directly from stock Qwen3.6 with zero training on KAT-Coder's own hidden-state distribution. A fine-tuning pass (freezing KAT-Coder's own 40 layers entirely, training only this ~850M-parameter head) is a planned follow-up; this repo documents the honest baseline before that.

What's actually in this repo

Just one new shard (model-00014-of-mtp.safetensors, 1.69GB) containing the 19 MTP-related tensors, plus a config.json/model.safetensors.index.json already merged to reference it alongside KAT-Coder's own 13 original shards. Nothing else was touched — KAT-Coder's own 13 shards are byte-identical to the original release; this is a drop-in addition, not a full re-upload of the 65GB checkpoint.

To use: put this repo's 3 files into a local copy of Kwaipilot/KAT-Coder-V2.5-Dev alongside its own 13 model-*-of-00013.safetensors shards (the config.json/index here already reference all 14 shards together).

Baseline: draft-acceptance rate (before fine-tuning)

Measured via llama-server --spec-type draft-mtp --spec-draft-n-max 2 on an APEX i-quality quant (4.67 bpw) built from this checkpoint, 5 real from-scratch coding prompts, n_predict=400 each:

prompt	draft tokens	accepted	rate	tok/s
binary search tree (insert/delete/traverse)	286	255	89.2%	105.5
CSV parser (stdlib only)	114	84	73.7%	94.6
retry decorator w/ exponential backoff	188	152	80.9%	100.6
merge two sorted linked lists	74	58	78.4%	97.6
thread-safe LRU cache	360	218	60.6%	85.9
overall	1022	767	75.1%	—

For reference, the same KAT-Coder quant tier without any MTP head runs at ~70.8 tok/s solo decode speed (measured separately, no speculative decoding available at all). So even this un-fine-tuned transplant is already a real ~20-40% speedup, not a wash — acceptance rate varies by task in a way that tracks predictability (the BST task, mostly standard tree-traversal boilerplate, hits 89%; the LRU cache, with genuine locking/eviction logic, is the hardest at 61%).

Full coding-benchmark validation (real agentic tasks, not just completion speed)

Ran the same two task suites used elsewhere in this project's Qwen3.6-lineage work (an existing 5-task hard tier, plus a new 3-task tier designed to require genuine algorithmic reasoning rather than spec-transcription) against this MTP-enabled build vs the same quant without MTP, via opencode's real agentic tool-calling loop (not just raw completion). Correctness was identical — 100% on both suites, both with and without MTP — exactly what the theory predicts: the MTP head only drafts candidate tokens, the main model always verifies against its own true computation, so a bad draft just gets rejected and never reaches the output. Speed differed substantially:

hard-tier task	no MTP	with MTP	speedup
batch_scheduler	22.5s	20.8s	1.08x
buddy_allocator	123.5s	63.5s	1.94x
circuit_breaker	50.7s	14.0s	3.62x
expr_parser_advanced	54.7s	21.1s	2.59x
inventory_fix_and_extend	16.4s	12.3s	1.33x
total	267.8s	131.7s	2.03x

tier-4 task	no MTP	with MTP	speedup
lru_ttl_cache	20.5s	25.2s	0.81x
rate_limiter_fix_and_extend	19.7s	12.1s	1.63x
weighted_interval_scheduling	22.5s	17.4s	1.29x
total	62.7s	54.7s	1.15x

Both suites hit 100% (59/59 and 28/28 test cases respectively) in both configurations. The one slower result (lru_ttl_cache, 0.81x) is most likely ordinary run-to-run variance in how much iteration that specific attempt needed, not a systematic MTP cost — every other task improved, and the hard-tier set as a whole is a genuine 2x.

Attribution

KAT-Coder base model: Kwaipilot — Kwaipilot/KAT-Coder-V2.5-Dev (Apache-2.0).
MTP head weights: Qwen — Qwen/Qwen3.6-35B-A3B (Apache-2.0), transplanted unmodified.
Quantization/serving: llama.cpp (ggml-org).

Unofficial community experiment; not affiliated with or endorsed by Kwaipilot or Qwen.