jakeroxs/KAT-Coder-V2.5-Dev-35B-A3B-MTP-ABLITERATED-GGUF

KAT-Coder V2.5 Dev 35B-A3B MTP Abliterated

KAT-Coder V2.5 Dev 35B-A3B with a transplanted Multi-Token Prediction (MTP) head, based on the abliteration / "Philadelphia Class" variant of KAT-Coder V2.5 Dev.

This repository contains:

• The merged Hugging Face-format checkpoint
• A BF16 GGUF
• A Q4_K_M GGUF
• A transplanted single-layer MTP head for llama.cpp draft-mtp speculative decoding

Model lineage

Original model

KwaiPilot/KAT-Coder-V2.5-Dev

Abliterated base

KridgeDookie/KAT-Coder-V2.5-Dev-35B-A3B-ABLITERATED-UNCENSORED-PHILADELPHIA-CLASS

MTP head

Myric/KAT-Coder-V2.5-Dev-MTP-head

The Philadelphia Class checkpoint is used as the target/base model, with the MTP tensors from Myric's KAT-Coder V2.5 Dev MTP head transplanted into it.

MTP modification

The source Philadelphia Class model has:

mtp_num_hidden_layers = 0

This merged checkpoint changes it to:

mtp_num_hidden_layers = 1

and adds the 19 MTP tensors from the MTP-head checkpoint.

The resulting Hugging Face weight layout is:

File	Tensor count
model.safetensors	693
model-00014-of-mtp.safetensors	19
Total	712

GGUF files

• KAT-Philly-MTP-BF16.gguf
• KAT-Philly-MTP-Q4_K_M.gguf

The GGUF conversion exposes the transplanted MTP layer as the additional blk.40 / nextn tensors expected by llama.cpp.

llama.cpp usage

A recent llama.cpp build with Qwen3.5 MoE MTP support is required.

Example:

llama-server \
  -m KAT-Philly-MTP-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --parallel 1 \
  --reasoning-preserve \
  --fit-ctx 262144 \
  --fit on \
  --flash-attn on \
  --cache-type-k q4_0 \
  --cache-type-v q4_0 \
  --cache-type-k-draft q4_0 \
  --cache-type-v-draft q4_0 \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

Validation

The model has been successfully tested with llama.cpp draft-mtp.

The converted model reports:

architecture:          qwen35moe
model type:            35B.A3B
parameters:            35.51 B
training context:      262144
experts:               256
experts used/token:    8
nextn predict layers:  1

Observed first-position MTP acceptance was generally around 70-76%. Second-position acceptance with --spec-draft-n-max 2 was generally around 48-55%.

Performance note

Testing on:

GPU: NVIDIA GeForce RTX 3080 Ti, 12 GB
CPU: AMD Ryzen 7 3700X
RAM: 48 GB

showed that MTP worked correctly, but the additional MTP working set reduced the amount of the MoE model that could remain resident in VRAM.

Representative results:

Configuration	Prompt processing	Sustained generation
Non-MTP KAT Q4_K_M	~537 t/s	~38.4 t/s
MTP Q4_K_M, n_max=2	~350-465 t/s	~29-30 t/s
MTP Q4_K_M, n_max=1	~411 t/s	~26.8 t/s

Systems with substantially more VRAM may behave differently.

Credits

• Kwaipilot — KAT-Coder V2.5 Dev
• KridgeDookie — Abliterated / Uncensored Philadelphia Class variant
• Myric — KAT-Coder V2.5 Dev MTP head
• llama.cpp contributors — GGUF conversion, quantization, and MTP inference support