---
license: apache-2.0
base_model:
- Kwaipilot/KAT-Coder-V2.5-Dev
- Ttimms/KAT-Coder-V2.5-Dev-REAP-50-bf16
base_model_relation: quantized
tags:
- gguf
- llama.cpp
- reap
- expert-pruning
- moe
- qwen3.5
- code
- agentic-coding
- text-generation
language:
- en
pipeline_tag: text-generation
quantized_by: Ttimms
license_link: https://huggingface.co/Kwaipilot/KAT-Coder-V2.5-Dev/blob/main/LICENSE
---

# KAT-Coder-V2.5-Dev · REAP-50 — GGUF

GGUF quants of the **50%-REAP-pruned** KAT-Coder-V2.5-Dev (Gated-DeltaNet + MoE
hybrid): 256 → 128 experts, `~19B` parameters, no MTP head.

- **Flagship (NVFP4A16, vLLM):** [`Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16`](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16)
- **Pruned bf16 source:** [`Ttimms/KAT-Coder-V2.5-Dev-REAP-50-bf16`](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-bf16)
- **Pipeline + benchmarks:** <https://github.com/t-timms/kat-coder-nvfp4>

Base-model quality (NVFP4A16 quant, greedy, instruct): HumanEval+ ~90%, MBPP+ ~90%. These are single greedy draws; the accuracy harness has a measured run-to-run spread of 1.85–4.27 pp per task (2026-09-05), so read small differences as unresolved.

## Architecture

```mermaid
graph TD
    Base["Kwaipilot/KAT-Coder-V2.5-Dev<br/>Qwen3.5 MoE - 256 experts - ~69 GB bf16"]
    subgraph Build ["Build - RTX 5070 Ti, SM120"]
        REAP["REAP expert prune 50%  256 -> 128 experts + router-renorm fix"]
        Strip["strip vision tower + 333 untrained tensors"]
        Quant["NVFP4A16 quantize (weight-only, data-free, 82 s)"]
    end
    subgraph HF ["Published formats"]
        A16["REAP-50-NVFP4A16 - 12.45 GiB (default, vLLM)"]
        W4A4["REAP-50-NVFP4-W4A4 (native FP4)"]
        GPTQ["REAP-50-NVFP4A16-GPTQ (null result, kept for transparency)"]
        GGUF["REAP-50-GGUF (Q4_K_M / Q5_K_M / Q6_K / Q8_0)"]
        BF16["REAP-50-bf16 (pruned source)"]
    end
    Bench["A16 single draws - HumanEval+ 89.0% - MBPP+ 90.5% - SWE-bench Verified 52.0%"]
    Base --> REAP --> Strip --> Quant --> A16
    Strip --> BF16
    BF16 -. re-quant .-> W4A4
    BF16 -. re-quant .-> GPTQ
    BF16 -. convert .-> GGUF
    A16 --> Bench
```

## Download

Grab **one file**, not the whole repo.

| File | Quant | Size | Notes |
|---|---|--:|---|
| [`kat-reap50-Q4_K_M.gguf`](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-GGUF/blob/main/kat-reap50-Q4_K_M.gguf) | Q4_K_M | 11.4 GB | **recommended** — fits 16 GB with room for context |
| [`kat-reap50-Q5_K_M.gguf`](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-GGUF/blob/main/kat-reap50-Q5_K_M.gguf) | Q5_K_M | 13.3 GB | higher quality, tighter KV budget |
| [`kat-reap50-Q6_K.gguf`](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-GGUF/blob/main/kat-reap50-Q6_K.gguf) | Q6_K | 15.3 GB | near-lossless; little headroom on 16 GB |
| [`kat-reap50-Q8_0.gguf`](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-GGUF/blob/main/kat-reap50-Q8_0.gguf) | Q8_0 | 19.7 GB | needs >16 GB or CPU offload |

MoE with ~3B active parameters — CPU / Apple-Silicon inference is fast for the size.
For full precision, re-quantizing, or merging, use the
[pruned bf16 source](https://huggingface.co/Ttimms/KAT-Coder-V2.5-Dev-REAP-50-bf16)
and run `convert_hf_to_gguf.py --no-mtp` yourself.

## Run it in

- [llama.cpp](https://github.com/ggml-org/llama.cpp) — `llama-server -m <file>.gguf -ngl 99`
- [LM Studio](https://lmstudio.ai/) (bundles a recent llama.cpp)
- [koboldcpp](https://github.com/LostRuins/koboldcpp)
- [Jan](https://jan.ai/)
- [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
- Ollama — once its bundled llama.cpp includes this arch (see Requirements)

## Requirements

Needs a **recent llama.cpp** (the `qwen35moe` hybrid arch). Verified with
`llama-server` from a source build. Ollama support pending its bundled-llama.cpp bump.

```bash
llama-server -m kat-reap50-Q4_K_M.gguf -ngl 99 --port 8080
```

The chat template defaults to thinking on; pass
`"chat_template_kwargs": {"enable_thinking": false}` for direct output.

## License

Apache-2.0 (inherits from `Kwaipilot/KAT-Coder-V2.5-Dev`). Pruning: REAP
(github.com/CerebrasResearch/reap, with a router-renormalization fix).