---
base_model: moonshotai/Kimi-K3
license: other
license_name: kimi-k3
license_link: https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE
language:
- ja
tags:
- gguf
- kimi-k3
- moe
- reap
---

# Kimi-K3 REAP50 Width50 UD GGUF

[日本語](./README-jp.md)

![Kimi-K3 REAP50 and Width50 overview](./reduction-overview.png)

This is an experimental, reduced GGUF build of [Kimi-K3](https://huggingface.co/moonshotai/Kimi-K3).

## Calibration and selection

Prepared Japanese responses for chat, code generation, reasoning, and tool use were tokenized and evaluated with a forward pass of the original Kimi-K3. Only semantic assistant tokens were scored; prompts and control or structure tokens were excluded.

### REAP50: expert axis

For expert `e`, the score is:

```text
REAP(e) = mean[t routed to e](top-16-renormalized router weight × ||unweighted expert output||₂)
```

Experts are ranked separately in each of the 92 MoE layers. The top 448 of 896 are retained. No experts are reserved by hand.

### Width50: intermediate axis

For every calibration token `t` routed to expert `e`, Width50 follows the actual expert MLP:

```text
a(t,e)     = SiTU(Wgate,e × h(t), Wup,e × h(t))
y(t,e,b)   = router_weight(t,e) × Wdown,e[:,b] × a(t,e)[b]
score(e,b) = Σ[t routed to e] ||y(t,e,b)||₂²
```

Here, `b` is one 32-channel slice of the 3072-channel intermediate activation. Each slice is passed through only the matching columns of `Wdown`, producing a hidden-size output vector. Its router-weighted squared L2 norm is accumulated over semantic assistant tokens.

Eight adjacent 32-channel scores are summed into one physical QK256 score. The highest 6 of 12 QK256 blocks are retained independently for each expert. Low-coverage experts blend this activation score with a weight-based prior. The width map is measured on the original expert IDs and remapped through the REAP50 keep list before GGUF slicing.

Blocks are scored independently; cross-block cancellation and the post-mixture RMSNorm are not part of this ranking.

## GGUF build flow

```text
Kimi-K3 weights
  → Quantization
  → Q1 / Q2 GGUF
  → REAP50 expert-axis slice
  → Width50 QK256-block slice
  → final split GGUF
```

The REAP50 step slices the expert axis of `ffn_gate_inp.weight`, `exp_probs_b.bias`, and `ffn_{gate,up,down}_exps.weight` from 896 to 448 for every MoE layer.

The Width50 step slices the intermediate axis of `ffn_gate_exps.weight`, `ffn_up_exps.weight`, and `ffn_down_exps.weight` from 3072 to 1536. Complete quantization blocks are copied directly, so retained data is not dequantized or requantized.

| | Original | This build |
|---|---:|---:|
| Routed experts per MoE layer | 896 | 448 |
| Routed expert FFN width | 3072 | 1536 |
| Experts used per token | 16 | 16 |

All other tensor data is unchanged. The shared expert metadata is represented as `4 × 1536` instead of `2 × 3072` so that its physical width remains 6144 in llama.cpp.

## Files

| Folder | Shards | Size |
|---|---:|---:|
| `UD-IQ1_S` | 14 | about 181 GiB |
| `UD-IQ1_M` | 15 | about 194 GiB |
| `UD-Q2_K_XL` | 19 | about 243 GiB |

Load the first shard in the selected folder.

## Usage

Use the [Kimi-K3 Width support branch](https://github.com/mmnga/llama.cpp/tree/kimi-k3-width-support) of llama.cpp.

```bash
git clone --branch kimi-k3-width-support https://github.com/mmnga/llama.cpp
cmake -S llama.cpp -B llama.cpp/build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build llama.cpp/build -j --target llama-server
```

Example using `UD-IQ1_S`:

```bash
./llama.cpp/build/bin/llama-server \
  -m ./UD-IQ1_S/Kimi-K3-UD-IQ1_S-00001-of-00014.gguf \
  -ot ".*.ffn_.*_exps.*=CPU" \
  -ngl 45 \
  --ctx-size 8192 \
  --flash-attn on \
  --jinja \
  --override-kv kimi-k3.expert_shared_count=int:2 \
  --override-kv kimi-k3.expert_shared_feed_forward_length=int:6144 \
  --override-kv kimi-k3.expert_used_count=int:16
```

The example was tested with a 32 GB RTX 5090 while keeping routed experts on the CPU. Adjust `-ngl` and context size for your available VRAM and RAM. Load the first shard when using another quantization folder.

This is a heavily reduced experimental model. Quality and stability may differ from the original Kimi-K3.