Instructions to use Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4", trust_remote_code=True)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4

SGLang

How to use Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4 with Docker Model Runner:
```
docker model run hf.co/Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4
```

KIMI-K3-CODER-REAP-320-MXFP4

Coding-calibrated expert prune of Kimi K3 · 320/896 experts · native MXFP4

Built by Blackfrost · Las Vegas, NV

As far as we know, the first Kimi K3 running on a single 8× RTX 6000 node.

⚠️ EXPERIMENTAL

Bugs have been observed — decode, coherence, and serve edge cases under load. Expect sharp edges. This is not production-polished.

Please report them. Open a Community discussion with GPU SKU, driver, SGLang image/version, full serve flags, prompt, temperature, max_tokens, and failure mode (load-time / empty content / loop / tool parse / OOM). Fixes go faster with real repros.

Why this model exists

Full Kimi K3 is a multi-node, multi-terabyte deploy. That puts it out of reach of every team that has one good box and no fabric.

KIMI-K3-CODER-REAP-320 is the same LatentMoE architecture, expert-pruned against measured coding routing load until it fits on 8× RTX 6000 (96 GB class), single node. Not a distill, not a different architecture, not a BF16 remaster — the surviving expert tensors are bit-exact copies of the parent MXFP4 packs.

The single-node footprint is the product. It is free and open — no licence fee, no commercial arrangement, no access request. Download it and tell us what breaks.

Read this before you decide: it is a coding specialist

We pruned on coding routing load, and that is exactly what survived — and what didn't. Early measurements on a REAP-320 sibling show coding perplexity essentially unchanged against the parent while encyclopedic-prose perplexity degrades by more than an order of magnitude — the expected shape of a code-calibrated 64% expert cut.

Numbers for this checkpoint are being measured now and will be published here, rather than borrowed from a sibling build. Use this for coding on one box. If you need general knowledge or strong bilingual capability, use the parent — that is what it is for.

Specifications


Architecture	Kimi K3 LatentMoE + KDA
Base	`moonshotai/Kimi-K3` — official
Transform	REAP only — expert prune + router slice
Experts kept	320 of 896 routed per MoE layer (a 64.3% cut), uniform across layers 1–92
Precision	Native MXFP4 / `compressed-tensors` layout — no re-quantization of kept packs
On-disk	~588 GB, 15 safetensors shards
Context	131,072 declared — see the KV note under Deployment
Serve shape	1× node · 8× RTX 6000 · TP 8 · `marlin` MoE backend
Status	EXPERIMENTAL

Deployment scripts ship in deployment-kit/.

What "CODER" means

Not a separate coding SFT or DPO with an undisclosed recipe. No post-training was applied at all.

"Coder" means coding-calibrated expert survival under a fixed expert budget:

Routing measured under coding load — coding prompts and traces
Keep the top 320 / 896 experts per MoE layer by selection frequency
Routers sliced to the survivor index set, num_experts → 320
Survivor expert tensors copied bit-exact from the parent MXFP4 packs

REAP-320 denotes a uniform keep across MoE layers. It is not non-uniform (NU) pruning — memory and routing behaviour are predictable layer to layer.

Lineage


Base	Official `moonshotai/Kimi-K3`
Applied	REAP expert prune + router slice
Not applied	Refusal-direction or residual-scale ablation · additional SFT/DPO · re-quantization of kept packs
Format	HF safetensors · native MXFP4 / `compressed-tensors`

On refusal behaviour: this checkpoint carries the safety behaviour of upstream Kimi K3, unmodified. Blackfrost also publishes de-risked models with refusal behaviour deliberately removed at the weight level — this is not one of them, and nothing in this repository has been ablated. If you are evaluating Blackfrost models against an internal approval process, this one should be treated as a standard pruned derivative of its parent.

Measured behaviour

Full benchmark retention — HumanEval+, MBPP+, IFEval and others, paired against the parent on an identical harness — is in progress and will be filled in here.

Benchmark	Parent Kimi K3	This (REAP-320)	Retention
pending	—	—	—%

Expert pruning is a capability trade, and a card that claims a 64% expert reduction without publishing what it cost is asking the reader to take the trade on faith. We will state the harness, the conditions, and the retention figures — including any benchmark where retention is poor.

Throughput, measured

On 8× RTX 6000 (SM120), marlin backend, 32k context. Measured on a REAP-320 sibling of identical architecture, shard count and tensor shapes — throughput is determined by those, not by weight values, so these transfer:


Single-stream decode	~24.7 tok/s (TPOT 38.8 ms)
Aggregate, saturated	~114 tok/s (at 8 concurrent)
Prefill	~1,880 tok/s
At 131k context	15.7 tok/s single-stream — 37% slower than 32k

Full K3 vs. this checkpoint

	Full Kimi K3	This (CODER-REAP-320)
Routed experts / MoE layer	896	320
Disk (MXFP4 packs)	~1.5 TB	~588 GB
Practical serve shape	multi-node fabric	1× node · 8× RTX 6000
Expert weights	parent packs	bit-exact survivors, no re-quant
Routing calibration	general	coding load

Deployment notes

⚠️ RTX 6000 / SM120 requires two non-obvious changes

RTX 6000 Blackwell reports compute capability 12.0 (SM120). Two things in the stock K3 serving path assume SM100 (B200) and will fail:

1. MoE backend must be marlin, not flashinfer_mxfp4. flashinfer_mxfp4 raises NotImplementedError: requires SM90 or SM100. deep_gemm is SM100-only as well.

2. SGLang's fused-attention gate must be patched. (mandatory — without it the server dies at CUDA graph capture)

sglang/srt/layers/attn_residual.py gates K3's fused TMA attention on major >= 10. SM120 reports major 12, so it takes the SM100 fast path and JIT-compiles tcgen05 tensor-memory PTX that the card cannot execute — ptxas fails and you get RuntimeError: ninja exited with status 255. One-line fix routes it to SGLang's own triton pipeline:

# sglang/srt/layers/attn_residual.py, in _use_fast()
- _FAST_SUPPORTED = major >= 10
+ _FAST_SUPPORTED = major == 10   # SM120 lacks tcgen05; use the triton path

Extract the file, patch it, and bind-mount it read-only over the container copy. deployment-kit/ does this for you.

Memory and context

--mem-fraction-static 0.94. After marlin repack the weights occupy ~84 GB of each 96 GB card, leaving ~10 GB. The common 0.85 is already under water before KV is sized and will fail with a negative mamba-cache error. Do not lower it — that makes it worse.
KV pool is ~120,370 tokens. The server accepts --context-length 131072, but the pool is smaller than the declared window, so a single request cannot actually fill 131k. 32k is the better working setting and is ~37% faster per token.
Concurrency clamps to 9. Requesting more is silently reduced — the hybrid mamba state cache is the limiter, not KV. This is independent of context length.

Sampling

Follow Moonshot's guidance: temperature 1.0, top_p 0.95 for single-step tasks (1.0 for agentic), reasoning effort max.
max_tokens ≥ 32768. Thinking is always on; the answer arrives in content and the chain of thought in reasoning_content. Small max_tokens returns an empty content with finish_reason: length — that is not a model fault, it is a budget you set too low. At 4096 roughly 2% of completions come back empty; at 16384 we have seen the majority of long generations return reasoning with no answer.
Integrity. Verify shard count and byte totals after download before attributing a load failure to the weights.

Quick serve (SGLang)

hf download Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4 \
  --local-dir ./KIMI-K3-CODER-REAP-320-MXFP4

export MODEL_DIR=$PWD/KIMI-K3-CODER-REAP-320-MXFP4
export HOST=0.0.0.0
export PORT=8000
bash deployment-kit/scripts/patch_sm120.sh      # writes the attn_residual override
bash deployment-kit/scripts/serve_docker.sh
bash deployment-kit/scripts/smoke_chat.sh http://127.0.0.1:8000

Equivalent explicit command:

sglang serve "$MODEL_DIR" \
  --trust-remote-code \
  --served-model-name KIMI-K3-CODER-REAP-320-MXFP4 \
  --tp-size 8 \
  --moe-runner-backend marlin \
  --kv-cache-dtype fp8_e4m3 \
  --mamba-ssm-dtype bfloat16 \
  --mem-fraction-static 0.94 \
  --context-length 32768 \
  --reasoning-parser kimi_k3 \
  --tool-call-parser kimi_k3 \
  --host 0.0.0.0 --port 8000

…with sglang-patch/attn_residual.py bind-mounted over /sgl-workspace/sglang/python/sglang/srt/layers/attn_residual.py.

On B200 / SM100 class hardware the stock flashinfer_mxfp4 path works and neither change is needed.

OpenAI-compatible: POST /v1/chat/completions, GET /v1/models.

Access & licensing

This model is free and ungated. No licence fee, no commercial terms from Blackfrost, no access request — clone it and go.

Because there is no gate, we have no way to reach you when we ship a fix to a model we have labelled experimental. Watch the repo and check the Community tab before filing a bug — the fix may already be there.

Base licence: Kimi K3 — Moonshot AI's terms apply to this derivative and travel with it.
Redistribution: follow the upstream licence.
Ask us about other expert budgets, non-uniform keep sets, or calibration against your own workload rather than generic coding load.

Contact Blackfrost

@Blackfrost_AI on X

DMs are open. Fastest route to a human.

Bug reports are better on the Community tab
so other users can see the fix.

Blackfrost · Las Vegas, Nevada
Frontier model engineering

Downloads last month: 35

Safetensors

Model size

1T params

Tensor type

F32

BF16

Model tree for Blackfrost-AI/KIMI-K3-CODER-REAP-320-MXFP4

Base model

moonshotai/Kimi-K3

Finetuned

(23)

this model