license: other
license_name: kimi-k3
base_model: moonshotai/Kimi-K3
base_model_relation: quantized
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- moe
- reap
- pruned
- kimi
- apple-silicon
Kimi-K3-REAP73-zh-code-MLX-mxfp4-q8
Kimi-K3, REAP expert-pruned (73%) and converted to MLX — 451 GB, sized to actually load on a 512 GB Apple Silicon machine.
Base: moonshotai/Kimi-K3 — 2.78T total / 104B active, native-multimodal MoE with Kimi Delta Attention, Attention Residuals and a 1M context.
Why this exists
Unpruned Kimi-K3 does not fit on any Mac. The full model is 1.56 TB; even a 2-bit quant is ~870 GB against a 512 GB ceiling. Fitting 4-bit into 512 GB would need ≤1.38 bits/weight.
REAP (Cerebras) scores each expert by
gate x ||expert_output|| over a calibration set and keeps the most salient.
This build keeps 242 of 896 experts per layer across 92 MoE
layers → 793B params, 451 GB.
Precision: mxfp4, and it is lossless
K3's routed experts ship from Moonshot as MXFP4 (weight_packed + e8m0
weight_scale, group 32). MLX's native mxfp4 mode uses the identical encoding,
so the surviving experts here are a bit-exact byte copy of the source — the
only information lost in this repo is the pruning itself, not the quantization.
Requantizing those same weights to affine 4-bit would cost ~9.8% mean relative error and be larger (4.5 vs 4.25 bits/weight), so no affine 4-bit tier is published.
Non-expert tensors (mxfp4 global config; attention, shared experts, latent projections, embeddings) are carried at higher precision.
Measured behaviour
Loaded on a 512 GiB M3 Ultra (62 s, 451 GB peak). Verbatim, greedy, unedited:
prompt: 'def merge_intervals(intervals):\n """Merge overlapping intervals."""\n'
--> if not intervals:
return []
intervals.sort(key=lambda x: x[0])
merged = [list
prompt: 'The capital of France is'
--> Paris",
+ "The capital of Germany is Berlin",
"The capital of Italy is Rome",
"The
prompt: '机器学习的基本原理是'
--> ：通过训练数据，学习算法，然后对未知数据进行预测。机器学习的过程是：输入数据→学习算法→
Samples above are verbatim, greedy-decoded, and unedited — including the failure modes. Degradation from the pruning shows up as drift into list-like or source-file-like continuations rather than answering directly, and at heavier prune ratios as outright repetition loops.
Code completions stay structurally correct across every ratio tested, which matches the calibration data: code experts form a dense, self-similar cluster (57% self-overlap in a top-242 set, versus a 27% chance baseline) and so survive pruning better than more diffuse language capability.
Speed — read this before downloading
~0.14 tok/s on a 512 GiB M3 Ultra. Not a typo, and not interactive.
Each decoded token reads roughly 87 GB of weights: 25.8 GB of routed experts plus 60.8 GB of non-expert weights, which every token touches. This is a bandwidth wall, not a headroom problem -- a 350 GB build with 160 GiB of spare memory measured 0.20 tok/s versus 0.16 for a 451 GB build, i.e. gains track size almost exactly. No prune ratio makes K3 interactive on this hardware.
Note the non-experts dominate per-token traffic despite being ~2% of parameters: all of them are read every token, while only 16 of 242 experts are.
Quality expectations — read this
This is an aggressive prune. Top-16 routing over 242 experts is 6.6% density, comparable to a REAP-50 of a 256-expert model. Expect noticeable degradation versus full K3. It is the "fits on one machine" build, not a quality build.
Two things work in its favour that a plain expert cull would not have: K3 keeps 2 shared experts that fire on every token regardless of pruning, and its LatentMoE applies RMSNorm to the combined expert output, which partially self-corrects the magnitude lost when experts are removed.
Calibration — targeted at Chinese + code
This build is calibrated on Chinese and code only, not the full mixed corpus. Kimi-K3's experts cluster by domain — measured over a top-242 set against a 27% chance baseline, code-python↔code-multi overlap is 57.2% while chinese↔code-python is 17.8%, i.e. below chance — so dropping the languages you do not need frees expert slots for the ones you do. Saliency retained rises from 59.1% (mixed) to 69.3% here at identical size.
The tradeoff is real: Japanese, Korean, Russian, Arabic, German, French and
Spanish are materially degraded relative to the mixed build. Use
Kimi-K3-REAP73-MLX-mxfp4-q8 if you need those.
Measured against the mixed build on the same prompts: Chinese improves (the mixed build drifts into restating the prompt by ~18 tokens; this one does not), and code is unchanged.
Saliency was measured on a deliberately mixed 12.6 MB corpus — 40% code
(multi-language + real Python), 30% English web, 15% Chinese, 15% across
ja/ru/ko/de/fr/es/ar. The mix matters: whatever a calibration corpus
under-represents gets pruned away silently. An earlier attempt using C4's pooled
multilingual config left CJK at 0.03% of the corpus, which would have quietly
removed the experts handling Chinese.
Usage
Requires mlx-lm plus the bundled kimi_k3.py loader (the architecture is not
upstream yet). On a 512 GB machine you must raise the GPU wired limit first —
the default is ~75% of RAM, below this model's footprint:
sudo sysctl iogpu.wired_limit_mb=480000
pip install mlx-lm
python - <<'PY'
import os, shutil, mlx_lm
from huggingface_hub import hf_hub_download
for f in ("kimi_k3.py",):
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", f)
shutil.copy(hf_hub_download("pipenetwork/Kimi-K3-REAP73-zh-code-MLX-mxfp4-q8", f), dst)
PY
mlx_lm.generate --model pipenetwork/Kimi-K3-REAP73-zh-code-MLX-mxfp4-q8 --max-tokens 256 \
--prompt "Write a Python function that merges overlapping intervals."
kimi_k3_vision.py and kimi_k3_vl/ ship alongside for the vision tower; the
image path needs mlx-vlm and is not exercised by mlx_lm.generate.
Provenance
Converted with PipeNetwork/kimi-k3-mlx: a streaming converter (the model never fits in memory at any stage) and a streaming REAP calibration harness. Weights remain under the Kimi K3 License.