Kimi-K3-GGUF

GGUF quantizations of moonshotai/Kimi-K3, mirrored at AtomicChat/Kimi-K3.

Kimi K3 is a ~2.9T-parameter (104B active) mixture-of-experts model: 93 layers, 896 experts, 7168 hidden size, 163840 vocab, 1M context, natively trained in MXFP4 (QAT). The original weights ship only in MXFP4, so every GGUF here is derived from a direct MXFP4 to Q8_0 conversion; lower-bit quants are requantized from that Q8_0 base.

Uploads are in progress. Quants land newest first in priority order (1-2 bit first). If a folder is missing, it is still building or uploading.

Requirements

kimi_k3 is not yet in mainline llama.cpp. These files need a build with Kimi K3 support:

• Fork: AtomicBot-ai/atomic-llama-cpp-turboquant, branch feat/kimi-k3-support

Vanilla llama.cpp will reject these files with an unknown-architecture error until the support is merged upstream.

Files

Each quant lives in its own folder as 45 GB shards. Point llama.cpp at the first shard (...-00001-of-000NN.gguf); the rest load automatically.

Folder	Bits	imatrix	Notes
Q8_0	8.5	no	Conversion base, highest fidelity
Q6_K	6.6	yes	Near-lossless
Q5_K_M	5.7	yes
Q4_K_M	4.8	yes	Balanced default
Q3_K_M	3.9	yes
Q2_K	2.9	yes
IQ4_XS	4.3	yes
IQ3_XXS	3.1	yes
IQ2_M	2.7	yes
IQ2_S	2.5	yes
IQ2_XXS	2.1	yes
IQ1_M	1.8	yes	Smallest usable
IQ1_S	1.6	yes	Extreme, quality-degraded

The importance matrix (imatrix/) was computed on the Q8_0 base with the calibration_datav3 corpus and is applied to every i-quant and K-quant.

Usage

# download one quant folder
huggingface-cli download AtomicChat/Kimi-K3-GGUF --include "IQ2_S/*" --local-dir ./kimi-k3

# run with the fork build
./llama-cli -m ./kimi-k3/IQ2_S/Kimi-K3-IQ2_S-00001-of-000NN.gguf -p "Hello"

Attribution and license

This is a derivative of Kimi K3 by Moonshot AI, redistributed under the Kimi K3 License. The original copyright notice and license are preserved in LICENSE.

Kimi K3 License
Copyright (c) 2026 Moonshot AI

Quantized and published by AtomicChat.