Swift-Qwen3.8-27B-EXL3

EXL3 quantizations of ukisai/Swift-Qwen3.8-27b, with calibration and quantized vision towers.

Each bitrate is on its own branch. Pick a revision:

hf download erlidev/Swift-Qwen3.8-27B-EXL3 --revision SC_4.00bpw_H5_V6 --local-dir ./Swift-exl3-4.0bpw

branch	body bpw	lm_head	vision	size	predicted KLD¹
SC_3.00bpw_H4_V4	3.00	4	4	13.04 GB	0.0385
SC_3.50bpw_H4_V6	3.50	4	6	14.67 GB	0.0200
SC_4.00bpw_H5_V6	4.00	5	6	16.35 GB	0.0100
SC_4.50bpw_H5_V6	4.50	5	6	17.87 GB	0.0052
SC_5.00bpw_H6_V6	5.00	6	6	19.55 GB	0.0026

MTP layers are quantized at 4 bpw throughout. ¹ Values are from sc_optimize calculated from the base model, not the Swift finetune. Should be relatively close, however.

Requirements

ExLlamaV3 ≥ 1.4.4 (earlier versions cannot load a quantized vision tower). Built with 1.5.0

Method

These are self-calibrated (SC_) quants, following the method turboderp used for turboderp/Qwen3.8-27B-exl3:

1. A calibration trace self-sampled by the original Qwen3.8 base model (250 × 2048 token rows, thinking on, reasoning_effort: medium).
2. Per-tensor sensitivity measurements reused from turboderp's noise_attrib.json for the base model.
3. sc_optimize.py --alpha 2.0 per-tensor bit recipe for each target bitrate, rather than a uniform bitrate (seems to be all the hype recently for some reason). Tied groups: k_proj+v_proj, gate_proj+up_proj.
4. convert.py with that recipe and the trace as calibration data; vision tower quantized with an identity Hessian (no vision calibration).

License

Swift weights are distributed under the Swift Open License v1.0, and that license covers these quants as derivative works.

Credits

• UkisAI
• turboderp