Jon-Nielsen/Swift-Qwen3.8-27B-exl3-SC_4.00bpw_H5_V6

Swift-Qwen3.8-27B — EXL3 quant (turboderp SC_4.00bpw_H5_V6 recipe)

EXL3 quantization of ukisai/Swift-Qwen3.8-27b, converted with turboderp's SC_4.00bpw_H5_V6 recipe — the same per-tensor bit plan, calibration data and settings he published for the base model on the SC_4.00bpw_H5_V6 branch of Qwen3.8-27B-exl3: 4.00 bpw average, 5-bit lm_head, 6-bit vision tower, 4-bit MTP head.

Method

• Per-tensor bit plan taken from the published quant's quantization_config.json (707 quantized tensors) and applied as a fixed recipe, not re-allocated by a budget solver.
• Calibration tokens from turboderp's cal_trace.safetensors (250 rows × 2048 tokens). The trace is token IDs only; all Hessian/error metrics were computed by running those tokens through this finetune's own bf16 weights.
• Converted with exllamav3 1.4.6. The original was made with a pre-1.4.3 dev build, so the version field in quantization_config.json is the only fingerprint field that differs.

Applying a base-model recipe to a finetune

The bit plan was optimized for the base model's per-tensor sensitivities, not re-measured on the finetune — I did not re-run the sensitivity optimizer (it wants the full bf16 model resident, which doesn't fit on my hardware).

• as a sanity check, the per-module proxy errors from this conversion scale cleanly with the assigned bits (overall mean 0.0014; 3 bpw ≈ 0.0044, 4 bpw ≈ 0.0014, 5 bpw ≈ 0.00025, 6 bpw ≈ 0.00003) with no outlier tensors.

To be clear about the limits: no quality evaluations (perplexity, KL divergence, side-by-side comparisons) against the bf16 original or the base-model quant were run.

Structure, verified against the published quant

• 707/707 quantized tensors receive identical bit assignments.
• Identical index counts: 3,080 tensors total, 987 visual, 39 MTP.
• Total shard payload within 12 KB of the original (16,349,981,140 vs 16,349,968,660 bytes — scale-metadata noise).
• Fingerprint: bits 4.0, head_bits 5, vision_bits 6, mtp_bits 4, codebook mul1, out_scales always, calibration 250×2048.

Serving notes

Verified on a single RTX 3090 (24 GB) with TabbyAPI and exllamav3 1.5.0: full 262,144-token window with Q6 KV cache, the 6-bit vision tower offloaded to host RAM, and MTP drafting from the embedded 4-bit head. Short smoke tests only (text and vision); sustained throughput and generation quality were not benchmarked.

License

The weights are a quantization of ukisai's Swift-Qwen3.8-27b and carry its Swift Open License v1.0 (see LICENSE, LICENSE-APACHE-2.0 and NOTICE, reproduced from the source repo). Base-model portions remain under Apache 2.0.

Credits

• turboderp — the recipe, the calibration trace, and exllamav3.
• ukisai — the finetune.