---
license: apache-2.0
base_model: EigenLabs/Qwen3.8-27B-MTP-bf16
tags:
  - mlx
  - qwen3.8
  - multi-token-prediction
  - speculative-decoding
  - qwen3_5
  - apple-silicon
  - 4-bit
library_name: mlx
pipeline_tag: text-generation
---

# Qwen3.8-27B MTP head — mixed precision (bf16 `fc`, 4-bit/g64 elsewhere)

A drafting head for Qwen 3.8 27B native-MTP speculative decode on MLX /
Apple Silicon. This is an **optional experiment, not a speed product**. Acceptance on hard, varied prose is **unmeasured**. Do not quote tok/s for this file. Derived from the organizer-pinned
`EigenLabs/Qwen3.8-27B-MTP-bf16` @ `26a328e070875b0314d652a039b6b59902690f03`.


## Pairing
Hub collection: [https://huggingface.co/collections/junafinity/qwen-38-27b-uncensored-apple-silicon-6a896c726b52be3a0b63400e](https://huggingface.co/collections/junafinity/qwen-38-27b-uncensored-apple-silicon-6a896c726b52be3a0b63400e)


This is a **drafting head only**, not a standalone model. Pair it with a Qwen3.8-27B MLX backbone (for example [Qwen-3.8-27B-Uncensored-8-Bit-MLX](https://huggingface.co/junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX)) in a loader that accepts a scales-keyed mixed-precision head.

```text
backbone: junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX
draft head: junafinity/qwen38-mtp-head-fc-bf16-4bit
```

Exact loader flags: **[PLACEHOLDER]** (depends on your mlx / speculative-decode entrypoint). Acceptance-rate benefit on hard, varied prose is **unmeasured**; see the caveat below.

## What is different

Existing 4-bit heads for this model quantize all eight head linears. This one
keeps **`fc` in bf16** and quantizes only the other seven (q/k/v/o_proj,
gate/up/down_proj) to MLX affine 4-bit, group size 64 — the geometry that
matches the 4-bit backbone, so a stock scales-keyed loader converts exactly
the modules that carry a `.scales` sibling and leaves `fc` a dense `Linear`
with no code changes.

## Why `fc`

Qwen's own official FP8 release, `Qwen/Qwen3.8-27B-FP8`, lists `mtp.fc` in
`modules_to_not_convert` — it declines to quantize that matrix while
quantizing the head's other large linears. `fc` is the projection that fuses
the embedding stream with the backbone hidden state
(`concat[embed, hidden]` at 10240 → 5120), so representation error there
propagates into every downstream head computation at every draft position.
This artifact tests whether respecting that exclusion improves draft
acceptance enough to pay for the extra bandwidth.

## Cost/benefit

`fc` in bf16 is 104.9 MB versus 29.5 MB at 4-bit — about **+75 MB of weight
traffic per draft step** (total head read ≈ 314 MB vs ≈ 239 MB). Break-even
needs only a small acceptance gain at mid/deep draft positions, since an
accepted draft token amortizes a full ~15.6 GB target verify pass on this
dense 27B model.

## Contents

29 tensors: `fc.weight` (bf16, `[5120, 10240]`), seven bf16 norm vectors, and
weight/scales/biases triples for the seven quantized linears. Single
`model.safetensors` (314,300,360 bytes) plus `config.json` and an index,
carried unchanged from the pinned head so the head loader accepts the tree.

## Caveat on evaluation

Short local decode windows over a model's own greedy self-continuation
saturate draft acceptance (measured: effective draft length 5.4, accept rate
1.000, identical for this head and a uniformly 4-bit one). Such a setup can
measure this artifact's cost but not its benefit; evaluation needs harder,
more varied prose where acceptance is genuinely below 1.