File size: 2,698 Bytes
0183d1e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | ---
language: en
pipeline_tag: text-generation
library_name: mlx
license: apache-2.0
base_model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
tags:
- mlx
- mtplx
- mtp
- multi-token-prediction
- qwen3_5
- apple-silicon
---
# Qwythos-9B-Claude-Mythos-5-MTPLX
An [MTPLX](https://github.com/youssofal/MTPLX)-ready build of
[empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M)
with a working **multi-token-prediction (MTP) head**, for faster local decoding
on Apple Silicon with exact rejection-sampling.
- **Body:** converted to MLX and quantized to **4-bit** (affine, group size 64).
- **MTP head:** kept in **bf16** (small but drives draft acceptance).
- **On disk:** ~5.2 GB.
## Why this exists
The original `empero-ai/Qwythos-9B-Claude-Mythos-5-1M` safetensors release ships
**without** an MTP head — only the companion
[`...-1M-GGUF`](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF)
repo carries a restored Qwen3.5-compatible MTP head (for llama.cpp
`--spec-type draft-mtp`). MTPLX runs MLX, not GGUF, and its Forge tool only
carries over an existing MTP head rather than training one.
This build bridges the gap: the bf16 MTP head was extracted from the GGUF's
`Qwythos-9B-Claude-Mythos-5-1M-MTP-BF16.gguf` (`blk.32` nextn tensors),
remapped to MLX/MTPLX key names, paired with the full-precision body, and run
through `mtplx forge build`. MTP-contract calibration reached **exact agreement
(1.0)** at depths 1–3, confirming the head matches the trunk.
## Architecture
| | |
|---|---|
| Base | Qwen3.5-9B (hybrid: 3:1 linear-attention to full-attention) |
| Layers | 32 + 1 MTP layer |
| Hidden / heads | 4096 / 16 attn, 4 KV, head_dim 256 |
| Vocab | 248320 |
| Max context | 1,048,576 (YaRN) |
| MTPLX arch_id | `qwen3-next-mtp` |
## Usage (MTPLX)
```bash
mtplx serve --model <this-model-dir> \
--chat-template-profile tokenizer \
--reasoning-parser qwen3
```
Notes:
- This is a **reasoning model** — it thinks before answering (`<think>` block),
so allow a generous `max_tokens` (>=2048).
- Use `--chat-template-profile tokenizer` so the model's own chat template (with
`<think>` handling) is used, and `--reasoning-parser qwen3` to fold the
thinking segment.
- Default sampling: `temperature=0.6, top_p=0.95, top_k=20`.
## Provenance & license
- Body: `empero-ai/Qwythos-9B-Claude-Mythos-5-1M`
- MTP head: `empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF`
- Forged locally with MTPLX Forge (see `mtplx_runtime.json` for the contract and
verification details).
- License: Apache-2.0, inherited from the base model. All credit for the base
model and the restored MTP head belongs to Empero AI.
|