File size: 3,219 Bytes
5b4a734 4db03b9 a03d76a 5b4a734 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
license: apache-2.0
base_model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M
tags:
- mlx
- quantized
- apple-silicon
---
# Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx
MLX quantization of [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M) for Apple Silicon.
> **Note — text tower only.** The source model is a **Qwen3.5-VL multimodal** model (`Qwen3_5ForConditionalGeneration`, with a vision encoder). This MLX conversion contains **only the text/language tower** — the vision encoder weights are **not** included, so this is a **text-only** model and does not accept image or video input. The text reasoning the original is benchmarked for (GSM8K, MMLU) is unaffected.
>
> It loads via the standard MLX **LLM** path (mlx-lm, LM Studio). For LM Studio compatibility the config carries `partial_rotary_factor` **inside** `rope_parameters` (LM Studio's engine hard-indexes that key, unlike mlx-lm which defaults it); the config is also tagged as a causal LM (`architectures: ["Qwen3_5ForCausalLM"]`, vision/image/video token ids removed) to reflect that it is text-only.
**Variant**: Block float MX FP8
**Disk size**: 8826 MB
**Quantized by**: [sahilchachra](https://huggingface.co/sahilchachra)
## Benchmark results
Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.
### Performance
| | This model | FP16 baseline |
|---|---:|---:|
| Decode tok/s (avg, long traces) | 30.67 | N/A |
| Peak memory (GB) | 9.599 | N/A |
| Disk size (MB) | 8826 | 17969 |
### Quality
| Benchmark | This model | FP16 baseline | n |
|---|---:|---:|---:|
| GSM8K (math, accuracy) | 100.0% | N/A | 50 |
| MMLU (knowledge, accuracy) | 80.0% | N/A | 50 |
### Context scaling (decode tok/s)
| Context length | Decode tok/s |
|---:|---:|
| ~128 tokens | 33.7 |
| ~256 tokens | 33.6 |
| ~512 tokens | 33.6 |
| ~1024 tokens | 33.5 |
## Usage
```bash
pip install mlx-lm
```
```python
from mlx_lm import load, generate
model, tokenizer = load("sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256, verbose=True)
```
## All variants in this collection
| Model | Variant |
|---|---|
| [sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp4-mlx](https://huggingface.co/sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp4-mlx) | Block float MX FP4 |
| [sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx](https://huggingface.co/sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-mxfp8-mlx) | Block float MX FP8 ← this model |
| [sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-optiq-5bpw-mlx](https://huggingface.co/sahilchachra/Qwythos-9B-Claude-Mythos-5-1M-optiq-5bpw-mlx) | OptiQ mixed-precision (target 5.0 bpw) |
## Notes
- Requires Apple Silicon (M1 or later) with MLX
- Benchmarks run on Apple M5 Pro, 24 GB unified memory
- License: see [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M) for the original model's license
## Original model
See [empero-ai/Qwythos-9B-Claude-Mythos-5-1M](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M) for full model details and intended use. |