File size: 3,435 Bytes
d919f4d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ---
license: apache-2.0
base_model: AliesTaha/fable-traces
base_model_relation: quantized
pipeline_tag: text-generation
tags:
- mlx
- mlx-lm
- quantized
- apple-silicon
- qwen3
- instruct
- conversational
- egypt-won
language:
- en
---
# fable-traces — MLX Block float MX FP8
MLX quantization of [**AliesTaha/fable-traces**](https://huggingface.co/AliesTaha/fable-traces),
a fine-tuned [Qwen3-4B-Instruct-2507](Qwen/Qwen3-4B-Instruct-2507) for short, conversational
replies. This variant uses **Block float MX FP8** quantization (8.25 effective bits/weight).
**Quantized by**: [sahilchachra](https://huggingface.co/sahilchachra)
Closest to FP16 quality; 8-bit block-float precision.
## About the base model
- **Architecture:** Qwen3ForCausalLM — 36 layers, hidden 2560, 32 attention heads, 8 KV heads (GQA)
- **Context length:** 262 144 tokens
- **Thinking mode:** Qwen3 hybrid — supports `<think>` chain-of-thought with `enable_thinking=True`
- **Fine-tune domain:** Conversational / instruct (see `egypt-won` tag)
- **License:** Apache 2.0
## Quick start
```bash
pip install mlx-lm
```
```python
from mlx_lm import load, generate
model, tokenizer = load("sahilchachra/fable-traces-mxfp8-mlx")
messages = [{"role": "user", "content": "Tell me something interesting."}]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)
```
### With thinking mode (Qwen3 chain-of-thought)
```python
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True,
enable_thinking=True, # injects <think> block before answer
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=1024, verbose=True)
```
### CLI
```bash
mlx_lm.generate --model sahilchachra/fable-traces-mxfp8-mlx \
--prompt "What's the fastest animal on Earth?" \
--max-tokens 256
```
## Quantization details
| Variant | Format | bpw | Disk | Peak RAM |
|---|---|---:|---:|---:|
| FP16 (original) | BF16 safetensors | 16.0 | 7688 MB | ~8 GB |
| **mxfp8 ← this** | **Block float MX FP8** | **8.25** | **3968 MB** | **3.98 GB** |
| [sahilchachra/fable-traces-4bit-mlx](https://huggingface.co/sahilchachra/fable-traces-4bit-mlx) | Affine int4 (group size 64) | 4.50 | 2184 MB | 2.22 GB |
| [sahilchachra/fable-traces-mxfp4-mlx](https://huggingface.co/sahilchachra/fable-traces-mxfp4-mlx) | Block float MX FP4 | 4.25 | 2050 MB | 2.12 GB |
> **Note on bpw:** Embedding and norm layers are kept at bf16; the reported bpw
> is across all linear weights.
## All MLX variants
| Repo | Format | bpw | Disk |
|---|---|---:|---:|
| [sahilchachra/fable-traces-mxfp4-mlx](https://huggingface.co/sahilchachra/fable-traces-mxfp4-mlx) | MX FP4 | 4.25 | 2050 MB |
| [sahilchachra/fable-traces-4bit-mlx](https://huggingface.co/sahilchachra/fable-traces-4bit-mlx) | Affine int4 | 4.50 | 2184 MB |
| [sahilchachra/fable-traces-mxfp8-mlx](https://huggingface.co/sahilchachra/fable-traces-mxfp8-mlx) ← this | MX FP8 | 8.25 | 3968 MB |
## Credits
- Base fine-tune: [AliesTaha/fable-traces](https://huggingface.co/AliesTaha/fable-traces) by AliesTaha (Apache 2.0)
- Base architecture: [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) by Qwen team
- MLX quantization by [sahilchachra](https://huggingface.co/sahilchachra)
|