README.md

---
license: apache-2.0
base_model: AliesTaha/fable-traces
base_model_relation: quantized
pipeline_tag: text-generation
tags:
  - mlx
  - mlx-lm
  - quantized
  - apple-silicon
  - qwen3
  - instruct
  - conversational
  - egypt-won
language:
  - en
---

# fable-traces — MLX Block float MX FP8

MLX quantization of [**AliesTaha/fable-traces**](https://huggingface.co/AliesTaha/fable-traces),
a fine-tuned [Qwen3-4B-Instruct-2507](Qwen/Qwen3-4B-Instruct-2507) for short, conversational
replies. This variant uses **Block float MX FP8** quantization (8.25 effective bits/weight).

**Quantized by**: [sahilchachra](https://huggingface.co/sahilchachra)  
Closest to FP16 quality; 8-bit block-float precision.

## About the base model

- **Architecture:** Qwen3ForCausalLM — 36 layers, hidden 2560, 32 attention heads, 8 KV heads (GQA)
- **Context length:** 262 144 tokens
- **Thinking mode:** Qwen3 hybrid — supports `<think>` chain-of-thought with `enable_thinking=True`
- **Fine-tune domain:** Conversational / instruct (see `egypt-won` tag)
- **License:** Apache 2.0

## Quick start

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/fable-traces-mxfp8-mlx")

messages = [{"role": "user", "content": "Tell me something interesting."}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)
```

### With thinking mode (Qwen3 chain-of-thought)

```python
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True,
    enable_thinking=True,   # injects <think> block before answer
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=1024, verbose=True)
```

### CLI

```bash
mlx_lm.generate --model sahilchachra/fable-traces-mxfp8-mlx \
    --prompt "What's the fastest animal on Earth?" \
    --max-tokens 256
```

## Quantization details

| Variant | Format | bpw | Disk | Peak RAM |
|---|---|---:|---:|---:|
| FP16 (original) | BF16 safetensors | 16.0 | 7688 MB | ~8 GB |
| **mxfp8 ← this** | **Block float MX FP8** | **8.25** | **3968 MB** | **3.98 GB** |
| [sahilchachra/fable-traces-4bit-mlx](https://huggingface.co/sahilchachra/fable-traces-4bit-mlx) | Affine int4 (group size 64) | 4.50 | 2184 MB | 2.22 GB |
| [sahilchachra/fable-traces-mxfp4-mlx](https://huggingface.co/sahilchachra/fable-traces-mxfp4-mlx) | Block float MX FP4 | 4.25 | 2050 MB | 2.12 GB |

> **Note on bpw:** Embedding and norm layers are kept at bf16; the reported bpw
> is across all linear weights.

## All MLX variants

| Repo | Format | bpw | Disk |
|---|---|---:|---:|
| [sahilchachra/fable-traces-mxfp4-mlx](https://huggingface.co/sahilchachra/fable-traces-mxfp4-mlx) | MX FP4 | 4.25 | 2050 MB |
| [sahilchachra/fable-traces-4bit-mlx](https://huggingface.co/sahilchachra/fable-traces-4bit-mlx) | Affine int4 | 4.50 | 2184 MB |
| [sahilchachra/fable-traces-mxfp8-mlx](https://huggingface.co/sahilchachra/fable-traces-mxfp8-mlx) ← this | MX FP8 | 8.25 | 3968 MB |

## Credits

- Base fine-tune: [AliesTaha/fable-traces](https://huggingface.co/AliesTaha/fable-traces) by AliesTaha (Apache 2.0)
- Base architecture: [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) by Qwen team
- MLX quantization by [sahilchachra](https://huggingface.co/sahilchachra)