---
license: apache-2.0
language:
- en
library_name: mlx
base_model: talkie-lm/talkie-1930-13b-it
base_model_relation: quantized
pipeline_tag: text-generation
tags:
- mlx
- talkie
- vintage
- pre-1931
- apple-silicon
- 8-bit
---

# talkie-1930-13b-it MLX (8-bit quantized)

This is a 8-bit MLX-quantized port of [`talkie-lm/talkie-1930-13b-it`](https://huggingface.co/talkie-lm/talkie-1930-13b-it) — a 13B language model trained on pre-1931 English text — for use on Apple Silicon (M1/M2/M3/M4) via [MLX](https://github.com/ml-explore/mlx).

- **~13.4 GB on disk**, fits comfortably in 24 GB of unified memory.
- **~20 tok/s decode** on an M4 Max.
- The original CUDA-only package needs ≥28 GB of VRAM. This makes it run on any modern Mac.

## Usage

Use the [talkie-mlx](https://github.com/joseluissaorin/talkie-mlx) runtime:

```bash
git clone https://github.com/joseluissaorin/talkie-mlx.git
cd talkie-mlx
pip install -r requirements.txt

hf download joseluissaorin/talkie-1930-13b-it-mlx-q8 \
  --local-dir ~/models/talkie-1930-13b-it-mlx-q8

python run.py generate \
  --model ~/models/talkie-1930-13b-it-mlx-q8 \
  --prompt "Write a brief letter from London, dated July 1925, describing the weather and news of the day." \
  --max-tokens 200 --temperature 0.7 --top-p 0.9
```

Sample output:

> *Dear Friend, It is raining to-day, and the roads are heavy. The weather is very unsettled. The news of the day is the report that the King is ill, and that his life is in danger. Believe me, Yours sincerely, JOHN BROWN.*

For interactive chat:

```bash
python run.py chat --model ~/models/talkie-1930-13b-it-mlx-q8
```

## Quantization details

- 8-bit weight-only quantization via `mx.quantize`, group size 64.
- Quantized: every `nn.Linear` (`attn_query`, `attn_key`, `attn_value`, `attn_resid`, `mlp_gate`, `mlp_linear`, `mlp_resid`) and the token embedding.
- **Not** quantized: `lm_head` (kept in bf16, ~700 MB), per-head/-layer scalar gains, and the `lm_head_gain` scalar.
- 7 sharded safetensors files plus `model.safetensors.index.json`.

## Files

```
config.json                        # architecture + quantization params
vocab.txt                          # tiktoken BPE merges
model.safetensors.index.json       # shard map
model-00001-of-00007.safetensors   # ~2.4 GB (embedding + first blocks)
model-0000{2..6}-of-00007.safetensors   # ~2.1 GB each
model-00007-of-00007.safetensors   # ~0.5 GB
```

## Model architecture

40-layer decoder-only transformer:

- 40 heads, head_dim 128 (MHA, no GQA).
- RoPE with base `1_000_000`, NeoX split-half rotation (with the upstream's specific sign convention).
- Per-head QK rms-norm after RoPE, learned per-head gain on `q`.
- SwiGLU MLP, `n_mlp = 13696`.
- Embedding skip: post-norm token embedding added to every block via a learned per-layer gain.
- Per-block `attn_gain` / `mlp_gain` (init `(2·n_layer)^−0.5`), global `lm_head_gain`.
- Vocab 65540 (base vocab 65536 + 4 IT special tokens), context 2048.

## Limitations

- **Pre-1931 only**: the model has no knowledge of anything after that date and will give incorrect or anachronistic answers about modern events.
- **2048 context** (upstream limit).
- **4-bit quality**: good for completion-style tasks; for tricky reasoning prefer the [4-bit version](https://huggingface.co/joseluissaorin/talkie-1930-13b-it-mlx-q4) or run bf16 on a Mac with ≥48 GB.

## Credits and license

All credit for the model and the underlying research goes to [**talkie-lm**](https://huggingface.co/talkie-lm). This is an unofficial port.

Licensed under **Apache License 2.0**, matching upstream. The original `talkie-lm/talkie-1930-13b-it` is also Apache 2.0.