---
license: mit
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- small-language-model
- slm
- from-scratch
- tiny
- nexus-erebus
- arithmetic
- specialist
---

![Nexus-Erebus-3M](./nexus_erebus.png)

# Nexus-Erebus-3M

A **3-million-parameter** language model trained from scratch by [MaliosDark](https://huggingface.co/MaliosDark).

**It holds the record for tiny integer arithmetic.** On ArithMark-2 it scores **70.72%**, which beats
the previous best tiny model, Atom 2.7M (69.40%), on that task. It is an arithmetic *specialist*: near
chance on general tasks, sharp on math, at a size that runs anywhere.

## How it works

The model reads and writes numbers **least-significant-digit first** (LSD), with a digit-atomic
tokenizer. That alignment of carry propagation with reading order is what lets a 3M model do arithmetic.
The digit reversal is handled inside the tokenizer, so you pass and receive normal text. This requires
`trust_remote_code=True`.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("MaliosDark/Nexus-Erebus-3M", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("MaliosDark/Nexus-Erebus-3M")
```

## Results

Measured on the full test sets. 0-shot, `acc_norm` for multiple choice, accuracy for ArithMark-2.
Weighting follows the Open SLM Leaderboard (HellaSwag 25%, PIQA 25%, ArithMark-2 25%, ARC-Easy 12.5%,
ARC-Challenge 12.5%).

| Task | Nexus-Erebus-3M | Atom 2.7M |
|---|---:|---:|
| **ArithMark-2** | **70.72** | 69.40 |
| PIQA | 51.41 | 52.94 |
| HellaSwag | 26.59 | 27.10 |
| ARC-Easy | 27.02 | 31.19 |
| ARC-Challenge | 20.65 | 21.50 |
| **Weighted average** | 43.14 | 43.95 |

**Honest reading:** #1 on arithmetic itself (70.72 beats 69.40), #2 on the blended average by 0.81,
because Atom has slightly higher commonsense scores. As a 3M specialist, the non-arithmetic tasks sit
near chance, which is expected at this size.

## Reproduce the ArithMark-2 score

```bash
python benchmark_nexus_arithmark.py MaliosDark/Nexus-Erebus-3M
```

## Training

Pretrained from scratch with a digit-atomic 4k tokenizer, LSD digit order, and a large stream of
**freshly generated** synthetic integer arithmetic matched to the ArithMark-2 distribution (addition,
subtraction, multiplication, exact division, mixed and parenthesised multi-operator expressions), mixed
with TinyStories for language. Then a short fine-tune on public benchmark train splits plus more
arithmetic. No evaluation items were used at any stage. Fresh generation (never reusing a fixed pool)
is what stops the model from memorising instead of learning.

## Limitations

- It is an arithmetic specialist. On non-math tasks it performs near chance, by design.
- It reasons over integers in the ArithMark-2 style; it is not a general assistant.

## License

MIT.

Built by MaliosDark.