---
license: mit
language:
- akk
- sux
- en
base_model:
- google-t5/t5-small
pipeline_tag: translation
tags:
- cuneiform
- sumerian
- atf
- akkadian
- t5
- seq2seq
- ancient-languages
model-index:
  - name: atf-en-best-v2
    results:
      - task:
          type: translation
          name: ATF to English Translation
        metrics:
          - name: BLEU
            type: bleu
            value: 32.3
          - name: chrF
            type: chrf
            value: 46.5
        dataset:
          name: synthetic_rare_pairs (5% test split, seed=42)
          type: custom
---

# ATF-to-English Translation (T5-small)

Translates **cuneiform transliterations** (ATF format) into English. Handles Akkadian, Sumerian, and mixed-language texts spanning ~3,000 years of Mesopotamian history — from Presargonic royal inscriptions to Neo-Babylonian medical diagnostics.

## Usage

```python
from transformers import T5Tokenizer, T5ForConditionalGeneration

model = T5ForConditionalGeneration.from_pretrained("montycrypto/atf-to-english")
tokenizer = T5Tokenizer.from_pretrained("montycrypto/atf-to-english")

text = "translate Akkadian to English: {d}a-szur _en gal_ mus-te-szir3 kisz-szat _dingir-mesz_"
inputs = tokenizer(text, return_tensors="pt", max_length=256, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=100, num_beams=4, early_stopping=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# → "Aur, the great lord, who guides all the gods"
```

## Examples

| ATF Input | English Output |
|-----------|---------------|
| `u4 ri-a u4 su3-u4 ri-a` | In distant days, in distant days |
| `{d}a-szur _en gal_ mus-te-szir3 kisz-szat _dingir-mesz_` | Aur, the great lord, who guides all the gods |
| `an ki-ta ba-ta-bad-ra2-a-ba` | When heaven had been separated from earth, |
| `a-bu _dingir-mesz en kur-kur_` | father of the gods, lord of the lands, |
| `DIŠ NA ŠU.GIDIM.MA DAB-su` | If a man is seized by Hand of a Ghost |
| `lugal ki-en-gi-ra` | and king of Sumer |
| `zu2-gu10` | my tooth |

## Training

- **Base model**: [google-t5/t5-small](https://huggingface.co/google-t5/t5-small) (60M parameters)
- **Data**: 131k curated ATF-English pairs from eBL, CDLI, ORACC, and CuneiformTranslators, with synthetic augmentation for rare/specialized terms
- **Prompt prefix**: `"translate Akkadian to English: "`
- **Epochs**: 28 (cosine LR schedule, label smoothing 0.1)
- **Effective batch size**: 32 (per-device 16 × gradient accumulation 2)
- **Learning rate**: 3e-4 with cosine decay and 10% warmup
- **Precision**: fp32 (fp16 caused training collapse with NaN loss on this task)
- **Hardware**: NVIDIA GB10 (DGX Spark)

### Training Data Sources

The training set was assembled from multiple cuneiform databases and augmented with synthetic examples:

- **eBL** (electronic Babylonian Library) — inline `#tr.en:` translations from published fragments
- **CDLI** (Cuneiform Digital Library Initiative) — line-level translation pairs
- **ORACC** (Open Richly Annotated Cuneiform Corpus) — scholarly translations
- **CuneiformTranslators** — large-scale Akkadian/Sumerian→English pairs
- **Synthetic augmentation** — rare vocabulary, medical/ritual terminology, royal epithet oversampling

The dataset includes pairs spanning multiple historical periods (Presargonic, Sargonic, Ur-III, Old-Babylonian, Neo-Assyrian, Neo-Babylonian) and genres (archival, literary, royal, medical, lexicographic).

## Evaluation

| Metric | Score |
|--------|-------|
| BLEU   | 32.3  |
| chrF   | 46.5  |

Evaluated on a held-out 5% test split (6,582 samples, seed=42) with beam search (num_beams=4).

## Limitations

- Optimized for **line-level** translations; very long multi-line passages may lose coherence
- Based on t5-small (60M params) — capacity-limited on complex royal epithets and rare vocabulary
- ATF input must follow standard transliteration conventions (determinatives in `{}`, logograms in `_`, broken text in `[]`, etc.)
- Training data skews toward Neo-Assyrian and Old-Babylonian periods; earlier periods (Presargonic, Ur-III) are underrepresented
- The same cuneiform signs can have different meanings across historical periods; the model may not always select the period-appropriate translation without surrounding context

## Citation

If you use this model in your research, please cite:

```bibtex
@misc{montycrypto-atf-to-english,
  title={ATF-to-English Translation Model},
  author={montycrypto},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/montycrypto/atf-to-english}
}
```