README.md

metadata

license: mit
language:
  - akk
  - sux
  - en
base_model:
  - google-t5/t5-small
pipeline_tag: translation
tags:
  - cuneiform
  - sumerian
  - atf
  - akkadian
  - t5
  - seq2seq
  - ancient-languages
model-index:
  - name: atf-en-best-v2
    results:
      - task:
          type: translation
          name: ATF to English Translation
        metrics:
          - name: BLEU
            type: bleu
            value: 32.3
          - name: chrF
            type: chrf
            value: 46.5
        dataset:
          name: synthetic_rare_pairs (5% test split, seed=42)
          type: custom

ATF-to-English Translation (T5-small)

Translates cuneiform transliterations (ATF format) into English. Handles Akkadian, Sumerian, and mixed-language texts spanning ~3,000 years of Mesopotamian history — from Presargonic royal inscriptions to Neo-Babylonian medical diagnostics.

Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

model = T5ForConditionalGeneration.from_pretrained("montycrypto/atf-to-english")
tokenizer = T5Tokenizer.from_pretrained("montycrypto/atf-to-english")

text = "translate Akkadian to English: {d}a-szur _en gal_ mus-te-szir3 kisz-szat _dingir-mesz_"
inputs = tokenizer(text, return_tensors="pt", max_length=256, truncation=True)
outputs = model.generate(**inputs, max_new_tokens=100, num_beams=4, early_stopping=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# → "Aur, the great lord, who guides all the gods"

Examples

ATF Input	English Output
u4 ri-a u4 su3-u4 ri-a	In distant days, in distant days
{d}a-szur _en gal_ mus-te-szir3 kisz-szat _dingir-mesz_	Aur, the great lord, who guides all the gods
an ki-ta ba-ta-bad-ra2-a-ba	When heaven had been separated from earth,
a-bu _dingir-mesz en kur-kur_	father of the gods, lord of the lands,
DIŠ NA ŠU.GIDIM.MA DAB-su	If a man is seized by Hand of a Ghost
lugal ki-en-gi-ra	and king of Sumer
zu2-gu10	my tooth

Training

• Base model: google-t5/t5-small (60M parameters)
• Data: 131k curated ATF-English pairs from eBL, CDLI, ORACC, and CuneiformTranslators, with synthetic augmentation for rare/specialized terms
• Prompt prefix: "translate Akkadian to English: "
• Epochs: 28 (cosine LR schedule, label smoothing 0.1)
• Effective batch size: 32 (per-device 16 × gradient accumulation 2)
• Learning rate: 3e-4 with cosine decay and 10% warmup
• Precision: fp32 (fp16 caused training collapse with NaN loss on this task)
• Hardware: NVIDIA GB10 (DGX Spark)

Training Data Sources

The training set was assembled from multiple cuneiform databases and augmented with synthetic examples:

• eBL (electronic Babylonian Library) — inline #tr.en: translations from published fragments
• CDLI (Cuneiform Digital Library Initiative) — line-level translation pairs
• ORACC (Open Richly Annotated Cuneiform Corpus) — scholarly translations
• CuneiformTranslators — large-scale Akkadian/Sumerian→English pairs
• Synthetic augmentation — rare vocabulary, medical/ritual terminology, royal epithet oversampling

The dataset includes pairs spanning multiple historical periods (Presargonic, Sargonic, Ur-III, Old-Babylonian, Neo-Assyrian, Neo-Babylonian) and genres (archival, literary, royal, medical, lexicographic).

Evaluation

Metric	Score
BLEU	32.3
chrF	46.5

Evaluated on a held-out 5% test split (6,582 samples, seed=42) with beam search (num_beams=4).

Limitations

• Optimized for line-level translations; very long multi-line passages may lose coherence
• Based on t5-small (60M params) — capacity-limited on complex royal epithets and rare vocabulary
• ATF input must follow standard transliteration conventions (determinatives in {}, logograms in _, broken text in [], etc.)
• Training data skews toward Neo-Assyrian and Old-Babylonian periods; earlier periods (Presargonic, Ur-III) are underrepresented
• The same cuneiform signs can have different meanings across historical periods; the model may not always select the period-appropriate translation without surrounding context

Citation

If you use this model in your research, please cite:

@misc{montycrypto-atf-to-english,
  title={ATF-to-English Translation Model},
  author={montycrypto},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/montycrypto/atf-to-english}
}