---
license: mit
language:
- ru
- en
tags:
- translation
- transformer
- from-scratch
- pytorch
- russian
- english
datasets:
- Helsinki-NLP/opus-100
metrics:
- bleu
- chrf
library_name: pytorch
inference: false
---

# Russian → English Transformer (from scratch)

A compact **encoder–decoder Transformer trained from scratch** (no pretrained weights)
for Russian→English translation. Built as a learning project — the tokenizer, model,
training loop, and beam-search decoding are all hand-written.

- **Parameters:** ~11.5M
- **Architecture:** 4 encoder + 4 decoder layers, `d_model=256`, 8 heads, `d_ff=1024`,
  sinusoidal positional encoding, tied input/output embeddings
- **Tokenizer:** byte-level BPE, vocab 16,000 (shared RU/EN), included as `tokenizer.json`
- **Data:** 200,000 [opus-100](https://huggingface.co/datasets/Helsinki-NLP/opus-100) RU–EN pairs
- **Training:** 60 epochs max, early-stopped ~epoch 40 (patience 5), Adam + Noam LR schedule,
  label smoothing 0.1, batch size 64

## Results (held-out test split, 1,951 sentences)

| Decoding | BLEU | chrF |
|----------|:----:|:----:|
| Greedy   | 25.04 | 47.07 |
| Beam-5   | **25.91** | **47.85** |

Validation BLEU was 26.96. Note that opus-100 (subtitle-derived) contains some
misaligned reference pairs, so these BLEU numbers slightly **underestimate** true quality.

## Usage

```python
# pip install torch tokenizers huggingface_hub
from huggingface_hub import snapshot_download
import sys

path = snapshot_download("prplguyy/ru-en-transformer")
sys.path.insert(0, path)
from translator import translate

print(translate("Привет, как у тебя дела сегодня?", method="beam"))
# -> "Hey, how are you doing today?"
```

The repo bundles everything needed to run inference on CPU: `model.pt` (weights),
`tokenizer.json`, and the model/decoding code (`config.py`, `model.py`, `decoding.py`,
`translator.py`).

## Limitations

Small from-scratch model: strong on everyday conversational sentences, but expect rough
edges on rare proper names, idioms, and long or technical text. English→Russian is not
supported (trained one direction only).

## Links

- 🕹️ **Live demo:** https://transformertranslaterussian2english.streamlit.app/
- 💻 **Source / training code:** https://github.com/prplguyy/transformerTranslateRussianEnglish