---
language: hy
license: mit
library_name: transformers
pipeline_tag: token-classification
base_model: bert-base-multilingual-cased
tags:
- token-classification
- punctuation-restoration
- armenian
- eastern-armenian
---

# mBERT — Armenian Participle-Clause Punctuation

Fine-tuned `bert-base-multilingual-cased` that restores punctuation around
**Eastern Armenian participle clauses**, framed as 4-class token labeling.
From the CODASSCA 2026 paper *Sequence Labeling for Low-Resource Syntax:
Automatic Punctuation of Armenian Participle Clauses*.

## Labels
`0 O` · `1 COMMA_AFTER` · `2 BUTH_AFTER` · `3 REMOVE_COMMA`

## Results (macro-F1)
| Benchmark | macro-F1 |
|---|---|
| Gold 2K (noisy web text) | 0.4655 |
| Shtemaran 292 (clean textbook) | 0.5190 |

Training: 15% negative under-sampling of the majority `O` class; lower 6 layers
frozen; hyper-parameters tuned with Optuna (21 trials). See `artifacts/` for the
raw checkpoint, best config, trial log and final results; `training/` holds the notebook.

## Usage
```python
from transformers import AutoTokenizer, AutoModelForTokenClassification
tok   = AutoTokenizer.from_pretrained("AlbertHakobyan/mbert-armenian-participle-punct")
model = AutoModelForTokenClassification.from_pretrained("AlbertHakobyan/mbert-armenian-participle-punct")
```

## Ensemble
Best results come from a soft-vote of this model with the BiLSTM at alpha=0.45:
`P = 0.45 * P_bilstm + 0.55 * P_mbert`  (macro-F1 0.6745 on Shtemaran). See the
`bilstm-armenian-participle-punct` repo.

- Code: https://github.com/AlbertHakobyan070/Armenian-participle-phrase-punctuation