AlbertHakobyan/mbert-armenian-participle-punct

mBERT — Armenian Participle-Clause Punctuation

Fine-tuned bert-base-multilingual-cased that restores punctuation around Eastern Armenian participle clauses, framed as 4-class token labeling. From the CODASSCA 2026 paper Sequence Labeling for Low-Resource Syntax: Automatic Punctuation of Armenian Participle Clauses.

Labels

0 O · 1 COMMA_AFTER · 2 BUTH_AFTER · 3 REMOVE_COMMA

Results (macro-F1)

Benchmark	macro-F1
Gold 2K (noisy web text)	0.4655
Shtemaran 292 (clean textbook)	0.5190

Training: 15% negative under-sampling of the majority O class; lower 6 layers frozen; hyper-parameters tuned with Optuna (21 trials). See artifacts/ for the raw checkpoint, best config, trial log and final results; training/ holds the notebook.

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
tok   = AutoTokenizer.from_pretrained("AlbertHakobyan/mbert-armenian-participle-punct")
model = AutoModelForTokenClassification.from_pretrained("AlbertHakobyan/mbert-armenian-participle-punct")

Ensemble

Best results come from a soft-vote of this model with the BiLSTM at alpha=0.45: P = 0.45 * P_bilstm + 0.55 * P_mbert (macro-F1 0.6745 on Shtemaran). See the bilstm-armenian-participle-punct repo.

• Code: https://github.com/AlbertHakobyan070/Armenian-participle-phrase-punctuation