HyeBERT (fine-tuned) — Armenian Participle-Clause Punctuation

Fine-tuned Armenian-specific aking11/hyebert (6-layer, ~66M params) for punctuation of Eastern Armenian participle clauses, as 4-class token labeling. From the CODASSCA 2026 paper Sequence Labeling for Low-Resource Syntax.

Labels

0 O · 1 COMMA_AFTER · 2 BUTH_AFTER · 3 REMOVE_COMMA

Results (macro-F1)

Benchmark	macro-F1
Shtemaran 292 (clean textbook)	0.3260

Full fine-tuning (no frozen layers), sqrt inverse-frequency class weighting. In the paper, the deeper 12-layer mBERT outperformed this model, illustrating that model depth outweighed language-specificity for this task. Raw checkpoint and configs are in artifacts/; the notebook is in training/.

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification
tok   = AutoTokenizer.from_pretrained("AlbertHakobyan/hyebert-armenian-participle-punct")
model = AutoModelForTokenClassification.from_pretrained("AlbertHakobyan/hyebert-armenian-participle-punct")

Code: https://github.com/AlbertHakobyan070/Armenian-participle-phrase-punctuation

Downloads last month: 12

Safetensors

Model size

66M params

Tensor type

F32

Model tree for AlbertHakobyan/hyebert-armenian-participle-punct

Base model

aking11/hyebert

Finetuned

(1)

this model