onesvat/telaffuz-yz-wav2vec-mms1b

telaffuz-yz — MMS-1B Turkish Phoneme CTC

Fine-tuned from facebook/mms-1b-fl102 for Turkish phoneme-level CTC recognition. Developed as the acoustic backbone of a Turkish pronunciation assessment system (MSc thesis, Istanbul University–Cerrahpaşa, 2026).

License note: This model is derived from facebook/mms-1b-fl102 which is released under CC-BY-NC-4.0. Commercial use is therefore prohibited.

Model Description

Property	Value
Base model	facebook/mms-1b-fl102
Architecture	Wav2Vec2ForCTC
Parameters	~1 B
Feature encoder	Frozen (7-layer CNN)
Transformer layers	48
CTC vocabulary	54 tokens (49 Turkish phonemes + <pad>, <unk>, <blank>, <spc>)
Sample rate	16 kHz
Task	Turkish phoneme recognition (CTC)

Phoneme Inventory

The 54-token vocabulary covers the full Turkish phoneme inventory used in the thesis pipeline:

• 8 short vowels: a e i ɯ o œ u y
• 8 long vowels: aː eː iː ɯː oː œː uː yː
• 1 vowel allophone: æ
• 20 consonant phonemes: p b t d k ɡ t͡ʃ d͡ʒ f v s z ʃ ʒ h m n l ɾ j
• 7 base consonant allophones: c ɟ ɲ ŋ ɫ β β̞
• 5 pedagogical allophones: pʰ tʰ kʰ cʰ ɾ̞̊
• 1 suprasegmental: ˈ (primary stress)
• Special tokens: <pad> <unk> <blank> <spc>

This inventory is aligned with the rule-based G2P pipeline that produces reference transcriptions.

Training Data

Fine-tuned on ~310 hours of Turkish read speech from two sources:

Corpus	Style	Hours used
Mozilla Common Voice 24.0	Crowdsourced read speech	~126 h
ISSAI Turkish Speech Corpus (TSC)	Studio read speech	~163 h
Total		~310 h

Only segments with verified publisher transcripts are used (Whisper-transcribed segments excluded). Duration filter: 0.5 s ≤ clip ≤ 15 s.

Split composition (speaker-aware 90/5/5):

Split	Segments	Hours
train	239,555	275.5 h
dev	14,793	16.0 h
test	15,277	18.6 h

Speaker assignment uses SHA1 hashing on speaker IDs so no speaker appears in more than one split.

Training Procedure

scripts/train_wav2vec.py --model mms-1b --epochs 10 --batch-size 8 --grad-accum 4 \
    --lr 1e-4 --warmup-steps 1000 --early-stopping-patience 5 \
    --early-stopping-threshold 0.001 --seed 42

Hyperparameter	Value
Effective batch size	32 (per-device 8 × grad-accum 4)
Learning rate	1 × 10⁻⁴
LR schedule	Linear warmup → linear decay to 0
LR warmup steps	1,000
Weight decay	0.0
Epochs planned / run	10 / 7.4 (early stopping)
Early stopping patience	5 evaluations
Feature encoder	Frozen throughout
Precision	FP16
Gradient checkpointing	Enabled (use_reentrant=False)
Seed	42

Checkpoint selection: Best dev-set PER across all evaluations (every 500 steps). HuggingFace Trainer with load_best_model_at_end=True.

Hardware: NVIDIA GeForce RTX 3090 (24 GB VRAM), AMD Ryzen 9 5900X, 64 GB RAM, CUDA 12.4, PyTorch 2.6.0+cu124, Transformers 4.57.6.

Training time: ~37 GPU-hours (62,500 steps, single GPU).

Evaluation Results

Evaluated on the held-out test set (15,277 utterances, 661,008 reference phones) with atomic IPA token decoding (multi-character tokens like t͡ʃ, aː treated as single units):

Dataset	PER
Common Voice (test)	4.00%
ISSAI TSC (test)	4.41%
Combined test	4.13%

The companion XLS-R-300M model achieves 4.05% PER on the same test set.

Usage

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import torch, soundfile as sf

model_id = "onesvat/telaffuz-yz-wav2vec-mms1b"
processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)
model.eval()

audio, sr = sf.read("speech.wav", dtype="float32")
# resample to 16 kHz if needed

inputs = processor(audio, sampling_rate=16000, return_tensors="pt", padding=True)
with torch.no_grad():
    logits = model(**inputs).logits

pred_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(pred_ids)
print(transcription)  # IPA phoneme sequence

Intended Use

This model is intended for:

• Turkish pronunciation assessment and coaching systems
• Turkish phoneme recognition research
• Downstream tasks requiring Turkish phone-level posteriors

It is not intended for general Turkish ASR (word-level transcription). The vocabulary is phonemic, not graphemic.

Limitations

• Trained on read speech only; performance on spontaneous/conversational speech may be lower.
• Long vowels (aː, eː, …) are treated as distinct phoneme classes; recognition accuracy varies by phoneme (see per-phoneme error analysis in the thesis).
• Duration filter (0.5–15 s) means very short or very long utterances were excluded from training.

Citation

If you use this model, please cite:

@mastersthesis{nesvat2026telaffuz,
  author  = {Onur Nesvat},
  title   = {Türkçe Telaffuz Değerlendirme Sistemi},
  school  = {İstanbul Üniversitesi-Cerrahpaşa},
  year    = {2026},
}

License

CC-BY-NC-4.0 (inherited from facebook/mms-1b-fl102). Commercial use prohibited.