toiar/khasi-asr-transcribe-diarize-v2

MOSS-Transcribe-Diarize fine-tuned for Khasi

Fine-tuned from OpenMOSS-Team/MOSS-Transcribe-Diarize (0.9B) on a combined ~342-hour corpus spanning Khasi (kha), English (en), and Hindi (hi), with both timestamped-diarization and speaker-only output modes.

Training data: toiar/Khasi_ASR_Final_Combined (125,220 train rows / 1,153 held-out validation rows, stratified by language and output mode).

Usage

Load exactly like the base model, with trust_remote_code=True:

import torch
from transformers import AutoModelForCausalLM, AutoProcessor
from moss_transcribe_diarize.inference_utils import build_transcription_messages, generate_transcription

model_id = "toiar/khasi-asr-transcribe-diarize-checkpoints-v2"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True, dtype="auto").eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

prompt = (
    "请将音频转写为文本，每一段需以起始时间戳和说话人编号"
    "（[S01]、[S02]、[S03]…）开头，正文为对应的语音内容，"
    "并在段末标注结束时间戳，以清晰标明该段语音范围。"
)
messages = build_transcription_messages("audio.wav", prompt=prompt)
result = generate_transcription(model, processor, messages, max_new_tokens=1024, do_sample=False)
print(result["text"])

Training

• 2 epochs, effective batch size 16, learning rate 1e-5, bf16
• Final eval_loss: 0.2963 (converged from step ~9,000 onward)
• See the base model's official finetune.py for the training script this was built on.

Known limitations

• Hindi (the smallest language slice, ~13.5h) has shown occasional script failures on short utterances (falling back to Romanized text instead of Devanagari) in spot checks. Verify carefully before relying on this model for Hindi transcription.
• Rare numeric/digit transcription errors observed in Khasi spot checks.