indicconformer-stt-pa-ctc-shabad-preview

This is the acoustic model from a pipeline for live Gurbani captioning, described in karanbirsingh.com/gurbani-captioning. The prototype runs at bani.karanbirsingh.com, where these weights drive a corpus-constrained CTC decoder feeding a state-machine line tracker.

The model is not intended in isolation and therefore we deprioritize raw WER / CER in favor of a benchmark that better reflects the end-to-end problem. We frame the problem, benchmark, and initial baseline here: karanbirsingh/live-gurbani-captioning-benchmark-v1.

CTC-only finetune of ai4bharat/indicconformer_stt_pa_hybrid_ctc_rnnt_large, trained April 2026. The hybrid base ships both CTC and RNNT decoders; this release keeps only the CTC head for streaming and edge deployment.

Intended use

In Gurbani-facing applications, raw transcripts from this model should not be displayed directly to end users. Acoustic outputs should be normalized to real Gurbani before reaching any UI. See more in linked writing.

Files

file	size	purpose
model.int8.onnx	184 MB	INT8-quantized ONNX
model.onnx	480 MB	FP32 ONNX
tokenizer.model	242 KB	SentencePiece tokenizer
edge-vocab.json	2.6 KB	flat vocab list (decoder-side)

Usage

# pip install huggingface_hub onnxruntime sentencepiece numpy
import wave
import numpy as np
import onnxruntime as ort
import sentencepiece as spm
from huggingface_hub import snapshot_download

local = snapshot_download("karansea/indicconformer-stt-pa-ctc-shabad-preview")
sess = ort.InferenceSession(f"{local}/model.int8.onnx", providers=["CPUExecutionProvider"])
sp = spm.SentencePieceProcessor(); sp.Load(f"{local}/tokenizer.model")

# 16 kHz mono float32 waveform, shape [1, T]
with wave.open("clip.wav", "rb") as f:
    assert f.getframerate() == 16000 and f.getnchannels() == 1
    pcm = np.frombuffer(f.readframes(f.getnframes()), dtype=np.int16)
audio = (pcm.astype(np.float32) / 32768.0)[None, :]
audio_len = np.array([audio.shape[1]], dtype=np.int64)

log_probs, out_len = sess.run(
    ["log_probs", "out_len"],
    {"audio": audio, "audio_len": audio_len},
)

# CTC greedy decode (vocab = 256 PA tokens + blank at 256)
BLANK = 256
ids = log_probs[0, :int(out_len[0])].argmax(-1).tolist()
collapsed, prev = [], -1
for i in ids:
    if i != prev and i != BLANK:
        collapsed.append(int(i))
    prev = i
print(sp.decode(collapsed))

I/O signature: audio: float32 [B, T] @ 16 kHz, audio_len: int64 [B] → log_probs: float32 [B, T', 257], out_len: int64 [B]. Vocab is 256 Punjabi (Gurmukhi) SentencePiece pieces + CTC blank at index 256.

Attribution

Base model: AI4Bharat IndicConformer (Punjabi, hybrid CTC-RNNT large), released under MIT.

Finetune by @karansea.

License

MIT (inherits from the base model).