Hugging Face's logo

iamzhangship
/
fluency-nemo-conformer-ctc-aligner

Automatic Speech Recognition
ONNX
onnxruntime
ctc
forced-alignment
word-timestamps
mobile
flutter
Model card Files
xet
 Community
fluency-nemo-conformer-ctc-aligner / README.md
iamzhangship's picture
iamzhangship
Add NeMo Conformer CTC aligner ONNX v1
9813267 verified
preview code
|
Raw
History Blame Contribute Delete
2.91 kB
metadata 
library_name: onnxruntime
pipeline_tag: automatic-speech-recognition
tags:
  - onnx
  - ctc
  - forced-alignment
  - word-timestamps
  - mobile
  - flutter

Fluency NeMo Conformer CTC Aligner

This repository contains the mobile-distribution assets for Fluency's English word-level forced alignment pipeline.

The model is not used as a standalone ASR decoder in the app. Whisper provides the transcript, this model provides frame-level CTC log-probabilities, and the app runs constrained CTC Viterbi to recover word timestamps.

Runtime Pipeline 

Whisper transcript
→ text normalization
→ SentencePiece tokenizer
→ native log-mel feature extraction
→ ONNX Runtime Mobile
→ constrained CTC Viterbi
→ word timestamps

Required App Assets 

stt_en_conformer_ctc_small_features.onnx
metadata.json
tokenizer/tokenizer.model
tokenizer/vocab.txt
manifest.json

Optional debugging assets are under parity/.

ONNX Contract

  • Input processed_signal: [batch, 80, feature_frames] float32
  • Input processed_signal_length: [batch] int64
  • Output log_probs: [batch, encoded_frames, 1025] float32
  • Output encoded_len: [batch] int64
  • CTC blank id: 1024

Feature Contract

  • Sample rate: 16000 Hz
  • Channels: mono
  • Feature type: 80-bin log-mel spectrogram
  • Window size: 25 ms
  • Window stride: 10 ms
  • FFT: 512
  • Window: Hann
  • Normalization: per_feature

The ONNX graph intentionally starts after feature extraction. Mobile clients must implement NeMo-compatible log-mel features before invoking ONNX Runtime.

Validation

Validated on a 375.838s video using raw Whisper text fields only:

  • Windows checked: 7
  • Segments checked: 110
  • Matched words checked: 1029
  • PyTorch feature+model total: 3.106s
  • ONNX model total: 5.396s
  • Log-prob mean abs diff: 0.000014
  • Word center delta p50/p95/max: 0.0ms / 0.0ms / 0.0ms

See parity/parity_1gzKSyOBpZg.json and parity/parity_1gzKSyOBpZg.csv for details.

Download URLs 

https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/stt_en_conformer_ctc_small_features.onnx
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/metadata.json
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/tokenizer/tokenizer.model
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/tokenizer/vocab.txt
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/manifest.json

Checksums

  • ONNX sha256: 8998fd44bc2374c7d085f7c709fbfbd28ea6db09998622077071faf478db24fd
  • ONNX size: 53197388 bytes

Use manifest.json for full file sizes and sha256 checksums.

Source Model

  • NeMo pretrained model: stt_en_conformer_ctc_small
  • Export graph: Conformer encoder + CTC projection/log-softmax
  • Export metadata model bytes: 53197388