| --- |
| library_name: onnxruntime |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - onnx |
| - ctc |
| - forced-alignment |
| - word-timestamps |
| - mobile |
| - flutter |
| --- |
| |
| # Fluency NeMo Conformer CTC Aligner |
|
|
| This repository contains the mobile-distribution assets for Fluency's English word-level forced alignment pipeline. |
|
|
| The model is not used as a standalone ASR decoder in the app. Whisper provides the transcript, this model provides frame-level CTC log-probabilities, and the app runs constrained CTC Viterbi to recover word timestamps. |
|
|
| ## Runtime Pipeline |
|
|
| ```text |
| Whisper transcript |
| → text normalization |
| → SentencePiece tokenizer |
| → native log-mel feature extraction |
| → ONNX Runtime Mobile |
| → constrained CTC Viterbi |
| → word timestamps |
| ``` |
|
|
| ## Required App Assets |
|
|
| ```text |
| stt_en_conformer_ctc_small_features.onnx |
| metadata.json |
| tokenizer/tokenizer.model |
| tokenizer/vocab.txt |
| manifest.json |
| ``` |
|
|
| Optional debugging assets are under `parity/`. |
|
|
| ## ONNX Contract |
|
|
| - Input `processed_signal`: `[batch, 80, feature_frames]` float32 |
| - Input `processed_signal_length`: `[batch]` int64 |
| - Output `log_probs`: `[batch, encoded_frames, 1025]` float32 |
| - Output `encoded_len`: `[batch]` int64 |
| - CTC blank id: `1024` |
|
|
| ## Feature Contract |
|
|
| - Sample rate: `16000 Hz` |
| - Channels: mono |
| - Feature type: 80-bin log-mel spectrogram |
| - Window size: `25 ms` |
| - Window stride: `10 ms` |
| - FFT: `512` |
| - Window: Hann |
| - Normalization: `per_feature` |
|
|
| The ONNX graph intentionally starts after feature extraction. Mobile clients must implement NeMo-compatible log-mel features before invoking ONNX Runtime. |
|
|
| ## Validation |
|
|
| Validated on a 375.838s video using raw Whisper `text` fields only: |
|
|
| - Windows checked: 7 |
| - Segments checked: 110 |
| - Matched words checked: 1029 |
| - PyTorch feature+model total: 3.106s |
| - ONNX model total: 5.396s |
| - Log-prob mean abs diff: 0.000014 |
| - Word center delta p50/p95/max: 0.0ms / 0.0ms / 0.0ms |
|
|
| See `parity/parity_1gzKSyOBpZg.json` and `parity/parity_1gzKSyOBpZg.csv` for details. |
|
|
| ## Download URLs |
|
|
| ```text |
| https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/stt_en_conformer_ctc_small_features.onnx |
| https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/metadata.json |
| https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/tokenizer/tokenizer.model |
| https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/tokenizer/vocab.txt |
| https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/manifest.json |
| ``` |
|
|
| ## Checksums |
|
|
| - ONNX sha256: `8998fd44bc2374c7d085f7c709fbfbd28ea6db09998622077071faf478db24fd` |
| - ONNX size: `53197388` bytes |
|
|
| Use `manifest.json` for full file sizes and sha256 checksums. |
|
|
| ## Source Model |
|
|
| - NeMo pretrained model: `stt_en_conformer_ctc_small` |
| - Export graph: Conformer encoder + CTC projection/log-softmax |
| - Export metadata model bytes: `53197388` |
|
|
