File size: 2,908 Bytes
9813267 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | ---
library_name: onnxruntime
pipeline_tag: automatic-speech-recognition
tags:
- onnx
- ctc
- forced-alignment
- word-timestamps
- mobile
- flutter
---
# Fluency NeMo Conformer CTC Aligner
This repository contains the mobile-distribution assets for Fluency's English word-level forced alignment pipeline.
The model is not used as a standalone ASR decoder in the app. Whisper provides the transcript, this model provides frame-level CTC log-probabilities, and the app runs constrained CTC Viterbi to recover word timestamps.
## Runtime Pipeline
```text
Whisper transcript
→ text normalization
→ SentencePiece tokenizer
→ native log-mel feature extraction
→ ONNX Runtime Mobile
→ constrained CTC Viterbi
→ word timestamps
```
## Required App Assets
```text
stt_en_conformer_ctc_small_features.onnx
metadata.json
tokenizer/tokenizer.model
tokenizer/vocab.txt
manifest.json
```
Optional debugging assets are under `parity/`.
## ONNX Contract
- Input `processed_signal`: `[batch, 80, feature_frames]` float32
- Input `processed_signal_length`: `[batch]` int64
- Output `log_probs`: `[batch, encoded_frames, 1025]` float32
- Output `encoded_len`: `[batch]` int64
- CTC blank id: `1024`
## Feature Contract
- Sample rate: `16000 Hz`
- Channels: mono
- Feature type: 80-bin log-mel spectrogram
- Window size: `25 ms`
- Window stride: `10 ms`
- FFT: `512`
- Window: Hann
- Normalization: `per_feature`
The ONNX graph intentionally starts after feature extraction. Mobile clients must implement NeMo-compatible log-mel features before invoking ONNX Runtime.
## Validation
Validated on a 375.838s video using raw Whisper `text` fields only:
- Windows checked: 7
- Segments checked: 110
- Matched words checked: 1029
- PyTorch feature+model total: 3.106s
- ONNX model total: 5.396s
- Log-prob mean abs diff: 0.000014
- Word center delta p50/p95/max: 0.0ms / 0.0ms / 0.0ms
See `parity/parity_1gzKSyOBpZg.json` and `parity/parity_1gzKSyOBpZg.csv` for details.
## Download URLs
```text
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/stt_en_conformer_ctc_small_features.onnx
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/metadata.json
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/tokenizer/tokenizer.model
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/tokenizer/vocab.txt
https://huggingface.co/iamzhangship/fluency-nemo-conformer-ctc-aligner/resolve/main/manifest.json
```
## Checksums
- ONNX sha256: `8998fd44bc2374c7d085f7c709fbfbd28ea6db09998622077071faf478db24fd`
- ONNX size: `53197388` bytes
Use `manifest.json` for full file sizes and sha256 checksums.
## Source Model
- NeMo pretrained model: `stt_en_conformer_ctc_small`
- Export graph: Conformer encoder + CTC projection/log-softmax
- Export metadata model bytes: `53197388`
|