shameez/nemotron-speech-streaming-en-0.6b-onnx-560ms

Nemotron Speech Streaming EN 0.6B — ONNX FP16 (560ms latency)

ONNX FP16 export of nvidia/nemotron-speech-streaming-en-0.6b, NVIDIA's 600M-parameter cache-aware streaming ASR model (FastConformer encoder + RNNT decoder), configured for 560 ms streaming latency (att_context_size = [70, 6]).

This is a self-export produced directly from NVIDIA's official NeMo checkpoint — not derived from any third-party conversion. The export was made to guarantee clean license provenance and to make the 560 ms latency flavor available as a drop-in ONNX model.

License: Governed by the NVIDIA Open Model License Agreement (LICENSE.txt). This is NOT CC-BY-4.0 — a tag that has appeared on some third-party exports of this checkpoint and is incorrect. The source model's correct license tag is nvidia-open-model-license. Provenance: base model revision 7a9b763e6c5fb103da690219c049fac917aa50b1.

Model Architecture

• Encoder: Cache-aware streaming FastConformer — 24 layers, 1024 dim
• Decoder: RNNT with LSTM prediction network — 2 layers, 640 hidden
• Vocabulary: 1025 SentencePiece tokens (1024 + blank, id 1024)
• Input: 128-channel mel spectrogram, 16 kHz mono, 560 ms chunks (56 mel frames + 9 cache frames)
• Precision: FP16 weights with FP32-typed graph I/O (Cast-wrapped boundaries). The cache-aware pre_encode subsampling front-end is retained in FP32 for converter compatibility; the 24-layer Conformer body runs in FP16.

Files

config.json                 # Machine-readable runtime parameters
shared/
├── filterbank.bin          # Mel filterbank weights, shape [1, 128, 257], Slaney norm (131584 bytes)
├── filterbank.meta         # "shape=1x128x257"
├── tokens.txt              # SentencePiece vocabulary (1024 tokens, "piece id" per line, ids 0–1023)
└── preprocessor.config     # Feature extraction parameters (normalize = NA → CMVN OFF)
fp16/
├── encoder_model.onnx      # Streaming encoder graph
├── encoder_model.onnx.data # Encoder weights (external data)
├── decoder_model.onnx      # RNNT decoder+joint graph
└── decoder_model.onnx.data # Decoder weights (external data)
conversion/
├── export_nemotron.py             # NeMo → ONNX export driver
├── convert_fp16.py                # FP32 → FP16 conversion
└── dep_baseline_nemo_export_env_macos.txt  # Pinned toolkit versions
LICENSE.txt                 # NVIDIA Open Model License (full text)
NOTICE.txt                  # Attribution + provenance

Runtime Configuration (560 ms)

All values are in config.json.

Audio Preprocessing

Parameter	Value
Sample rate	16000 Hz
Sample format	S16_LE (16-bit signed little-endian)
Pre-emphasis	0.97
FFT size	512
Hop length	160 samples (10 ms)
Window length	400 samples (25 ms)
Window type	Hann
Mel bands	128
Mel norm	Slaney
Mel layout	Band-major [n_mels, n_frames]
Normalize (CMVN)	OFF (normalize = NA / null)

Mel filterbank weights are in shared/filterbank.bin (shape [1, 128, 257]). CMVN is off — the dither value in the config is ignored by the runtime (forced to 0).

Encoder Streaming (560 ms)

Parameter	Value
att_context_size	[70, 6]
chunk_mel_frames	56
pre_encode_cache_frames	9
total_input_frames	65 (56 + 9)
chunk_audio_samples	8960
chunk_duration_ms	560
cache_last_channel shape	[1, 24, 70, 1024] (init zeros)
cache_last_time shape	[1, 24, 1024, 8] (init zeros)
cache_last_channel_len	[1] int64 (init zero)

Feed each chunk's cache outputs back as the next chunk's cache inputs.

RNNT Decoder

Parameter	Value
Blank token id	1024
Vocab size	1025 (1024 + blank)
Max symbols per frame	10
Prediction layers	2 (LSTM), hidden 640
input_states_1/2 shape	[2, 1, 640] float32 (init zeros)

Usage

# Download
hf download <this-repo> fp16/ shared/ config.json --local-dir ./model

import onnxruntime as ort

encoder = ort.InferenceSession("model/fp16/encoder_model.onnx",
                               providers=["CPUExecutionProvider"])
decoder = ort.InferenceSession("model/fp16/decoder_model.onnx",
                               providers=["CPUExecutionProvider"])
# encoder inputs: audio_signal, length, cache_last_channel, cache_last_time, cache_last_channel_len
# decoder inputs: encoder_outputs, targets, target_length, input_states_1, input_states_2
# feed cache outputs back in for streaming

Export Reproducibility

The export + conversion scripts are in conversion/. The exact pinned toolkit versions are in conversion/dep_baseline_nemo_export_env_macos.txt. Export recipe: load the checkpoint with NeMo ASRModel.from_pretrained, set att_context_size=[70,6], enable cache-aware export (cache_support=True), export to ONNX opset 17, then convert to FP16 with keep_io_types=True (FP32 I/O). See NOTICE.txt for full provenance.

License

This model is a derivative of nvidia/nemotron-speech-streaming-en-0.6b and is distributed under the NVIDIA Open Model License Agreement (LICENSE.txt). Attribution per Section 3.1 of that license is in NOTICE.txt: "Licensed by NVIDIA Corporation under the NVIDIA Open Model License."