---
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
language: en
pipeline_tag: automatic-speech-recognition
library: onnx
base_model: nvidia/nemotron-speech-streaming-en-0.6b
tags:
  - onnx
  - automatic-speech-recognition
  - streaming-asr
  - fastconformer
  - rnnt
  - cache-aware
  - nemotron
  - fp16
---

# Nemotron Speech Streaming EN 0.6B — ONNX FP16 (560ms latency)

ONNX **FP16** export of [`nvidia/nemotron-speech-streaming-en-0.6b`](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b), NVIDIA's 600M-parameter cache-aware streaming ASR model (FastConformer encoder + RNNT decoder), configured for **560 ms** streaming latency (`att_context_size = [70, 6]`).

This is a **self-export produced directly from NVIDIA's official NeMo checkpoint** — not derived from any third-party conversion. The export was made to guarantee clean license provenance and to make the 560 ms latency flavor available as a drop-in ONNX model.

> **License:** Governed by the **NVIDIA Open Model License Agreement** ([LICENSE.txt](./LICENSE.txt)).
> This is **NOT** CC-BY-4.0 — a tag that has appeared on some third-party exports of this checkpoint and is incorrect. The source model's correct license tag is `nvidia-open-model-license`.
> Provenance: base model revision `7a9b763e6c5fb103da690219c049fac917aa50b1`.

## Model Architecture

- **Encoder:** Cache-aware streaming FastConformer — 24 layers, 1024 dim
- **Decoder:** RNNT with LSTM prediction network — 2 layers, 640 hidden
- **Vocabulary:** 1025 SentencePiece tokens (1024 + blank, id 1024)
- **Input:** 128-channel mel spectrogram, 16 kHz mono, 560 ms chunks (56 mel frames + 9 cache frames)
- **Precision:** FP16 weights with **FP32-typed graph I/O** (Cast-wrapped boundaries). The cache-aware `pre_encode` subsampling front-end is retained in FP32 for converter compatibility; the 24-layer Conformer body runs in FP16.

## Files

```
config.json                 # Machine-readable runtime parameters
shared/
├── filterbank.bin          # Mel filterbank weights, shape [1, 128, 257], Slaney norm (131584 bytes)
├── filterbank.meta         # "shape=1x128x257"
├── tokens.txt              # SentencePiece vocabulary (1024 tokens, "piece id" per line, ids 0–1023)
└── preprocessor.config     # Feature extraction parameters (normalize = NA → CMVN OFF)
fp16/
├── encoder_model.onnx      # Streaming encoder graph
├── encoder_model.onnx.data # Encoder weights (external data)
├── decoder_model.onnx      # RNNT decoder+joint graph
└── decoder_model.onnx.data # Decoder weights (external data)
conversion/
├── export_nemotron.py             # NeMo → ONNX export driver
├── convert_fp16.py                # FP32 → FP16 conversion
└── dep_baseline_nemo_export_env_macos.txt  # Pinned toolkit versions
LICENSE.txt                 # NVIDIA Open Model License (full text)
NOTICE.txt                  # Attribution + provenance
```

## Runtime Configuration (560 ms)

All values are in [`config.json`](./config.json).

### Audio Preprocessing
| Parameter | Value |
|---|---|
| Sample rate | 16000 Hz |
| Sample format | S16_LE (16-bit signed little-endian) |
| Pre-emphasis | 0.97 |
| FFT size | 512 |
| Hop length | 160 samples (10 ms) |
| Window length | 400 samples (25 ms) |
| Window type | Hann |
| Mel bands | 128 |
| Mel norm | Slaney |
| Mel layout | **Band-major** `[n_mels, n_frames]` |
| Normalize (CMVN) | **OFF** (`normalize = NA` / `null`) |

Mel filterbank weights are in `shared/filterbank.bin` (shape `[1, 128, 257]`). CMVN is off — the `dither` value in the config is ignored by the runtime (forced to 0).

### Encoder Streaming (560 ms)
| Parameter | Value |
|---|---|
| att_context_size | `[70, 6]` |
| chunk_mel_frames | 56 |
| pre_encode_cache_frames | 9 |
| total_input_frames | 65 (56 + 9) |
| chunk_audio_samples | 8960 |
| chunk_duration_ms | 560 |
| `cache_last_channel` shape | `[1, 24, 70, 1024]` (init zeros) |
| `cache_last_time` shape | `[1, 24, 1024, 8]` (init zeros) |
| `cache_last_channel_len` | `[1]` int64 (init zero) |

Feed each chunk's cache outputs back as the next chunk's cache inputs.

### RNNT Decoder
| Parameter | Value |
|---|---|
| Blank token id | 1024 |
| Vocab size | 1025 (1024 + blank) |
| Max symbols per frame | 10 |
| Prediction layers | 2 (LSTM), hidden 640 |
| `input_states_1/2` shape | `[2, 1, 640]` float32 (init zeros) |

## Usage

```bash
# Download
hf download <this-repo> fp16/ shared/ config.json --local-dir ./model
```

```python
import onnxruntime as ort

encoder = ort.InferenceSession("model/fp16/encoder_model.onnx",
                               providers=["CPUExecutionProvider"])
decoder = ort.InferenceSession("model/fp16/decoder_model.onnx",
                               providers=["CPUExecutionProvider"])
# encoder inputs: audio_signal, length, cache_last_channel, cache_last_time, cache_last_channel_len
# decoder inputs: encoder_outputs, targets, target_length, input_states_1, input_states_2
# feed cache outputs back in for streaming
```

## Export Reproducibility

The export + conversion scripts are in [`conversion/`](./conversion). The exact pinned toolkit versions are in `conversion/dep_baseline_nemo_export_env_macos.txt`. Export recipe: load the checkpoint with NeMo `ASRModel.from_pretrained`, set `att_context_size=[70,6]`, enable cache-aware export (`cache_support=True`), export to ONNX opset 17, then convert to FP16 with `keep_io_types=True` (FP32 I/O). See [NOTICE.txt](./NOTICE.txt) for full provenance.

## License

This model is a derivative of [`nvidia/nemotron-speech-streaming-en-0.6b`](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b) and is distributed under the **NVIDIA Open Model License Agreement** ([LICENSE.txt](./LICENSE.txt)). Attribution per Section 3.1 of that license is in [NOTICE.txt](./NOTICE.txt): *"Licensed by NVIDIA Corporation under the NVIDIA Open Model License."*