---
license: other
license_name: nvidia-open-model-license
language:
- en
pipeline_tag: automatic-speech-recognition
tags:
- onnx
- onnxruntime-web
- webgpu
- nemotron
- fastconformer
- rnnt
- int4
base_model: nvidia/nemotron-speech-streaming-en-0.6b
---

# nemotron-speech-en-0.6b — ONNX 1120ms, int4 (WebGPU)

A **4-bit (MatMulNBits) quantization** of the FP16 1120 ms ONNX export of NVIDIA's
[`nvidia/nemotron-speech-streaming-en-0.6b`](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b)
(English FastConformer encoder + RNN-T transducer, with built-in punctuation &
capitalization), for running **on-device in the browser via `onnxruntime-web` + WebGPU**.

- **~469 MB** total (encoder ~451 MB int4 + fused decoder ~18 MB fp16) vs ~1.25 GB for the FP16 build.
- Quantized only the encoder's MatMul weights to int4 `MatMulNBits` (217 ops); convs and the small
  fused prediction+joint decoder stay higher precision. **No `ConvInteger`**, so it loads and runs
  on the onnxruntime-web WebGPU execution provider.
- Accuracy matches the FP16 export on tested clips (proper nouns, punctuation preserved).

## Provenance

- Base model: `nvidia/nemotron-speech-streaming-en-0.6b` (see `LICENSE.txt` / `NOTICE.txt`).
- FP16 1120 ms ONNX export: [`shameez/nemotron-speech-streaming-en-0.6b-onnx-1120ms`](https://huggingface.co/shameez/nemotron-speech-streaming-en-0.6b-onnx-1120ms).
- int4 quantization: `onnxruntime.quantization.matmul_nbits_quantizer` (bits=4, block_size=128) — see `quantize.py`.

## Files

| file | precision | notes |
|---|---|---|
| `encoder_model.onnx` (+`.data`) | int4 | cache-aware streaming FastConformer encoder |
| `decoder_model.onnx` (+`.data`) | fp16 | fused RNN-T prediction net + joint |
| `tokens.txt` | — | SentencePiece pieces (`<piece> <id>`), vocab 1024 + blank 1024 |

## I/O (unchanged from the FP16 export)

- Encoder: `audio_signal[1,128,121]` (112 new + 9 overlap mel frames, log-mel 128/16kHz),
  `length` i64, batch-first caches `cache_last_channel[1,24,70,1024]` / `cache_last_time[1,24,1024,8]`,
  `cache_last_channel_len` i64 → `outputs[1,1024,14]` (channel-major) + `*_next` caches.
- Decoder (fused): `encoder_outputs[1,1024,1]`, `targets` i32, `target_length` i32,
  `input_states_1/2[2,1,640]` → `outputs[…,1025]` logits, `output_states_1/2`. blank id = 1024.