Jinhanw Claude Opus 4.8 (1M context) commited on
Commit
22a16ae
·
verified ·
1 Parent(s): a81a649

Update private_README.md and add figures (PNGs via LFS)

Browse files

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.gitattributes CHANGED
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  *.nemo filter=lfs diff=lfs merge=lfs -text
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  *.nemo filter=lfs diff=lfs merge=lfs -text
37
+ *.png filter=lfs diff=lfs merge=lfs -text
arch_slide10.png ADDED

Git LFS Details

  • SHA256: 2be5d8ccfcd8232c6fc57ffacda125221382ef181ec239e1b6fdd78cf0c1c020
  • Pointer size: 130 Bytes
  • Size of remote file: 87.8 kB
avg_wer_summary.png ADDED

Git LFS Details

  • SHA256: 9a5474f81b64e0ce844c5907f073f7fab56a353d21db9196cd11aba22f787060
  • Pointer size: 130 Bytes
  • Size of remote file: 65.5 kB
fleurs_langid_vs_auto.png ADDED

Git LFS Details

  • SHA256: 56c360a710c99cbf70c04006ec1f842b3796b72f5093178d438b2140049b3626
  • Pointer size: 130 Bytes
  • Size of remote file: 83.8 kB
fleurs_wer_vs_chunk_size.png ADDED

Git LFS Details

  • SHA256: 9b340101e45a7ba6268f1b9c90037d9c2cf00c2c1768272bd3bdcaffd9b231ff
  • Pointer size: 130 Bytes
  • Size of remote file: 92.2 kB
latency_vs_parallel.png ADDED

Git LFS Details

  • SHA256: 830d821d3cea9262d83b03df265040669be10773b2235747e55ce5f8ca7edf14
  • Pointer size: 131 Bytes
  • Size of remote file: 142 kB
model_architecture.png ADDED

Git LFS Details

  • SHA256: 3146643d1a7c8dd424adcb221a5dabbda6951da1ef3937f7d15b29e46e5fa272
  • Pointer size: 131 Bytes
  • Size of remote file: 151 kB
model_overview.png ADDED

Git LFS Details

  • SHA256: 133f6fd1664c0605845e48bc8ada8397846871757db731155368cd11c498d43d
  • Pointer size: 130 Bytes
  • Size of remote file: 96.3 kB
private_README.md CHANGED
@@ -1,12 +1,177 @@
1
  ---
2
  license: other
3
- license_name: nvidia-software-and-model-evaluation-license
4
  license_link: >-
5
- https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-and-model-evaluation-license/
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  ---
7
 
8
-
9
- # Nemotron-3.5-asr-streaming-0.6b
10
 
11
  <style>
12
  h1, h2, h3, h4, h5, h6 {
@@ -31,134 +196,256 @@ img {
31
  display: inline;
32
  vertical-align: middle;
33
  }
34
-
35
-
36
  </style>
37
 
38
- [![Model architecture](https://img.shields.io/badge/Model_Arch-FastConformer--CacheAware--RNNT-lightgrey#model-badge)](#model-architecture)
39
- | [![Model size](https://img.shields.io/badge/Params-600M-lightgrey#model-badge)](#model-architecture)
 
 
 
 
 
 
 
 
 
40
 
41
  > [!Note]
42
- > May 29, 2026: nemotron-3.5-asr-streaming-0.6b checkpoint was udpated.
43
 
 
44
 
45
- # Model Overview
46
 
 
47
 
48
- ### Description:
49
- **Nemotron 3 ASR Multilingual (Nemotron-ASR-Streaming-Multilingual)** is a multilingual, streaming Automatic Speech Recognition (ASR) engineered to deliver high-quality multilingual transcription across both low-latency streaming and high-throughput batch workloads. Developed by NVIDIA, this 600M parameter model transcribes speech into text with native support for punctuation and capitalization. <br>
50
 
51
- By leveraging a state-of-the-art **Cache-Aware FastConformer-RNNT** architecture, the model eliminates redundant overlapping computations common in traditional "buffered" streaming. This allows it to process only new audio chunks while reusing cached encoder context, significantly improving computational efficiency and minimizing end-to-end delay without sacrificing accuracy. <br>
52
 
53
- The model transcribes the following languages: English (en-US, en-GB), Spanish (es-US, es-ES), German (de-DE), French (fr-FR, fr-CA), Italian (it-IT), Arabic (ar-AR), Japanese (ja-JP), Korean (ko-KR), Portuguese (pt-BR, pt-PT), Russian (ru-RU), Hindi (hi-IN), Mandarin (zh-CN), Vietnamese (vi-VN), Hebrew (he-IL), Dutch (nl-NL), Czech (cz-CZ), Danish (da-DK), Polish (pl-PL), Norwegian (nn-NO, nb-NO), Swedish (sv-SE), Thai (th-TH), Turkish (tr-TR), Bulgarian (bg-BG), Greek (el-GR), Estonian (et-EE), Finnish (fi-FI), Croatian (hr-HR), Hungarian (hu-HU), Lithuanian (lt-LT), Latvian (lv-LV), Romanian (ro-RO), Slovak (sk-SK), Ukrainian (uk-UA), Maltese (mt-MT), and Slovenian (sl-SL). It supports uppercase and lowercase letters, punctuation, spaces, and apostrophes.
54
 
55
- It was trained on a massive ASR dataset of approximately 450,000 hours of speech and is engineered to perform across diverse and challenging acoustic conditions.
56
 
57
- **This model is ready for commercial/non-commercial use.** <br>
58
 
59
- ### License/Terms of Use:
60
- Governing Terms: Use of this model is governed by the [NVIDIA Model Evaluation License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-and-model-evaluation-license/).
61
 
62
- ### Deployment Geography:
63
- Global <br>
64
 
65
- ### Use Case: <br>
66
- This model is for transcription of multilingual audio. <br>
67
 
68
- ### Release Date: <br>
69
- Hugging Face [06/05/2026] via [https://huggingface.co/nvidia/nemotron-asr-streaming-multilingual-0.6b] <br>
70
 
71
- ## References(s):
72
- [1] [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://arxiv.org/pdf/2312.17279) <br>
73
- [2] [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://arxiv.org/abs/2305.05084) <nr>
74
 
 
75
 
76
- ## Model Architecture:
77
- **Architecture Type:** FastConformer-CacheAware-RNNT with Prompt <br>
78
- The model is based on the Cache-Aware [1] FastConformer [2] architecture with 24 encoder layers and an RNNT (Recurrent Neural Network Transducer) decoder. The cache-aware streaming design enables efficient processing of audio in chunks while maintaining context from previous frames. Unlike buffered inference, this model maintains caches for all encoder self-attention and convolution layers. This enables reuse of hidden states at every streaming step, where cached activations eliminate redundant computations. As a result, there are no overlapping computations; each processed frame is strictly non-overlapping. This model leverages prompts to guide the transcription process, enabling language-specific transcription from a single ASR model through language ID conditioning. <br>
79
 
80
- **Network Architecture:**
81
- * Encoder: Cache-Aware FastConformer with 24 layers
82
- * Decoder: RNNT (Recurrent Neural Network Transducer)
83
 
84
- **This model was developed based on [nvidia/nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b) [1]. <br>
85
 
86
- ** Number of model parameters: 600M <br>
87
 
88
- ## Input(s): <br>
89
- **Input Type(s):** Audio, Lang ID <br>
90
 
91
- **Input Format(s):** wav, string <br>
 
 
 
 
92
 
93
- **Input Parameters:** One-Dimensional (1D) for audio and One-Dimensional (1D) for Lang ID <br>
94
 
95
- **Other Properties Related to Input:** Maximum Length in seconds specific to GPU Memory, No Pre-Processing Needed, Mono channel is required.
96
 
97
- Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
98
 
99
- ## Output(s)
 
 
100
 
101
- **Output Type(s):** Text String in Input Language <br>
 
 
 
 
102
 
103
- **Output Format(s):** String <br>
104
 
105
- **Output Parameters:** One-Dimensional (1D) <br>
 
106
 
107
- **Other Properties Related to Output:** No Maximum Character Length, transcribe punctuation and capitalization.
 
 
 
 
 
 
108
 
109
- Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
 
 
110
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
 
112
- ## How to use this model
113
 
114
- **NeMo-framework (EA):** <br>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115
 
116
  ```bash
117
- git clone https://github.com/kingformatty/NeMo.git
118
- git checkout prompt_unitifed_architecture_hf_EA
 
119
  ```
120
 
121
- **Streaming inference with target language:** <br>
122
 
123
- Supported att_context_size: [56,0], [56,1], [56,3] (Default), [56,6], [56,13]
124
 
125
- ```bash
126
- python ${NEMO_ROOT}/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \
127
- model_path=${MODEL_PATH} \
128
- dataset_manifest=${MANIFEST_PATH} \
129
- output_path=${OUTPUT_FOLDER} \
130
- target_lang=es-ES \
131
- att_context_size="[56,3]" \
132
- strip_lang_tags=true
133
  ```
134
- (es-ES for example, supported languages can be found in model description)
135
 
136
- **Streaming inference with auto model (agnostic language):** <br>
 
 
 
 
137
 
138
  ```bash
139
- python ${NEMO_ROOT}/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \
140
- model_path=${MODEL_PATH} \
141
- dataset_manifest=${MANIFEST_PATH} \
142
- output_path=${OUTPUT_FOLDER} \
143
- target_lang=auto \
144
- att_context_size="[56,3]" \
145
- strip_lang_tags=true
 
 
146
  ```
147
 
 
 
 
148
 
149
- ## Software Integration:
150
- **Runtime Engine(s):**
151
- * NeMo 25.11, Riva 2.25.0 or higher <br>
152
 
153
- **Supported Hardware Microarchitecture Compatibility:** <br>
154
- * NVIDIA Ampere <br>
155
- * NVIDIA Blackwell <br>
156
- * NVIDIA Jetson <br>
157
- * NVIDIA Hopper <br>
158
- * NVIDIA Lovelace <br>
159
- * NVIDIA Turing <br>
160
- * NVIDIA Volta <br>
 
 
 
 
 
 
 
 
 
 
 
161
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
162
 
163
  **Supported Operating System(s):**
164
  * Linux <br>
@@ -166,12 +453,27 @@ python ${NEMO_ROOT}/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_
166
 
167
  The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.<vr>
168
 
 
 
 
 
 
169
  ## Model Version(s):
170
- nemotron-asr-streaming-multilingual-v1 <br>
171
 
172
  ## Training and Evaluation Datasets:
173
 
174
- ## Training Dataset
 
 
 
 
 
 
 
 
 
 
175
 
176
  ** Data Modality: Audio <br>
177
 
@@ -182,21 +484,21 @@ nemotron-asr-streaming-multilingual-v1 <br>
182
 
183
  ** Labeling Method by dataset <br>
184
  * Human <br>
185
- * Synthetic: labels generated from Parakeet-CTC-XXL-1.1b model, PnC generated from [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B) <br>
186
 
187
- **Properties:**
188
- In excess of 450,000 hours of speech across the following languages: English (en-US, en-GB), Spanish (es-US, es-ES), German (de-DE), French (fr-FR, fr-CA), Italian (it-IT), Arabic (ar-AR), Japanese (ja-JP), Korean (ko-KR), Portuguese (pt-BR, pt-PT), Russian (ru-RU), Hindi (hi-IN), Mandarin (zh-CN), Vietnamese (vi-VN), Hebrew (he-IL), Dutch (nl-NL), Czech (cz-CZ), Danish (da-DK), Polish (pl-PL), Norwegian (nn-NO, nb-NO), Swedish (sv-SE), Thai (th-TH), Turkish (tr-TR), Bulgarian (bg-BG), Greek (el-GR), Estonian (et-EE), Finnish (fi-FI), Croatian (hr-HR), Hungarian (hu-HU), Lithuanian (lt-LT), Latvian (lv-LV), Romanian (ro-RO), Slovak (sk-SK), Ukrainian (uk-UA), Maltese (mt-MT), and Slovenian (sl-SL). The training data is a dynamic blend of public and proprietary internal datasets normalized to have spoken forms in text with punctuation and capitalization. <br>
189
 
190
- ## Evaluation Dataset
191
 
192
- ** Data Collection Method by dataset <br>
193
- * Human <br>
194
 
195
- ** Labeling Method by dataset <br>
196
- * Human <br>
197
 
198
- **Properties:** A dynamic blend of public and internal proprietary normalized to spoken forms in text with punctuation and capitalization. <br>
199
 
 
 
 
 
 
 
200
 
201
  # Inference:
202
  **Acceleration Engine:** Triton <br>
@@ -210,15 +512,85 @@ In excess of 450,000 hours of speech across the following languages: English (en
210
  * NVIDIA L40 <br>
211
 
212
 
213
- ## Ethical Considerations:
214
- NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
215
 
216
- For more detailed information on ethical considerations for this model, please see the [Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards](Insert Link Here). <br>
217
 
218
- Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). <br>
219
 
220
- ## Get Help
221
 
222
- ### Enterprise Support
 
 
223
 
224
- Get access to knowledge base articles and support cases or [submit a ticket](https://www.nvidia.com/en-us/data-center/products/ai-enterprise-suite/support/).
 
1
  ---
2
  license: other
3
+ license_name: nvidia-open-model-license
4
  license_link: >-
5
+ https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
6
+ library_name: nemo
7
+ language:
8
+ - en
9
+ - es
10
+ - de
11
+ - fr
12
+ - it
13
+ - ar
14
+ - ja
15
+ - ko
16
+ - pt
17
+ - ru
18
+ - hi
19
+ - zh
20
+ - vi
21
+ - he
22
+ - nl
23
+ - cs
24
+ - da
25
+ - pl
26
+ - 'no'
27
+ - sv
28
+ - th
29
+ - tr
30
+ - bg
31
+ - el
32
+ - et
33
+ - fi
34
+ - hr
35
+ - hu
36
+ - lt
37
+ - lv
38
+ - ro
39
+ - sk
40
+ - uk
41
+ - mt
42
+ - sl
43
+ datasets:
44
+ - nvidia/Granary
45
+ - multilingual_librispeech
46
+ - fleurs
47
+ - mozilla-foundation/common_voice_8_0
48
+ - voxpopuli
49
+ - europarl
50
+ thumbnail: null
51
+ tags:
52
+ - speech-recognition
53
+ - cache-aware ASR
54
+ - automatic-speech-recognition
55
+ - streaming-asr
56
+ - multilingual
57
+ - speech
58
+ - audio
59
+ - FastConformer
60
+ - RNNT
61
+ - Parakeet
62
+ - ASR
63
+ - pytorch
64
+ - NeMo
65
+ widget:
66
+ - example_title: Librispeech sample 1
67
+ src: https://cdn-media.huggingface.co/speech_samples/sample1.flac
68
+ - example_title: Librispeech sample 2
69
+ src: https://cdn-media.huggingface.co/speech_samples/sample2.flac
70
+ model-index:
71
+ - name: nemotron-asr-streaming-multilingual-0.6b
72
+ results:
73
+ - task:
74
+ name: Automatic Speech Recognition
75
+ type: automatic-speech-recognition
76
+ dataset:
77
+ name: FLEURS (English)
78
+ type: google/fleurs
79
+ config: en_us
80
+ split: test
81
+ metrics:
82
+ - name: WER (1.12s frame size, LangID)
83
+ type: wer
84
+ value: 7.91
85
+ - task:
86
+ name: Automatic Speech Recognition
87
+ type: automatic-speech-recognition
88
+ dataset:
89
+ name: FLEURS (Spanish)
90
+ type: google/fleurs
91
+ config: es_419
92
+ split: test
93
+ metrics:
94
+ - name: WER (1.12s frame size, LangID)
95
+ type: wer
96
+ value: 4.11
97
+ - task:
98
+ name: Automatic Speech Recognition
99
+ type: automatic-speech-recognition
100
+ dataset:
101
+ name: FLEURS (French)
102
+ type: google/fleurs
103
+ config: fr_fr
104
+ split: test
105
+ metrics:
106
+ - name: WER (1.12s frame size, LangID)
107
+ type: wer
108
+ value: 9.03
109
+ - task:
110
+ name: Automatic Speech Recognition
111
+ type: automatic-speech-recognition
112
+ dataset:
113
+ name: FLEURS (Italian)
114
+ type: google/fleurs
115
+ config: it_it
116
+ split: test
117
+ metrics:
118
+ - name: WER (1.12s frame size, LangID)
119
+ type: wer
120
+ value: 4.25
121
+ - task:
122
+ name: Automatic Speech Recognition
123
+ type: automatic-speech-recognition
124
+ dataset:
125
+ name: FLEURS (Portuguese)
126
+ type: google/fleurs
127
+ config: pt_br
128
+ split: test
129
+ metrics:
130
+ - name: WER (1.12s frame size, LangID)
131
+ type: wer
132
+ value: 5.48
133
+ - task:
134
+ name: Automatic Speech Recognition
135
+ type: automatic-speech-recognition
136
+ dataset:
137
+ name: FLEURS (German)
138
+ type: google/fleurs
139
+ config: de_de
140
+ split: test
141
+ metrics:
142
+ - name: WER (1.12s frame size, LangID)
143
+ type: wer
144
+ value: 8.31
145
+ - task:
146
+ name: Automatic Speech Recognition
147
+ type: automatic-speech-recognition
148
+ dataset:
149
+ name: FLEURS (Hindi)
150
+ type: google/fleurs
151
+ config: hi_in
152
+ split: test
153
+ metrics:
154
+ - name: WER (1.12s frame size, LangID)
155
+ type: wer
156
+ value: 6.81
157
+ - task:
158
+ name: Automatic Speech Recognition
159
+ type: automatic-speech-recognition
160
+ dataset:
161
+ name: FLEURS (Korean)
162
+ type: google/fleurs
163
+ config: ko_kr
164
+ split: test
165
+ metrics:
166
+ - name: WER (1.12s frame size, LangID)
167
+ type: wer
168
+ value: 7.12
169
+ metrics:
170
+ - wer
171
+ pipeline_tag: automatic-speech-recognition
172
  ---
173
 
174
+ # Nemotron-3.5 ASR
 
175
 
176
  <style>
177
  h1, h2, h3, h4, h5, h6 {
 
196
  display: inline;
197
  vertical-align: middle;
198
  }
 
 
199
  </style>
200
 
201
+ <p align="center">
202
+ <a href="#model-architecture"><img src="https://img.shields.io/badge/Model_Arch-FastConformer--CacheAware--RNNT-lightgrey#model-badge" alt="Model architecture"/></a>
203
+ &nbsp;
204
+ <a href="#model-architecture"><img src="https://img.shields.io/badge/Params-600M-lightgrey#model-badge" alt="Model size"/></a>
205
+ &nbsp;
206
+ <a href="#supported-languages"><img src="https://img.shields.io/badge/Language-Multilingual-lightgrey#model-badge" alt="Language"/></a>
207
+ </p>
208
+
209
+ <p align="center">
210
+ <img src="model_overview.png" alt="Nemotron-3.5-ASR-Streaming overview: multilingual audio across 40 language-locales is transcribed by a cache-aware FastConformer-RNNT model with language-ID prompting into punctuated text with an automatic language tag" width="900"/>
211
+ </p>
212
 
213
  > [!Note]
214
+ > This model is the multilingual extension of [nvidia/nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b), adding language-ID prompt conditioning to support transcription across **40 language-locales** from a single model.
215
 
216
+ **Nemotron-3.5-ASR** is a multilingual, streaming Automatic Speech Recognition (ASR) model engineered to deliver high-quality multilingual transcription across both low-latency streaming and high-throughput batch workloads. Developed by NVIDIA, this 600M parameter model transcribes speech into text with native support for punctuation and capitalization, and offers runtime flexibility with configurable chunk sizes, including 80ms, 160ms, 320ms, 560ms, and 1120ms.
217
 
218
+ By leveraging a state-of-the-art Cache-Aware FastConformer-RNNT architecture, the model eliminates redundant overlapping computations common in traditional "buffered" streaming. This allows it to process only new audio chunks while reusing cached encoder context, significantly improving computational efficiency and minimizing end-to-end delay without sacrificing accuracy.
219
 
220
+ It was trained on a massive ASR dataset and is engineered to perform across diverse and challenging acoustic conditions.
221
 
222
+ This model is ready for commercial/non-commercial use.
 
223
 
224
+ ---
225
 
226
+ ## License/Terms of Use
227
 
228
+ Governing Terms: Use of the model is governed by the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/).
229
 
230
+ ## Deployment Geography
231
 
232
+ Global
 
233
 
234
+ ## Use Case
 
235
 
236
+ This model is for transcription of multilingual audio.
 
237
 
238
+ ## Release Date
 
239
 
240
+ - Hugging Face [06/05/2026] via https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b
 
 
241
 
242
+ ## References
243
 
244
+ <a id="ref-1"></a>[1] [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://arxiv.org/abs/2312.17279)
 
 
245
 
246
+ <a id="ref-2"></a>[2] [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://arxiv.org/abs/2305.05084)
 
 
247
 
248
+ <a id="ref-3"></a>[3] [NVIDIA Granary](https://huggingface.co/datasets/nvidia/Granary)
249
 
250
+ <a id="ref-4"></a>[4] [NVIDIA NeMo Framework](https://github.com/NVIDIA/NeMo)
251
 
252
+ ## Why Choose Nemotron-3.5-ASR-Streaming?
 
253
 
254
+ - 🌍 **Single Multilingual Model:** Transcribes 40 language-locales from one model through language-ID prompt conditioning, with optional automatic language detection.
255
+ - ⚡ **Native Streaming Architecture:** Cache-aware design enables efficient processing of continuous audio streams, designed and optimized for low-latency voice agent applications.
256
+ - 💰 **Improved Operational Efficiency:** Delivers superior throughput compared to traditional buffered streaming approaches. This allows for a higher number of parallel streams within the same GPU memory constraints, directly reducing operational costs for production environments.
257
+ - 🎛️ **Dynamic Runtime Flexibility:** Choose the optimal operating point on the latency-accuracy Pareto curve at inference time. No re-training is required to adjust for different use-case requirements.
258
+ - 📝 **Punctuation & Capitalization:** Built-in support for punctuation and capitalization in output text.
259
 
260
+ ---
261
 
262
+ ## Supported Languages
263
 
264
+ The model supports **40 language-locales** in total, across three tiers:
265
 
266
+ - **Transcription-ready (19 locales):** highest-accuracy ASR, ready out of the box.
267
+ - **Broad-coverage (13 locales):** production ASR across an additional 13 locales.
268
+ - **Adaptation-ready (8 locales):** recognized by the tokenizer; fine-tune on in-domain data to unlock full transcription.
269
 
270
+ | Tier | Languages (locales) |
271
+ | :--- | :--- |
272
+ | **Transcription-ready (19 locales)** | English (en-US, en-GB), Spanish (es-US, es-ES), French (fr-FR, fr-CA), Italian (it-IT), Portuguese (pt-BR, pt-PT), Dutch (nl-NL), German (de-DE), Turkish (tr-TR), Russian (ru-RU), Arabic (ar-AR), Hindi (hi-IN), Japanese (ja-JP), Korean (ko-KR), Vietnamese (vi-VN), Ukrainian (uk-UA) |
273
+ | **Broad-coverage (13 locales)** | Polish (pl-PL), Swedish (sv-SE), Czech (cs-CZ), Norwegian Bokmål (nb-NO), Danish (da-DK), Bulgarian (bg-BG), Finnish (fi-FI), Croatian (hr-HR), Slovak (sk-SK), Mandarin (zh-CN), Hungarian (hu-HU), Romanian (ro-RO), Estonian (et-EE) |
274
+ | **Adaptation-ready (8 locales)** | Greek (el-GR), Lithuanian (lt-LT), Latvian (lv-LV), Maltese (mt-MT), Slovenian (sl-SL), Hebrew (he-IL), Thai (th-TH), Norwegian Nynorsk (nn-NO) |
275
 
276
+ > **Note:** Transcription-ready and broad-coverage locales (**32 total**) produce ASR transcription out of the box; adaptation-ready locales require fine-tuning on in-domain data to enable full transcription. The model supports uppercase and lowercase letters, punctuation, spaces, and apostrophes.
277
 
278
+ > [!Tip]
279
+ > **Automatic language detection / language tagging:** When run with `target_lang=auto`, the model detects the spoken language and emits the corresponding **language code/tag** in the output following the terminal punctuation. This lets a single deployment transcribe mixed-language traffic and automatically label each utterance with its detected language — no separate language-ID component required.
280
 
281
+ ---
282
+
283
+ ## Model Architecture
284
+
285
+ **Architecture Type:** FastConformer-CacheAware-RNNT with Prompt
286
+
287
+ This model consists of a cache-aware streaming Parakeet (FastConformer) encoder with an RNN-T decoder and language-ID prompt conditioning. It is based on the Cache-Aware [\[1\]](#ref-1) FastConformer [\[2\]](#ref-2) architecture with 24 encoder layers and an RNNT (Recurrent Neural Network Transducer) decoder. The cache-aware streaming design enables efficient processing of audio in chunks while maintaining context from previous frames. Unlike buffered inference, this model maintains caches for all encoder self-attention and convolution layers. This enables reuse of hidden states at every streaming step, where cached activations eliminate redundant computations. As a result, there are no overlapping computations; each processed frame is strictly non-overlapping. This model leverages prompts to guide the transcription process, enabling language-specific transcription from a single ASR model through language ID conditioning.
288
 
289
+ <p align="center">
290
+ <img src="model_architecture.png" alt="Nemotron-3.5-ASR-Streaming architecture: FastConformer encoder and language-ID encoding are concatenated, projected, and fed to the RNNT decoder" width="900"/>
291
+ </p>
292
 
293
+ The language-ID prompt is fused with the acoustic representation as follows:
294
+
295
+ - **FastConformer encoder** processes audio into an acoustic embedding of shape (D=512, T).
296
+ - **Language Encoding** expands a 128-dim one-hot language vector across the time axis → (K=128, T), broadcasting the language identity to every frame.
297
+ - **Concatenation** along the feature axis → fused tensor (D + K, T).
298
+ - **Projection layer** maps the fused features to the RNNT decoder.
299
+
300
+ **Network Architecture:**
301
+ - Encoder: Cache-Aware FastConformer with 24 layers
302
+ - Decoder: RNNT (Recurrent Neural Network Transducer)
303
+ - Parameters: 600M
304
+
305
+ **This model was developed based on [nvidia/nemotron-speech-streaming-en-0.6b](https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b).**
306
+
307
+ ---
308
 
309
+ ## Results at a Glance
310
 
311
+ ASR performance is measured using Word Error Rate (WER) on the **FLEURS** test sets. Accuracy stays strong across both modes and improves as the chunk size grows, while remaining competitive even at the lowest-latency 80ms setting. Full tables are in [Performance](#performance).
312
+
313
+ <table>
314
+ <tr>
315
+ <td width="50%"><img src="fleurs_wer_vs_chunk_size.png" alt="FLEURS average WER vs streaming chunk size (LangID vs Auto-detect)"/></td>
316
+ <td width="50%"><img src="fleurs_langid_vs_auto.png" alt="FLEURS WER by language: LangID vs Auto-detect at 320ms chunk"/></td>
317
+ </tr>
318
+ </table>
319
+
320
+ > **Note:** Japanese and Korean are measured using Character Error Rate (CER) rather than WER, as is standard for these languages.
321
+
322
+ ---
323
+
324
+ ## Throughput & Efficiency
325
+
326
+ Despite being **roughly half the size** (0.6B vs. 1.1B), Nemotron-3.5-ASR-Streaming serves **far more concurrent streams at far lower latency** than the [Parakeet RNNT 1.1B multilingual model](https://build.nvidia.com/nvidia/parakeet-1_1b-rnnt-multilingual-asr), which runs on buffered streaming. The cache-aware streaming design avoids the redundant recomputation of buffered inference, so a single H100 can sustain dramatically higher concurrency at every chunk size — directly lowering the cost per stream in production. At the lowest-latency 80ms setting, Nemotron sustains **~17× more concurrent streams** (240 vs. 14); at the 1120ms setting it sustains **6× more** (2,400 vs. 400). The latency-vs-concurrency curves tell the same story: Nemotron (solid green) holds low final-token latency well past 1,000 parallel requests, while Parakeet RNNT 1.1B (dashed blue) saturates after only a few hundred.
327
+
328
+ <table>
329
+ <tr>
330
+ <td width="50%"><img src="throughput_vs_chunk.png" alt="Concurrent streams supported on a single H100: Nemotron ASR streaming vs Parakeet RNNT, across chunk sizes"/></td>
331
+ <td width="50%"><img src="latency_vs_parallel.png" alt="Median final-token latency vs number of parallel requests on a single H100, Nemotron vs Parakeet RNNT across chunk sizes"/></td>
332
+ </tr>
333
+ </table>
334
+
335
+ > Measured on a single NVIDIA H100. Throughput is the number of real-time streams sustainable in parallel; latency is the median final-token latency at a given level of concurrency.
336
+
337
+ ---
338
+
339
+ ## Explore more from NVIDIA
340
+
341
+ For documentation, deployment guides, enterprise-ready APIs, and the latest open models—including Nemotron and other cutting-edge speech, translation, and generative AI—visit the NVIDIA Developer Portal at [developer.nvidia.com](https://developer.nvidia.com/).
342
+ Join the community to access tools, support, and resources to accelerate your development with NVIDIA's NeMo, Riva, NIM, and foundation models.
343
+
344
+ - What is [Nemotron](https://www.nvidia.com/en-us/ai-data-science/foundation-models/nemotron/)?
345
+ - NVIDIA Developer [Nemotron](https://developer.nvidia.com/nemotron)
346
+ - [NVIDIA Riva Speech](https://developer.nvidia.com/riva?sortBy=developer_learning_library%2Fsort%2Ffeatured_in.riva%3Adesc%2Ctitle%3Aasc#demos)
347
+ - [NeMo Documentation](https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/asr/models.html)
348
+
349
+ Also, check out the following NVIDIA speech models:
350
+ - Nemotron Speech Streaming (English) - https://huggingface.co/nvidia/nemotron-speech-streaming-en-0.6b
351
+ - Multitalker Parakeet Streaming - https://huggingface.co/nvidia/multitalker-parakeet-streaming-0.6b-v1
352
+ - Parakeet Realtime EOU - https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1
353
+
354
+ ---
355
+
356
+ ## NVIDIA NeMo
357
+
358
+ To train, fine-tune or perform inference with this model, you will need to install [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) [\[4\]](#ref-4). We recommend you install it after you've installed Cython and latest PyTorch version.
359
 
360
  ```bash
361
+ apt-get update && apt-get install -y libsndfile1 ffmpeg
362
+ pip install Cython packaging
363
+ pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]
364
  ```
365
 
366
+ ## How to Use this Model
367
 
368
+ The model is available for use in the NeMo Framework, and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.
369
 
370
+ ### Loading the Model
371
+
372
+ ```python
373
+ import nemo.collections.asr as nemo_asr
374
+ asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="nvidia/nemotron-3.5-asr-streaming-0.6b")
 
 
 
375
  ```
 
376
 
377
+ ### Streaming Inference
378
+
379
+ You can use the cache-aware streaming inference script from NeMo - [NeMo/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py](https://github.com/NVIDIA-NeMo/NeMo/blob/main/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py)
380
+
381
+ This is a prompt-conditioned multilingual model: pass the target language with `target_lang` (e.g. `en-US`, `es-ES`, `de-DE`), or use `target_lang=auto` for automatic language detection.
382
 
383
  ```bash
384
+ cd NeMo
385
+ python examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \
386
+ model_path=<model_path> \
387
+ dataset_manifest=<dataset_manifest> \
388
+ batch_size=<batch_size> \
389
+ target_lang=<lang_id> \ #language key (e.g. en-US) or "auto" for automatic language detection
390
+ att_context_size="[56,13]" \ #set the second value to the desired right context from {0,1,3,6,13}
391
+ strip_lang_tags=true \ #true: remove the detected language tag from the text; false: keep it in the output
392
+ output_path=<output_folder>
393
  ```
394
 
395
+ **`strip_lang_tags`** controls how the detected language tag is handled in the output. The model appends a language tag (e.g. `<en-US>`) after the transcript's terminal punctuation:
396
+ - `strip_lang_tags=false` (keep): the tag is left in the output, so you can read the detected language directly from each utterance — useful for mixed-language traffic and language labeling.
397
+ - `strip_lang_tags=true` (remove): the tag is stripped, leaving only the clean transcript text — useful when you only need the spoken words.
398
 
399
+ ### Setting up Streaming Configuration
 
 
400
 
401
+ Latency is defined by the `att_context_size` param, where att_context_size = `{num_frames_left_context, num_frame_right_context}`, all measured in **80ms frames**:
402
+
403
+ * [56, 0]: Chunk size = 1 (1 × 80ms = 0.08s)
404
+ * [56, 1]: Chunk size = 2 (2 × 80ms = 0.16s)
405
+ * [56, 3]: Chunk size = 4 (4 × 80ms = 0.32s)
406
+ * [56, 6]: Chunk size = 7 (7 × 80ms = 0.56s)
407
+ * [56, 13]: Chunk size = 14 (14 × 80ms = 1.12s)
408
+
409
+ Here, chunk size = current frame + right context; each chunk is processed in non-overlapping fashion.
410
+
411
+ ### Input(s): <br>
412
+
413
+ **Input Type(s):** Audio, Lang ID <br>
414
+
415
+ **Input Format(s):** wav, string <br>
416
+
417
+ **Input Parameters:** One-Dimensional (1D) for audio and One-Dimensional (1D) for Lang ID <br>
418
+
419
+ **Other Properties Related to Input:** Maximum Length in seconds specific to GPU Memory, No Pre-Processing Needed, Mono channel is required.
420
 
421
+ By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
422
+
423
+ ### Output
424
+
425
+ **Output Type(s):** Text String in Input Language <br>
426
+
427
+ **Output Format(s):** String <br>
428
+
429
+ **Output Parameters:** One-Dimensional (1D) <br>
430
+
431
+ **Other Properties Related to Output:** No Maximum Character Length, transcribe punctuation and capitalization.
432
+
433
+ By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
434
+
435
+ ---
436
+
437
+ ## Software Integration
438
+
439
+ **Runtime Engine:** NeMo 26.06
440
+
441
+ **Supported Hardware Microarchitecture Compatibility:**
442
+ - NVIDIA Ampere
443
+ - NVIDIA Blackwell
444
+ - NVIDIA Hopper
445
+ - NVIDIA Jetson
446
+ - NVIDIA Lovelace
447
+ - NVIDIA Turing
448
+ - NVIDIA Volta
449
 
450
  **Supported Operating System(s):**
451
  * Linux <br>
 
453
 
454
  The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.<vr>
455
 
456
+
457
+
458
+ ---
459
+
460
+
461
  ## Model Version(s):
462
+ nemotron-3.5-asr <br>
463
 
464
  ## Training and Evaluation Datasets:
465
 
466
+ ### Training Datasets
467
+
468
+ It was trained on speech data across 40 language-locales. The training data is a dynamic blend of public and proprietary internal datasets normalized to have spoken forms in text with punctuation and capitalization, including:
469
+
470
+
471
+ - NVIDIA Riva multilingual ASR training set (Proprietary)
472
+ - NVIDIA Granary [\[3\]](#ref-3)
473
+ - Multilingual LibriSpeech (MLS)
474
+ - Mozilla Common Voice
475
+ - FLEURS
476
+ - VoxPopuli / Europarl-ASR
477
 
478
  ** Data Modality: Audio <br>
479
 
 
484
 
485
  ** Labeling Method by dataset <br>
486
  * Human <br>
487
+ * Synthetic: Synthetic labels were generated from an ensemble of ASR models ([NVIDIA Canary](https://huggingface.co/nvidia/canary-1b), [Parakeet Multilingual 1.1B RNNT](https://build.nvidia.com/nvidia/parakeet-1_1b-rnnt-multilingual-asr), [Parakeet CTC 1.1B](https://build.nvidia.com/nvidia/parakeet-ctc-1_1b-asr), [OpenAI Whisper](https://huggingface.co/openai/whisper-large-v3), and [FunASR](https://github.com/modelscope/FunASR)), with punctuation and capitalization (PnC) generated from [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B).
488
 
 
 
489
 
 
490
 
 
 
491
 
492
+ ### Evaluation Datasets
 
493
 
494
+ The model was evaluated on multilingual ASR benchmarks:
495
 
496
+ - FLEURS
497
+ - Mozilla Common Voice (MCV)
498
+ - Multilingual LibriSpeech (MLS)
499
+ - NVIDIA internal multilingual evaluation sets
500
+
501
+ ---
502
 
503
  # Inference:
504
  **Acceleration Engine:** Triton <br>
 
512
  * NVIDIA L40 <br>
513
 
514
 
515
+ ## Performance
516
+
517
+ ASR performance is measured using the Word Error Rate (WER). The tables below report WER (%) on the **FLEURS** test sets across configurable streaming chunk sizes, in two modes:
518
+ - **Language Input (LangID):** the target language is provided to the model.
519
+ - **Auto-detect:** the model automatically detects the spoken language.
520
+
521
+ > **Note:** Japanese, Korean, and Mandarin are evaluated using Character Error Rate (CER) rather than WER, as is standard for these languages.
522
+ > **Note on text normalization:** WER/CER are computed after text normalization that aligns the reference and hypothesis (e.g., casing, punctuation, numerals, and formatting conventions). Normalization is not perfect across all 40 language-locales, and residual mismatches between normalized text can inflate the reported error rates — actual transcription quality may be somewhat better than the numbers suggest.
523
+
524
+ ### Transcription-ready (19 locales)
525
+
526
+ _Languages are ordered by accuracy (lowest WER first)._
527
+
528
+ <table>
529
+ <thead>
530
+ <tr><th rowspan="2" align="left">Language</th><th colspan="5" align="center" style="background-color:#76b900;color:#ffffff">Language Input (LangID)</th><th colspan="5" align="center" style="background-color:#6b7280;color:#ffffff;border-left:2px solid #cbd5e1;">Auto-detect</th></tr>
531
+ <tr><th align="center" style="background-color:#eef6e0">80ms</th><th align="center" style="background-color:#eef6e0">160ms</th><th align="center" style="background-color:#eef6e0">320ms</th><th align="center" style="background-color:#eef6e0">560ms</th><th align="center" style="background-color:#eef6e0">1.12s</th><th align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">80ms</th><th align="center" style="background-color:#f3f4f6;">160ms</th><th align="center" style="background-color:#f3f4f6;">320ms</th><th align="center" style="background-color:#f3f4f6;">560ms</th><th align="center" style="background-color:#f3f4f6;">1.12s</th></tr>
532
+ </thead>
533
+ <tbody>
534
+ <tr><td align="left">Spanish (es-US, es-ES)</td><td align="center" style="background-color:#eef6e0;">4.87</td><td align="center" style="background-color:#eef6e0;">4.64</td><td align="center" style="background-color:#eef6e0;">4.39</td><td align="center" style="background-color:#eef6e0;">4.26</td><td align="center" style="background-color:#eef6e0;">4.11</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">5.04</td><td align="center" style="background-color:#f3f4f6;">4.82</td><td align="center" style="background-color:#f3f4f6;">4.48</td><td align="center" style="background-color:#f3f4f6;">4.34</td><td align="center" style="background-color:#f3f4f6;">4.13</td></tr>
535
+ <tr><td align="left">Italian (it-IT)</td><td align="center" style="background-color:#eef6e0;">5.23</td><td align="center" style="background-color:#eef6e0;">4.85</td><td align="center" style="background-color:#eef6e0;">4.83</td><td align="center" style="background-color:#eef6e0;">4.41</td><td align="center" style="background-color:#eef6e0;">4.25</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">5.28</td><td align="center" style="background-color:#f3f4f6;">4.89</td><td align="center" style="background-color:#f3f4f6;">4.84</td><td align="center" style="background-color:#f3f4f6;">4.47</td><td align="center" style="background-color:#f3f4f6;">4.32</td></tr>
536
+ <tr><td align="left">Portuguese (pt-BR, pt-PT)</td><td align="center" style="background-color:#eef6e0;">6.29</td><td align="center" style="background-color:#eef6e0;">6.10</td><td align="center" style="background-color:#eef6e0;">5.81</td><td align="center" style="background-color:#eef6e0;">5.65</td><td align="center" style="background-color:#eef6e0;">5.48</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">6.41</td><td align="center" style="background-color:#f3f4f6;">6.19</td><td align="center" style="background-color:#f3f4f6;">5.82</td><td align="center" style="background-color:#f3f4f6;">5.57</td><td align="center" style="background-color:#f3f4f6;">5.47</td></tr>
537
+ <tr><td align="left">Hindi (hi-IN)</td><td align="center" style="background-color:#eef6e0;">8.13</td><td align="center" style="background-color:#eef6e0;">7.97</td><td align="center" style="background-color:#eef6e0;">7.41</td><td align="center" style="background-color:#eef6e0;">7.05</td><td align="center" style="background-color:#eef6e0;">6.81</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">11.47</td><td align="center" style="background-color:#f3f4f6;">10.83</td><td align="center" style="background-color:#f3f4f6;">9.88</td><td align="center" style="background-color:#f3f4f6;">9.26</td><td align="center" style="background-color:#f3f4f6;">8.23</td></tr>
538
+ <tr><td align="left">Korean (ko-KR)</td><td align="center" style="background-color:#eef6e0;">7.59</td><td align="center" style="background-color:#eef6e0;">7.70</td><td align="center" style="background-color:#eef6e0;">7.27</td><td align="center" style="background-color:#eef6e0;">7.18</td><td align="center" style="background-color:#eef6e0;">7.12</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">8.31</td><td align="center" style="background-color:#f3f4f6;">8.18</td><td align="center" style="background-color:#f3f4f6;">7.81</td><td align="center" style="background-color:#f3f4f6;">7.49</td><td align="center" style="background-color:#f3f4f6;">7.30</td></tr>
539
+ <tr><td align="left">English (en-US, en-GB)</td><td align="center" style="background-color:#eef6e0;">9.43</td><td align="center" style="background-color:#eef6e0;">8.88</td><td align="center" style="background-color:#eef6e0;">8.27</td><td align="center" style="background-color:#eef6e0;">7.99</td><td align="center" style="background-color:#eef6e0;">7.91</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">9.72</td><td align="center" style="background-color:#f3f4f6;">9.34</td><td align="center" style="background-color:#f3f4f6;">8.84</td><td align="center" style="background-color:#f3f4f6;">8.80</td><td align="center" style="background-color:#f3f4f6;">8.84</td></tr>
540
+ <tr><td align="left">German (de-DE)</td><td align="center" style="background-color:#eef6e0;">9.81</td><td align="center" style="background-color:#eef6e0;">9.21</td><td align="center" style="background-color:#eef6e0;">8.83</td><td align="center" style="background-color:#eef6e0;">8.42</td><td align="center" style="background-color:#eef6e0;">8.31</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">9.90</td><td align="center" style="background-color:#f3f4f6;">9.37</td><td align="center" style="background-color:#f3f4f6;">8.87</td><td align="center" style="background-color:#f3f4f6;">8.58</td><td align="center" style="background-color:#f3f4f6;">8.22</td></tr>
541
+ <tr><td align="left">French (fr-FR, fr-CA)</td><td align="center" style="background-color:#eef6e0;">10.97</td><td align="center" style="background-color:#eef6e0;">10.60</td><td align="center" style="background-color:#eef6e0;">9.79</td><td align="center" style="background-color:#eef6e0;">9.45</td><td align="center" style="background-color:#eef6e0;">9.03</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">11.03</td><td align="center" style="background-color:#f3f4f6;">10.60</td><td align="center" style="background-color:#f3f4f6;">9.84</td><td align="center" style="background-color:#f3f4f6;">9.46</td><td align="center" style="background-color:#f3f4f6;">9.02</td></tr>
542
+ <tr><td align="left">Russian (ru-RU)</td><td align="center" style="background-color:#eef6e0;">10.84</td><td align="center" style="background-color:#eef6e0;">10.73</td><td align="center" style="background-color:#eef6e0;">9.87</td><td align="center" style="background-color:#eef6e0;">9.60</td><td align="center" style="background-color:#eef6e0;">9.17</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">12.47</td><td align="center" style="background-color:#f3f4f6;">12.09</td><td align="center" style="background-color:#f3f4f6;">11.01</td><td align="center" style="background-color:#f3f4f6;">10.57</td><td align="center" style="background-color:#f3f4f6;">10.03</td></tr>
543
+ <tr><td align="left">Turkish (tr-TR)</td><td align="center" style="background-color:#eef6e0;">12.34</td><td align="center" style="background-color:#eef6e0;">12.33</td><td align="center" style="background-color:#eef6e0;">12.05</td><td align="center" style="background-color:#eef6e0;">11.34</td><td align="center" style="background-color:#eef6e0;">11.17</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">12.61</td><td align="center" style="background-color:#f3f4f6;">12.28</td><td align="center" style="background-color:#f3f4f6;">11.93</td><td align="center" style="background-color:#f3f4f6;">11.51</td><td align="center" style="background-color:#f3f4f6;">11.32</td></tr>
544
+ <tr><td align="left">Vietnamese (vi-VN)</td><td align="center" style="background-color:#eef6e0;">13.41</td><td align="center" style="background-color:#eef6e0;">12.87</td><td align="center" style="background-color:#eef6e0;">12.29</td><td align="center" style="background-color:#eef6e0;">11.78</td><td align="center" style="background-color:#eef6e0;">11.18</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">13.59</td><td align="center" style="background-color:#f3f4f6;">13.02</td><td align="center" style="background-color:#f3f4f6;">12.40</td><td align="center" style="background-color:#f3f4f6;">12.02</td><td align="center" style="background-color:#f3f4f6;">11.22</td></tr>
545
+ <tr><td align="left">Dutch (nl-NL)</td><td align="center" style="background-color:#eef6e0;">14.03</td><td align="center" style="background-color:#eef6e0;">13.43</td><td align="center" style="background-color:#eef6e0;">12.17</td><td align="center" style="background-color:#eef6e0;">11.97</td><td align="center" style="background-color:#eef6e0;">11.46</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">14.09</td><td align="center" style="background-color:#f3f4f6;">13.80</td><td align="center" style="background-color:#f3f4f6;">12.62</td><td align="center" style="background-color:#f3f4f6;">12.24</td><td align="center" style="background-color:#f3f4f6;">11.70</td></tr>
546
+ <tr><td align="left">Japanese (ja-JP)</td><td align="center" style="background-color:#eef6e0;">13.87</td><td align="center" style="background-color:#eef6e0;">12.90</td><td align="center" style="background-color:#eef6e0;">12.22</td><td align="center" style="background-color:#eef6e0;">11.91</td><td align="center" style="background-color:#eef6e0;">11.48</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">14.97</td><td align="center" style="background-color:#f3f4f6;">13.85</td><td align="center" style="background-color:#f3f4f6;">13.00</td><td align="center" style="background-color:#f3f4f6;">12.38</td><td align="center" style="background-color:#f3f4f6;">11.66</td></tr>
547
+ <tr><td align="left">Arabic (ar-AR)</td><td align="center" style="background-color:#eef6e0;">13.17</td><td align="center" style="background-color:#eef6e0;">12.65</td><td align="center" style="background-color:#eef6e0;">12.55</td><td align="center" style="background-color:#eef6e0;">12.13</td><td align="center" style="background-color:#eef6e0;">12.03</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">13.47</td><td align="center" style="background-color:#f3f4f6;">12.85</td><td align="center" style="background-color:#f3f4f6;">12.67</td><td align="center" style="background-color:#f3f4f6;">12.18</td><td align="center" style="background-color:#f3f4f6;">12.06</td></tr>
548
+ <tr><td align="left">Ukrainian (uk-UA)</td><td align="center" style="background-color:#eef6e0;">15.70</td><td align="center" style="background-color:#eef6e0;">15.21</td><td align="center" style="background-color:#eef6e0;">14.55</td><td align="center" style="background-color:#eef6e0;">13.67</td><td align="center" style="background-color:#eef6e0;">13.07</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">18.81</td><td align="center" style="background-color:#f3f4f6;">17.96</td><td align="center" style="background-color:#f3f4f6;">16.79</td><td align="center" style="background-color:#f3f4f6;">15.60</td><td align="center" style="background-color:#f3f4f6;">14.59</td></tr>
549
+ <tr><td align="left"><strong>Average</strong></td><td align="center" style="background-color:#eef6e0;"><strong>10.38</strong></td><td align="center" style="background-color:#eef6e0;"><strong>10.00</strong></td><td align="center" style="background-color:#eef6e0;"><strong>9.49</strong></td><td align="center" style="background-color:#eef6e0;"><strong>9.12</strong></td><td align="center" style="background-color:#eef6e0;"><strong>8.84</strong></td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;"><strong>11.14</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>10.67</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>10.05</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>9.63</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>9.21</strong></td></tr>
550
+ </tbody>
551
+ </table>
552
+
553
+ ### Broad-coverage (13 locales)
554
+
555
+ _Languages are ordered by accuracy (lowest WER first)._
556
+
557
+ <table>
558
+ <thead>
559
+ <tr><th rowspan="2" align="left">Language</th><th colspan="5" align="center" style="background-color:#76b900;color:#ffffff">Language Input (LangID)</th><th colspan="5" align="center" style="background-color:#6b7280;color:#ffffff;border-left:2px solid #cbd5e1;">Auto-detect</th></tr>
560
+ <tr><th align="center" style="background-color:#eef6e0">80ms</th><th align="center" style="background-color:#eef6e0">160ms</th><th align="center" style="background-color:#eef6e0">320ms</th><th align="center" style="background-color:#eef6e0">560ms</th><th align="center" style="background-color:#eef6e0">1.12s</th><th align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">80ms</th><th align="center" style="background-color:#f3f4f6;">160ms</th><th align="center" style="background-color:#f3f4f6;">320ms</th><th align="center" style="background-color:#f3f4f6;">560ms</th><th align="center" style="background-color:#f3f4f6;">1.12s</th></tr>
561
+ </thead>
562
+ <tbody>
563
+ <tr><td align="left">Polish</td><td align="center" style="background-color:#eef6e0;">19.88</td><td align="center" style="background-color:#eef6e0;">18.92</td><td align="center" style="background-color:#eef6e0;">17.48</td><td align="center" style="background-color:#eef6e0;">16.61</td><td align="center" style="background-color:#eef6e0;">15.15</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">22.65</td><td align="center" style="background-color:#f3f4f6;">21.63</td><td align="center" style="background-color:#f3f4f6;">20.05</td><td align="center" style="background-color:#f3f4f6;">18.52</td><td align="center" style="background-color:#f3f4f6;">16.55</td></tr>
564
+ <tr><td align="left">Norwegian Bokmål</td><td align="center" style="background-color:#eef6e0;">20.43</td><td align="center" style="background-color:#eef6e0;">20.07</td><td align="center" style="background-color:#eef6e0;">18.90</td><td align="center" style="background-color:#eef6e0;">18.44</td><td align="center" style="background-color:#eef6e0;">18.10</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">20.91</td><td align="center" style="background-color:#f3f4f6;">20.19</td><td align="center" style="background-color:#f3f4f6;">19.29</td><td align="center" style="background-color:#f3f4f6;">18.76</td><td align="center" style="background-color:#f3f4f6;">18.01</td></tr>
565
+ <tr><td align="left">Finnish</td><td align="center" style="background-color:#eef6e0;">21.19</td><td align="center" style="background-color:#eef6e0;">20.57</td><td align="center" style="background-color:#eef6e0;">20.05</td><td align="center" style="background-color:#eef6e0;">18.94</td><td align="center" style="background-color:#eef6e0;">18.34</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">21.61</td><td align="center" style="background-color:#f3f4f6;">20.88</td><td align="center" style="background-color:#f3f4f6;">20.40</td><td align="center" style="background-color:#f3f4f6;">19.36</td><td align="center" style="background-color:#f3f4f6;">18.72</td></tr>
566
+ <tr><td align="left">Mandarin</td><td align="center" style="background-color:#eef6e0;">20.56</td><td align="center" style="background-color:#eef6e0;">20.22</td><td align="center" style="background-color:#eef6e0;">20.03</td><td align="center" style="background-color:#eef6e0;">19.51</td><td align="center" style="background-color:#eef6e0;">19.28</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">22.45</td><td align="center" style="background-color:#f3f4f6;">21.07</td><td align="center" style="background-color:#f3f4f6;">20.59</td><td align="center" style="background-color:#f3f4f6;">20.40</td><td align="center" style="background-color:#f3f4f6;">19.87</td></tr>
567
+ <tr><td align="left">Czech</td><td align="center" style="background-color:#eef6e0;">24.18</td><td align="center" style="background-color:#eef6e0;">23.20</td><td align="center" style="background-color:#eef6e0;">22.41</td><td align="center" style="background-color:#eef6e0;">21.04</td><td align="center" style="background-color:#eef6e0;">20.41</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">25.81</td><td align="center" style="background-color:#f3f4f6;">25.12</td><td align="center" style="background-color:#f3f4f6;">23.68</td><td align="center" style="background-color:#f3f4f6;">22.55</td><td align="center" style="background-color:#f3f4f6;">21.45</td></tr>
568
+ <tr><td align="left">Bulgarian</td><td align="center" style="background-color:#eef6e0;">24.50</td><td align="center" style="background-color:#eef6e0;">23.58</td><td align="center" style="background-color:#eef6e0;">22.80</td><td align="center" style="background-color:#eef6e0;">21.70</td><td align="center" style="background-color:#eef6e0;">20.53</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">28.28</td><td align="center" style="background-color:#f3f4f6;">27.22</td><td align="center" style="background-color:#f3f4f6;">25.54</td><td align="center" style="background-color:#f3f4f6;">24.05</td><td align="center" style="background-color:#f3f4f6;">21.84</td></tr>
569
+ <tr><td align="left">Slovak</td><td align="center" style="background-color:#eef6e0;">25.08</td><td align="center" style="background-color:#eef6e0;">24.14</td><td align="center" style="background-color:#eef6e0;">23.73</td><td align="center" style="background-color:#eef6e0;">22.51</td><td align="center" style="background-color:#eef6e0;">21.28</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">27.59</td><td align="center" style="background-color:#f3f4f6;">26.06</td><td align="center" style="background-color:#f3f4f6;">25.61</td><td align="center" style="background-color:#f3f4f6;">24.15</td><td align="center" style="background-color:#f3f4f6;">22.68</td></tr>
570
+ <tr><td align="left">Swedish</td><td align="center" style="background-color:#eef6e0;">25.61</td><td align="center" style="background-color:#eef6e0;">24.85</td><td align="center" style="background-color:#eef6e0;">23.63</td><td align="center" style="background-color:#eef6e0;">22.72</td><td align="center" style="background-color:#eef6e0;">22.17</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">26.28</td><td align="center" style="background-color:#f3f4f6;">25.56</td><td align="center" style="background-color:#f3f4f6;">24.18</td><td align="center" style="background-color:#f3f4f6;">23.57</td><td align="center" style="background-color:#f3f4f6;">22.53</td></tr>
571
+ <tr><td align="left">Croatian</td><td align="center" style="background-color:#eef6e0;">27.92</td><td align="center" style="background-color:#eef6e0;">27.09</td><td align="center" style="background-color:#eef6e0;">25.79</td><td align="center" style="background-color:#eef6e0;">24.92</td><td align="center" style="background-color:#eef6e0;">23.97</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">32.13</td><td align="center" style="background-color:#f3f4f6;">31.20</td><td align="center" style="background-color:#f3f4f6;">29.65</td><td align="center" style="background-color:#f3f4f6;">28.95</td><td align="center" style="background-color:#f3f4f6;">27.46</td></tr>
572
+ <tr><td align="left">Romanian</td><td align="center" style="background-color:#eef6e0;">31.52</td><td align="center" style="background-color:#eef6e0;">30.93</td><td align="center" style="background-color:#eef6e0;">29.04</td><td align="center" style="background-color:#eef6e0;">27.77</td><td align="center" style="background-color:#eef6e0;">25.90</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">34.22</td><td align="center" style="background-color:#f3f4f6;">33.26</td><td align="center" style="background-color:#f3f4f6;">30.97</td><td align="center" style="background-color:#f3f4f6;">29.84</td><td align="center" style="background-color:#f3f4f6;">26.88</td></tr>
573
+ <tr><td align="left">Estonian</td><td align="center" style="background-color:#eef6e0;">29.95</td><td align="center" style="background-color:#eef6e0;">29.66</td><td align="center" style="background-color:#eef6e0;">28.59</td><td align="center" style="background-color:#eef6e0;">27.37</td><td align="center" style="background-color:#eef6e0;">26.35</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">30.58</td><td align="center" style="background-color:#f3f4f6;">30.09</td><td align="center" style="background-color:#f3f4f6;">28.72</td><td align="center" style="background-color:#f3f4f6;">28.03</td><td align="center" style="background-color:#f3f4f6;">27.19</td></tr>
574
+ <tr><td align="left">Danish</td><td align="center" style="background-color:#eef6e0;">32.62</td><td align="center" style="background-color:#eef6e0;">31.51</td><td align="center" style="background-color:#eef6e0;">30.00</td><td align="center" style="background-color:#eef6e0;">28.92</td><td align="center" style="background-color:#eef6e0;">27.49</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">33.15</td><td align="center" style="background-color:#f3f4f6;">31.77</td><td align="center" style="background-color:#f3f4f6;">30.22</td><td align="center" style="background-color:#f3f4f6;">29.33</td><td align="center" style="background-color:#f3f4f6;">27.81</td></tr>
575
+ <tr><td align="left">Hungarian</td><td align="center" style="background-color:#eef6e0;">32.70</td><td align="center" style="background-color:#eef6e0;">32.03</td><td align="center" style="background-color:#eef6e0;">30.92</td><td align="center" style="background-color:#eef6e0;">29.72</td><td align="center" style="background-color:#eef6e0;">28.68</td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;">33.40</td><td align="center" style="background-color:#f3f4f6;">32.39</td><td align="center" style="background-color:#f3f4f6;">31.49</td><td align="center" style="background-color:#f3f4f6;">30.20</td><td align="center" style="background-color:#f3f4f6;">29.18</td></tr>
576
+ <tr><td align="left"><strong>Average</strong></td><td align="center" style="background-color:#eef6e0;"><strong>25.86</strong></td><td align="center" style="background-color:#eef6e0;"><strong>25.14</strong></td><td align="center" style="background-color:#eef6e0;"><strong>24.11</strong></td><td align="center" style="background-color:#eef6e0;"><strong>23.09</strong></td><td align="center" style="background-color:#eef6e0;"><strong>22.13</strong></td><td align="center" style="background-color:#f3f4f6;border-left:2px solid #cbd5e1;"><strong>27.62</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>26.65</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>25.41</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>24.44</strong></td><td align="center" style="background-color:#f3f4f6;"><strong>23.09</strong></td></tr>
577
+ </tbody>
578
+ </table>
579
+
580
+ ### Adaptation-ready languages (fine-tune to enable)
581
+
582
+ These **8 language-locales** are recognized by the tokenizer but are not tuned for production transcription out of the box: **Greek (el-GR), Hebrew (he-IL), Lithuanian (lt-LT), Slovenian (sl-SL), Latvian (lv-LV), Maltese (mt-MT), Thai (th-TH), and Norwegian Nynorsk (nn-NO)**. Fine-tuning on in-domain data is recommended to bring them to production quality.
583
+
584
+ We are preparing a dedicated blog post that walks through **how to fine-tune Nemotron-3.5-ASR-Streaming to improve these languages**, including before/after results. _Link coming soon — [placeholder](#) until the blog is published._
585
+
586
 
587
+ ---
588
 
589
+ ## Ethical Considerations
590
 
591
+ NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
592
 
593
+ Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
594
+
595
+ ---
596
 
 
throughput_vs_chunk.png ADDED

Git LFS Details

  • SHA256: d879a2f5cc90f9c770d1aca45593a7c4de4f234f7e3836442f60b513f19dc081
  • Pointer size: 130 Bytes
  • Size of remote file: 69 kB