Model Underperformance with German Language
First, thank you for releasing this sophisticated model.
Unfortunately, the Nemotron 3.5 ASR streaming model performs quite poorly on German speech and terminology. Even with 1120ms chunk size and relatively simple cases—such as correctly spelling words—are often handled incorrectly, and these errors are not easily recoverable even with an LLM that understands the broader context.
Here are a few examples (among many others). In comparison, several state-of-the-art models such as Cohere and Voxtral perform noticeably better in this area.
Do you have any suggestions for improving performance in German? It would also be great if this could be taken into account for future model iterations.
Correct | Nemotron
Bahnhofplatz -> Barnusplatz
Heimatort->Heimator
stechender Schmerz -> Stechnerschmerz
rechte Hand->rechter Hand
Achillessehne->Achilleszene
Tee->T
Paracetamol->Paracetamona
vor einer Woche->bis zwar eine Woche
39 Grad->neun und dreißig Grad
Magen-Darm-Probleme->machen Darmprobleme
verschwommenes Sehen->verschworene Seen
Ausschläge, Pusteln->("Ausschläge" fully missing) Pusteln
reaktive Arthritis->reaktive Adritis
Amboss->Ambus
schwindelig->Schwindellich
Gleichgewichtsstörungen->gleiche Wichtstörungen
Herzkranzgefäße->Herzgrenzefäße
Urinieren->ruignieren
Thanks for the detailed report. Those problems can be fixed by fine-tuning.
Path 1 Word boosting / context biasing - Bias the decoder toward your term list, no retraining.
https://docs.nvidia.com/nemo-framework/user-guide/26.02/nemotoolkit/asr/asr_customization/word_boosting.html
https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_Context_Biasing.ipynb
Path 2 n-gram LM fusion - Train a German KenLM / NGPU-LM from text and shallow-fuse it at decode time.
Path 3 Fine-tune This is the durable fix. The whole thing is a few hours on a single GPU.
https://github.com/NVIDIA/NeMo/blob/main/examples/asr/speech_to_text_finetune.py
https://docs.nvidia.com/nemo-framework/user-guide/26.02/nemotoolkit/asr/configs.html#fine-tuning-configurations
A few practical tips that made the difference:
- Train mostly on German, but keep ~20–30% of the other languages in the data too. If you train on German alone, the model gets noticeably worse at other langs.
- keep that roughly half the data, and use a small learning rate so the model improves on German without forgetting what it already knows.
- This model expects a language tag on every training example, so make sure each clip is labeled with its language.
- A few very rare medical compounds (
Achillessehne,Magen-Darm-Probleme) will probably still need a handful of extra examples — you can even generate those with TTS.