Add NeMo-Speech.cpp GGUF

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 *.nemo filter=lfs diff=lfs merge=lfs -text
 *.png filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -355,30 +355,48 @@ Also, check out the following NVIDIA speech models:
 
 ---
 
-## NVIDIA NeMo
+## How to Use this Model
+
+There are several ways to use this model. Choose the one that fits your needs.
 
-To train, fine-tune or perform inference with this model, you will need to install [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) [\[4\]](#ref-4). We recommend you install it after you've installed python >= 3.11, Cython and latest PyTorch version.
+### Run locally with NeMo-Speech.cpp
+
+[NeMo-Speech.cpp](https://github.com/NVIDIA/NeMo-Speech.cpp) provides a
+lightweight native C++ runtime for local inference with
+this model. After [installing the runtime](https://github.com/NVIDIA/NeMo-Speech.cpp#installation):
 
 ```bash
-apt-get update && apt-get install -y libsndfile1 ffmpeg
-pip install Cython packaging
-pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]
+hf download nvidia/nemotron-3.5-asr-streaming-0.6b \
+  nemotron-3.5-asr-streaming-0.6b.q8_0.gguf \
+  --local-dir models
+
+nemo-speech transcribe audio.wav \
+  --model models/nemotron-3.5-asr-streaming-0.6b.q8_0.gguf \
+  --language en-US
 ```
 
-## How to Use this Model
+Use another supported locale or `--language auto` for automatic language
+detection. See the [NeMo-Speech.cpp documentation](https://github.com/NVIDIA/NeMo-Speech.cpp)
+for more details.
+
+### NVIDIA NeMo
 
-The model is available for use in the NeMo Framework, and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.
+To train, fine-tune or perform inference with this model, install [NVIDIA NeMo](https://github.com/NVIDIA/NeMo) [\[4\]](#ref-4) after installing Python 3.11 or later, Cython, and a recent PyTorch version.
 
-You can also run it with [🤗 Transformers](https://github.com/huggingface/transformers) (more below).
+```bash
+apt-get update && apt-get install -y libsndfile1 ffmpeg
+pip install Cython packaging
+pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[asr]
+```
 
-### Loading the Model
+#### Loading the Model
 
 ```python
 import nemo.collections.asr as nemo_asr
 asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="nvidia/nemotron-3.5-asr-streaming-0.6b")
 ```
 
-### Streaming Inference
+#### Streaming Inference
 
 You can use the cache-aware streaming inference script from NeMo - [NeMo/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py](https://github.com/NVIDIA-NeMo/NeMo/blob/main/examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py)
 
@@ -400,7 +418,7 @@ python examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streami
 - `strip_lang_tags=false` (keep): the tag is left in the output, so you can read the detected language directly from each utterance — useful for mixed-language traffic and language labeling.
 - `strip_lang_tags=true` (remove): the tag is stripped, leaving only the clean transcript text — useful when you only need the spoken words.
 
-### Setting up Streaming Configuration
+#### Setting up Streaming Configuration
 
 Latency is defined by the `att_context_size` param, where att_context_size = `{num_frames_left_context, num_frame_right_context}`, all measured in **80ms frames**:
 

nemotron-3.5-asr-streaming-0.6b.q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5c435f294eea8f88ce68dd27b8c3bfea7f777cb2fbba04fcd30eaa555f429ae
+size 741548352