Update with Transformers usage (#40)

- Update with Transformers usage (0b3616428140701461f515de309a29fbd69d2b49)


Co-authored-by: Eric Bezzam <bezzam@users.noreply.huggingface.co>

README.md

@@ -43,6 +43,7 @@ tags:
 - pytorch
 - NeMo
 - hf-asr-leaderboard
+- Transformers
 widget:
 - example_title: Librispeech sample 1
   src: https://cdn-media.huggingface.co/speech_samples/sample1.flac
@@ -804,6 +805,7 @@ This model is ready for commercial/non-commercial use.
 
 For full details on the model architecture, training methodology, datasets, and evaluation results, check out the **[Technical Report](https://arxiv.org/abs/2509.14128)**.
 
+
 ## <span style="color:#466f00;">License/Terms of Use:</span>
 
 GOVERNING TERMS: Use of this model is governed by the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode.en) license.
@@ -881,6 +883,10 @@ pip install -U nemo_toolkit['asr']
 ``` 
 The model is available for use in the NeMo toolkit [5], and can be used as a pre-trained checkpoint for inference or for fine-tuning on another dataset.
 
+You can also run Parakeet TDT with [Transformers](https://github.com/huggingface/transformers) 🤗 (more below).
+
+### 1) NeMo usage
+
 #### Automatically instantiate the model
 
 ```python
@@ -945,6 +951,122 @@ python NeMo/main/examples/asr/asr_chunked_inference/rnnt/speech_to_text_streamin
 
 NVIDIA NIM for v2 parakeet model is available at [https://build.nvidia.com/nvidia/parakeet-tdt-0_6b-v2](https://build.nvidia.com/nvidia/parakeet-tdt-0_6b-v2).
 
+
+### 2) [Transformers](https://github.com/huggingface/transformers) 🤗 usage 
+
+
+Until Parakeet TDT is part of an official Transformers release, you can use it by installing from source.
+
+```bash
+pip install git+https://github.com/huggingface/transformers
+```
+
+<details>
+  <summary>➡️ Pipeline usage</summary>
+
+```python
+from transformers import pipeline
+
+pipe = pipeline("automatic-speech-recognition", model="nvidia/parakeet-tdt-0.6b-v3")
+out = pipe("https://huggingface.co/datasets/hf-internal-testing/dummy-audio-samples/resolve/main/bcn_weather.mp3")
+print(out)
+```
+</details>
+
+<details>
+  <summary>➡️ AutoModel</summary>
+
+```python
+from transformers import AutoModelForTDT, AutoProcessor
+from datasets import load_dataset, Audio
+import torch
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+num_samples = 3
+
+model_id = "nvidia/parakeet-tdt-0.6b-v3"
+processor = AutoProcessor.from_pretrained(model_id)
+model = AutoModelForTDT.from_pretrained(model_id, dtype="auto", device_map=device)
+
+ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
+ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
+speech_samples = [el["array"] for el in ds["audio"][:num_samples]]
+
+inputs = processor(speech_samples, sampling_rate=processor.feature_extractor.sampling_rate)
+inputs.to(model.device, dtype=model.dtype)
+output = model.generate(**inputs, return_dict_in_generate=True)
+print(processor.decode(output.sequences, skip_special_tokens=True))
+```
+</details>
+
+
+<details>
+  <summary>➡️ Timestamping</summary>
+
+```python
+from datasets import Audio, load_dataset
+from transformers import AutoModelForTDT, AutoProcessor
+import torch
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+num_samples = 3
+
+model_id = "nvidia/parakeet-tdt-0.6b-v3"
+processor = AutoProcessor.from_pretrained(model_id)
+model = AutoModelForTDT.from_pretrained(model_id, dtype="auto", device_map=device)
+
+ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
+ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
+speech_samples = [el["array"] for el in ds["audio"][:num_samples]]
+
+inputs = processor(speech_samples, sampling_rate=processor.feature_extractor.sampling_rate)
+inputs.to(model.device, dtype=model.dtype)
+output = model.generate(**inputs, return_dict_in_generate=True)
+decoded_output, decoded_timestamps = processor.decode(
+    output.sequences,
+    durations=output.durations,
+    skip_special_tokens=True,
+)
+print("Transcription:", decoded_output)
+print("Timestamped tokens:", decoded_timestamps)
+```
+</details>
+
+<details>
+  <summary>➡️ Training</summary>
+
+```python
+from transformers import AutoModelForTDT, AutoProcessor
+from datasets import load_dataset, Audio
+import torch
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+
+model_id = "nvidia/parakeet-tdt-0.6b-v3"
+NUM_SAMPLES = 4
+
+processor = AutoProcessor.from_pretrained(model_id)
+model = AutoModelForTDT.from_pretrained(model_id, dtype=torch.bfloat16, device_map=device)
+model.train()
+
+ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
+ds = ds.cast_column("audio", Audio(sampling_rate=processor.feature_extractor.sampling_rate))
+speech_samples = [el["array"] for el in ds["audio"][:NUM_SAMPLES]]
+text_samples = ds["text"][:NUM_SAMPLES]
+
+# passing `text` to the processor will prepare inputs' `labels` key
+inputs = processor(audio=speech_samples, text=text_samples, sampling_rate=processor.feature_extractor.sampling_rate)
+inputs.to(device=model.device, dtype=model.dtype)
+
+outputs = model(**inputs)
+print("Loss:", outputs.loss.item())
+outputs.loss.backward()
+```
+</details>
+
+For more details about usage, please refer to the [Transformers' documentation](https://huggingface.co/docs/transformers/en/model_doc/parakeet).
+
+
 ## <span style="color:#466f00;">Software Integration:</span>
 
 **Runtime Engine(s):**
@@ -963,7 +1085,7 @@ NVIDIA NIM for v2 parakeet model is available at [https://build.nvidia.com/nvidi
 
 **Hardware Specific Requirements:**
 
-Atleast 2GB RAM for model to load. The bigger the RAM, the larger audio input it supports.
+At least 2GB RAM for model to load. The bigger the RAM, the larger audio input it supports.
 
 #### Model Version