model card = package README; error-tail figure

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+docs/buckeye_ccdf.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -3,121 +3,262 @@ license: other
 license_name: nyra-health-non-commercial-research-license
 license_link: LICENSE.md
 language: [en]
-library_name: nyra-align
+library_name: nyra-forced-aligner
 base_model: microsoft/wavlm-large
 tags: [forced-alignment, speech, phonetics, wavlm, verbatim, disfluency, timestamps]
 ---
 
-# nyra_forced_aligner_en
+# nyra-forced-aligner
 
-Verbatim-aware, noise-robust **forced aligner** for English: word and phone
-timestamps for conversational speech, including fillers (`[UH]`, `[UM]`),
-laughter, breaths and word cut-offs (`wor-*`). Runs **without Kaldi** through the
-[`nyra-forced-aligner`](https://github.com/nyrahealth/nyra-forced-aligner) package.
+[![Models](https://img.shields.io/badge/%F0%9F%A4%97%20models-nyralabs-yellow)](https://huggingface.co/nyralabs)
 
-This is the **standard** model, for clean and lightly noisy audio. A Pro model, more robust to noise and adverse acoustic conditions, is available on request.
+**The most accurate word timing you can get for conversational speech:
+verbatim-aware, noise-robust, and fast.**
 
-**This repository is self-contained**: it bundles the acoustic model (`model.npz`,
-`meta.json`, `lexicon.txt`, `phone_map.json`, ~3 MB) *and* the WavLM-large
-front-end weights (`wavlm/`), so one download gives you everything.
+[Blog post](https://nyra-labs.com/research/nyra-forced-aligner) ·
+[Paper](#) <!-- TODO: ICASSP 2026 link --> ·
+[Models](https://huggingface.co/nyralabs) ·
+[CrisperWhisper](https://github.com/nyrahealth/CrisperWhisper)
 
-## Usage
+A forced aligner answers one question: given a recording and its transcript,
+*when* was each word said? Most aligners were built for read speech and clean
+audio, and they quietly fall apart on the material people actually need to
+align: conversations full of `um`s, laughter, restarts and half-finished words,
+recorded in rooms that are not studios.
+
+`nyra-forced-aligner` was built for exactly that material.
+
+- **Vocal sounds are alignable.** Fillers (`[UH]`, `[UM]`), laughter, breaths,
+  coughs and word cut-offs (`th-`) are units of the model like any phone,
+  so they get timestamps of their own instead of being smeared into the
+  neighbouring words.
+- **Made for CrisperWhisper transcripts.** Trained on the verbatim
+  transcription convention of
+  [CrisperWhisper 2.0](https://github.com/nyrahealth/CrisperWhisper): feed
+  its output straight in and every filler, repetition and event token is
+  aligned.
+- **Accurate where it matters.** 19.9 ms mean word-boundary error on
+  conversational English (Buckeye): 21% lower than the best configuration of
+  the Montreal Forced Aligner, half the error of the best ASR-with-timestamps
+  system, and the only system whose error tail drops below 1% of words before
+  100 ms.
+- **Noise-robust.** Every utterance stays aligned down to 0 dB SNR, where
+  classical aligners lose or mangle most of them. A Pro model goes further for
+  difficult recordings.
+- **Fast and longform.** Batched WavLM features, GPU emissions and a compiled
+  beam Viterbi: eight minutes of audio align in about three seconds on one
+  GPU.
+
+## Install
+
+```bash
+pip install nyra-forced-aligner
+
+# espeak-ng is needed to pronounce words outside the built-in 41k-word lexicon:
+#   apt install espeak-ng      (macOS: brew install espeak-ng)
+```
+
+Runs on CPU, but a CUDA GPU is strongly recommended (WavLM-large front-end).
+The model is downloaded from the Hugging Face Hub on first use.
+
+## Quickstart
 
 ```python
 from nyra_align import Aligner
 
-aligner = Aligner()                              # this model is the default (downloads + caches)
-# aligner = Aligner("nyralabs/nyra_forced_aligner_en")       # equivalent, explicit repo id
-result  = aligner.align("interview.wav", "so i uhm [laughter] i went there")
+aligner = Aligner()                      # nyralabs/nyra_forced_aligner_en
+# aligner = Aligner(pro=True)            # noise-robust model
+
+result = aligner.align("interview.wav",
+                       "so i [UM] i went there on on thursday [laughter]")
 
 for w in result.words:
-    print(f"{w.word:>12s}  {w.start:7.3f} {w.end:7.3f}  {w.type}")   # type: w/f/s/c
-result.to_textgrid("interview.TextGrid")   # Praat
+    print(f"{w.start:7.3f} {w.end:7.3f}  {w.word:<12s} {w.type}")   # w / f(iller) / s(ound) / c(ut-off)
+
+result.to_textgrid("interview.TextGrid")     # Praat
 result.to_json("interview.json")
 ```
 
-While this repository is private, set `HF_TOKEN` (or `huggingface-cli login`) with access to the `nyralabs` org.
+Any audio length works. For the best timings follow the
+[transcript conventions](#transcript-conventions) below.
 
-Command line (single file, or an MFA-style corpus of paired `.wav` + `.txt`/`.lab`):
+Command line, single file or a whole corpus of paired `.wav` + `.txt`/`.lab`
+files (MFA-style layout):
 
 ```bash
-nyra-align audio.wav --text "full transcript ..." --out out.TextGrid
-nyra-align corpus_dir/ --out-dir aligned/ --format textgrid
+nyra-align interview.wav --text-file interview.txt --out interview.TextGrid
+nyra-align corpus_dir/ --pro --out-dir aligned/ --format textgrid
+```
+
+### Transcript conventions
+
+The aligner can only place what the transcript contains, so for the best
+timings the transcript should be **verbatim: write what you hear.**
+
+- **Cut-offs.** Interrupted words and word fragments are marked with a
+  trailing hyphen: `th-`, `w-`, `resched-`.
+- **Fillers.** Filled pauses are bracketed and upper-cased: `[UH]`, `[UM]`.
+- **Vocal sound events.** Non-speech sounds are bracketed: `[laughter]`,
+  `[cough]`, `[breath]`, `[sigh]`, `[sniff]`, `[lipsmack]`,
+  `[throatclearing]`, `[yawn]`, `[noise]`.
+- **Numbers, dates, times and emails** are written out exactly as spoken
+  (`March third at nine thirty`), not as digits or symbols.
+- **Everything else** is transcribed word for word. Normal casing and
+  punctuation are fine (the aligner ignores both); repetitions, false starts,
+  fillers, fragments and sounds are all left in place.
+
+```text
+so we we need to, to reschedule the th- Thursday meeting to [UH] March third at nine thirty [laughter]
 ```
 
-Transcripts may contain verbatim event tokens; out-of-lexicon words are
-phonemized with espeak-ng (`phonemizer`) through the same mapping used at
-training time. Long recordings are handled in one pass (8 min of audio in
-~3 s on one GPU; CPU works but is ~100x slower).
+What happens when a transcript deviates: a word that was spoken but is missing
+from the transcript gets absorbed into its neighbours, and a word in the
+transcript that was never spoken is squeezed into a few frames, so both shift
+the surrounding boundaries. Event tags are matched in any casing; a tag the
+model does not know (`[music]`) is left out and listed in `result.skipped`,
+as is any token that cannot be pronounced.
 
-## Model
+### With CrisperWhisper 2.0
 
-WavLM-large layer 23 (20 ms frames, upsampled to 10 ms) -> per-utterance CMVN ->
-supervised LDA applied **directly** to the 1024-dim WavLM features (no PCA, no
-frame splicing, 1024 -> 40) -> global MLLT -> triphone GMM-HMM (2500 tied states,
-6000 Gaussians, 55 phone classes = 39 speech phones + 15 vocal-event units + SIL).
-Trained on ~150 h of verbatim-transcribed conversational English.
+[CrisperWhisper 2.0](https://github.com/nyrahealth/CrisperWhisper) produces
+transcripts in exactly this convention, so its verbatim output can be aligned
+as is:
 
-The LDA projection is fitted on clean speech.
+```python
+from crisperwhisper import CrisperWhisperModel
+from nyra_align import Aligner
+
+audio = "interview.wav"
+transcript = CrisperWhisperModel("large").transcribe(audio, language="en").text
+result = Aligner().align(audio, transcript)
+
+for w in result.words:
+    print(f"{w.start:7.3f} {w.end:7.3f}  {w.word}")
+result.to_textgrid("interview.TextGrid")
+```
+
+### Models
+
+| Shorthand | Hugging Face ID | Use it for |
+|-----------|-----------------|------------|
+| `Aligner()` (default) | `nyralabs/nyra_forced_aligner_en` | Clean and lightly noisy audio; most accurate on clean speech |
+| `Aligner(pro=True)` | `nyralabs/nyra_forced_aligner_en_pro` | **Pro**: roughly the same accuracy on clean speech, meaningfully more robust to noise and adverse acoustic conditions. Not publicly released; available on request |
 
-## Results (word boundaries, mean of |onset| and |offset| error)
+Only English is available at the moment; `Aligner(language="de")` raises
+`NotImplementedError`.
 
-Full test sets; DP-matched canonical words; laughter tokens excluded from the
-word metric for every system; 100% of utterances aligned.
+## Performance
 
-| Test set | MAE | MedAE | mean IoU | acc <= 25 ms | acc <= 50 ms |
-|---|---:|---:|---:|---:|---:|
-| Buckeye (conversational, 80,475 words) | **19.9** | 13.9 | 0.830 | 79.5 | 93.7 |
-| TIMIT (read, 14,038 words) | 22.5 | 15.0 | 0.838 | 75.7 | 92.1 |
-| FluencyBank (stuttered, 5,487 words) | 77.1 | 25.0 | 0.743 | 58.0 | 82.4 |
+Mean absolute word-boundary error on **Buckeye** (conversational English,
+2,008 test recordings, 80,475 words), lower is better. Every system is scored
+with the same protocol: DP word matching on canonicalized forms, laughter
+tokens excluded, MAE = mean over words of (|onset error| + |offset error|)/2.
 
-Buckeye under added noise (MAE in ms; MUSAN = real noise recordings, Gauss = white noise):
+| # | System | Type | MAE (ms) | MedAE | ≤25 ms | ≤50 ms | ≤100 ms | mIoU |
+|--:|--------|------|---------:|------:|-------:|-------:|--------:|-----:|
+| 1 | **nyra_forced_aligner_en** | forced aligner | **19.9** | 13.9 | **79.5%** | **93.7%** | **98.3%** | **0.830** |
+| 2 | Montreal Forced Aligner `english_us_arpa`, wide beam¹ | forced aligner | 25.3 | 13.0 | 76.7% | 92.2% | 97.3% | 0.826 |
+| 3 | Montreal Forced Aligner `english_mfa`, wide beam¹ | forced aligner | 35.6 | 15.0 | 71.3% | 87.5% | 95.2% | 0.790 |
+| 4 | CrisperWhisper 2.0 | ASR | 38.7 | 25.0 | 54.2% | 78.7% | 91.7% | 0.717 |
+| 5 | Qwen3-ForcedAligner-0.6B | forced aligner | 39.5 | 25.0 | 52.8% | 85.9% | 96.6% | 0.711 |
+| 6 | xAI Grok Speech-to-Text | ASR | 45.5 | 34.5 | 58.3% | 82.8% | 94.0% | 0.630 |
+| 7 | Montreal Forced Aligner `english_us_arpa`, default beam | forced aligner | 48.0 | 13.5 | 75.2% | 90.4% | 95.6% | 0.810 |
+| 8 | Montreal Forced Aligner `english_mfa`, default beam | forced aligner | 48.9 | 15.5 | 70.7% | 86.7% | 94.3% | 0.782 |
+| 9 | MMS-FA (torchaudio) | forced aligner | 53.0 | 34.9 | 57.5% | 81.3% | 92.8% | 0.620 |
+| 10 | ElevenLabs Scribe v2 | ASR | 57.8 | 50.0 | 47.6% | 75.8% | 94.8% | 0.540 |
+| 11 | Charsiu | forced aligner | 70.3 | 28.0 | 51.8% | 68.5% | 85.1% | 0.649 |
+| 12 | NeMo Forced Aligner | forced aligner | 72.6 | 62.5 | 26.8% | 49.8% | 79.3% | 0.443 |
+| 13 | WhisperX (wav2vec2 alignment) | forced aligner | 80.9 | 61.5 | 35.1% | 68.1% | 94.1% | 0.456 |
+| 14 | CTC-Segmentation | forced aligner | 81.0 | 36.0 | 41.5% | 69.5% | 89.3% | 0.635 |
+| 15 | Deepgram Nova-3 | ASR | 86.9 | 71.0 | 18.1% | 36.4% | 68.9% | 0.496 |
+| 16 | Cartesia Ink-Whisper | ASR | 124.8 | 46.3 | 36.0% | 60.4% | 82.8% | 0.598 |
 
-| SNR | MUSAN 20 | 10 | 5 | 0 dB | Gauss 20 | 10 | 5 | 0 dB |
-|---|---:|---:|---:|---:|---:|---:|---:|---:|
-| this model | 20.0 | 22.2 | 26.9 | 56.3 | 20.2 | 25.3 | 38.9 | 110.9 |
-| Montreal Forced Aligner `english_us_arpa` (wide beam) | 33.3 | 65.3 | 127.2 | 281.2 | 36.2 | 74.4 | 156.3 | 416.9 |
+<sub>**MAE** and **MedAE** are over the per-word error (|onset error| +
+|offset error|)/2. The **≤ t ms** columns follow the convention of Rousso et
+al. (2024): the share of words whose *end* timestamp is within t of the
+reference (one boundary, not the per-word mean). **Forced aligners** are given
+the reference words and only predict their timing; every word is scored. **ASR** systems produce their own words, which
+are text-aligned to the reference, and only matched words are scored
+(88–93% of words), so their timing is judged independently of their
+transcription errors. ¹ MFA's shipped default beam (10) prunes the correct
+path on conversational speech; the "wide beam" rows use `beam 400 / retry
+4000`, the best configuration we found for it.</sub>
 
-For reference, MFA `english_us_arpa` scores 25.3 ms MAE on clean Buckeye with a
-widened search beam (49.0 ms at its shipped default beam).
+The mean hides where the difference really is. Averaged over all words, most
+systems are within a few tens of milliseconds; the gap is in the **tail**, the
+words that are misaligned by 100 ms or more, which is what breaks downstream
+use. Each curve shows the share of words whose per-word error (mean of the
+onset and offset errors, the quantity behind MAE) is larger than *t*; note
+this is not the single-boundary ≤ t ms column of the table:
+
+![Buckeye word-boundary error tails](https://huggingface.co/nyralabs/nyra_forced_aligner_en/resolve/main/docs/buckeye_ccdf.png)
+
+### Under noise
+
+The same Buckeye recordings with added noise, at signal-to-noise ratios from
+20 dB down to 0 dB (MAE in ms). MUSAN is real noise recordings; Gaussian is
+white noise, which the model never saw in training.
+
+| System | MUSAN 20 | 10 | 5 | 0 dB | Gauss 20 | 10 | 5 | 0 dB |
+|--------|---:|---:|---:|---:|---:|---:|---:|---:|
+| **nyra_forced_aligner_en** | 20.0 | 22.2 | 26.9 | 56.3 | 20.2 | 25.3 | 38.9 | 110.9 |
+| CrisperWhisper 2.0² | 38.7 | 42.4 | 48.0 | 55.7 | 40.1 | 50.9 | 67.3 | 103.4 |
+| Qwen3-ForcedAligner-0.6B | 40.6 | 54.3 | 78.7 | 153.9 | 43.4 | 85.2 | 153.9 | 339.2 |
+| MMS-FA (torchaudio) | 52.8 | 54.9 | 57.7 | 67.0 | 52.0 | 55.2 | 61.0 | 86.1 |
+| Montreal Forced Aligner `english_us_arpa`, wide beam | 33.3 | 65.3 | 127.2 | 281.2 | 36.2 | 74.4 | 156.3 | 416.9 |
+| WhisperX (wav2vec2 alignment) | 81.7 | 90.4 | 138.1 | 468.5 | 84.2 | 126.3 | 415.2 | 2720.6 |
+
+<sub>² CrisperWhisper 2.0 is an ASR system: at 0 dB it still recognises only
+29% of the words, and only those are scored. All forced aligners above keep
+100% of words at every SNR.</sub>
 
 `nyra_forced_aligner_en_pro` delivers roughly the same accuracy on clean speech
-and is meaningfully more robust to noise and adverse acoustic conditions; it is
-not publicly released and available on request.
+and is meaningfully more robust to noise and adverse acoustic conditions.
 
-## Limitations
+Results on read speech (TIMIT) and stuttered speech (FluencyBank), the full
+set of ablations, and how the models are trained are in the paper.
 
-- English only; the phone inventory and lexicon are espeak-based.
-- 10 ms frame grid; median boundary error is ~14 ms on conversational speech.
-- Like other GMM-HMM aligners, word boundaries adjacent to long pauses tend to
-  extend ~25 ms into the pause.
-- The transcript must be verbatim for best results; missing or extra words are
-  absorbed into neighbouring words.
+## What else is in the box
 
-## Files
+| Option | What it does |
+|--------|--------------|
+| `Aligner(pro=True)` | Noise-robust model |
+| `Aligner(device="cuda" / "cpu")` | Device selection (auto by default) |
+| `Aligner(precision="fp32")` | WavLM in fp32 instead of fp16; bit-exact parity with the reference implementation (fp16 moves ≤30 ms on ~0.5% of word boundaries) |
+| `Aligner(wavlm_batch=16)` | Batch size for the 30 s WavLM windows on long audio (tune to VRAM) |
+| `aligner.align_features(feats, text)` | Re-align a transcript against cached features (decode is ~6 ms per 3.5 s utterance) |
+| `result.events` | Just the filler / laughter / cut-off tokens |
+| `result.silences` | Silence segments between words |
 
-| file | content |
-|---|---|
-| `model.npz` | GMM parameters, transition model, triphone->pdf table, LDA/MLLT matrices |
-| `meta.json` | topology, phone table, feature chain, decoding config |
-| `lexicon.txt` | 41k-word pronunciation lexicon |
-| `phone_map.json` | espeak IPA -> phone mapping used for out-of-lexicon words |
-| `wavlm/` | WavLM-large weights (Microsoft; redistributed from `microsoft/wavlm-large`) |
+## How it works
 
-## License
+The full story is in the deep-dive post:
 
-The model files in this repository and **the Outputs they generate** (word and
-phone timestamps, alignments, segmentations) are released under the
-[nyra health Non-Commercial Research License](LICENSE.md): free for research
-and other non-commercial use. **Any commercial use of the model or of its
-Outputs — including using alignments or timestamps to train, fine-tune or
-improve models intended for commercial use — requires a commercial license**
-from nyra health GmbH (licensing@nyra-labs.com).
+- [The nyra forced aligner: precise word timing for real conversational speech](https://nyra-labs.com/research/nyra-forced-aligner).
+  Why classical aligners break on spontaneous and noisy speech, how frozen
+  WavLM features, vocal-event units and a noise-pooled projection fix it, and
+  how the model bootstraps itself from verbatim transcripts without any
+  manual boundary labels.
 
-The inference code (`nyra-forced-aligner`) is MIT-licensed. The `wavlm/`
-directory contains Microsoft's WavLM-large weights, redistributed unchanged
-under their own license.
+In short: WavLM-large hidden states are projected by a supervised LDA onto 40
+dimensions and modelled by a triphone GMM-HMM whose inventory holds 15
+vocal-event units next to the 39 English phones. At inference the package
+builds the alignment graph for the transcript (optional silence between words,
+full context across word boundaries) and runs a compiled beam Viterbi over the
+model's likelihoods, with a beam ladder so that no utterance is ever silently
+dropped. Words outside the built-in lexicon are phonemized with espeak through
+the same mapping used at training time.
+
+## License
 
-## Citation
+The model files are released under the
+[nyra health Non-Commercial Research License](https://huggingface.co/nyralabs/nyra_forced_aligner_en/blob/main/LICENSE.md):
+free for research and other non-commercial use. This license covers the
+model **and the alignments it produces**: any commercial use of the
+timestamps, and any use of them to train, fine-tune or otherwise improve
+models intended for commercial use, requires a commercial license. The Pro
+model is available under commercial license only. For commercial licensing
+of either, [contact nyra health](mailto:licensing@nyra-labs.com).
 
-Paper under review (ICASSP 2026). Until then please cite this repository.
+The model repositories bundle Microsoft's WavLM-large weights, redistributed
+unchanged under their upstream license.