Upload README.md with huggingface_hub

README.md

@@ -1,3 +1,123 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+base_model: ibm-granite/granite-speech-4.1-2b-nar
+base_model_relation: quantized
+library_name: transformers
+pipeline_tag: automatic-speech-recognition
+tags:
+  - onnx
+  - onnxruntime
+  - speech
+  - asr
+  - automatic-speech-recognition
+  - granite
+  - ibm
+  - quantized
+  - int8
+  - non-autoregressive
+  - nar
+language:
+  - en
+---
+
+# IBM Granite Speech 4.1 2b NAR - ONNX export
+
+ONNX export of [`ibm-granite/granite-speech-4.1-2b-nar`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar) produced by Sam McLeod
+(<https://smcleod.net>). Repository: `smcleod/ibm-granite-speech-4.1-2b-nar-onnx`. Both FP32 and INT8 weight-only
+graphs are included. The graphs target opset 20, IR 10, `ai.onnx` operators
+only - no `com.microsoft` ops - so they load under the `ort` 2.0-rc.x Rust
+crate as well as standard `onnxruntime` 1.17 - 1.25.
+
+> **Additional precision tiers in progress.** A statically-calibrated INT8 variant (better quality vs the dynamic INT8 already in this repo) and a half-precision encoder are in active development. The repo will be updated when those graphs pass the multi-clip parity gate.
+
+Two graphs: `encoder.onnx` runs the conformer + CTC heads + BPE-collapsing projector and emits `bpe_logits_dense` plus pooled audio embeddings. `editor.onnx` runs the bidirectional NLE editor over a packed prompt (`<|audio|>` placeholders are replaced with the projector output) and emits per-position vocab logits. Decoding is a single argmax pass; no KV cache, no autoregression.
+
+## Files
+
+- `encoder.onnx` + `encoder.onnx_data` (FP32) and `encoder_int8.onnx` + `encoder_int8.onnx_data` (INT8 weight-only quantisation)
+- `editor.onnx` + `editor.onnx_data` (FP32) and `editor_int8.onnx` + `editor_int8.onnx_data` (INT8 weight-only quantisation)
+- Tokeniser / processor: `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `preprocessor_config.json`
+- Export scripts: `export_nar_encoder.py`, `export_nar_editor.py`, `quantise.py`
+- `granite_export_metadata.json` (graph IO, parity numbers, toolchain)
+- `LICENSE` (Apache 2.0)
+
+## Parity
+
+Parity is taken against the upstream PyTorch reference on a single LibriSpeech
+clip (`10226_10111_000000.wav`, 8.43 seconds, 844 mel frames). FP32 graphs
+match the reference within numeric tolerance; INT8 graphs are validated in
+argmax-only mode (logit values shift but token argmax is preserved, so the
+decoded transcript is unchanged).
+
+| graph | precision | max-abs-err | argmax mismatches | transcript match |
+| --- | --- | --- | --- | --- |
+| encoder (bpe_logits_dense) | FP32 | 0.00204 | 0/211 | n/a |
+| encoder (bpe_logits_dense) | INT8 | 1.84 | 0/211 | n/a |
+| editor | FP32 | 0.00147 | 0/257 | Y |
+| editor | INT8 | 94.5 | 15/257 | Y |
+
+INT8 note: the encoder graph emits two CTC heads. `bpe_logits_dense` (used downstream) holds argmax-stable through quantisation; `char_logits` (unused downstream) drifts noticeably and is not part of the inference path. The editor INT8 graph reproduces the reference transcript despite logit max-abs delta, because argmax decoding is invariant to the residual quant error.
+
+### Multi-clip transcript parity
+
+Three additional 16 kHz mono clips covering longer utterances (39 to 94 seconds), single and two-speaker conversational content. Word error rate (WER) and Levenshtein edit distance computed against the upstream PyTorch reference. Numbers measured end-to-end through the full ONNX pipeline (no PyTorch encoder fallback).
+
+| Clip | Duration | FP32 byte-exact vs PT | INT8 byte-exact vs PT | INT8 WER vs PT | INT8 vs FP32 Lev |
+| --- | ---: | :---: | :---: | ---: | ---: |
+| is-it-more-wood | 46.9 s | Y | N | 3.6% | 13 |
+| two-speakers-1 | 93.8 s | N | N | 3.5% | 26 |
+| two-speakers-2 | 38.8 s | Y | N | 2.0% | 10 |
+
+Raw multi-clip data including full transcripts: see `granite_export_metadata.json` `multi_clip_parity` block.
+
+Reference transcript:
+
+> after his nap timothy lazily stretched first one gray velvet foot then another strolled indolently to his plate turning over the food carefully selecting choice bits nosing out that which he scorned upon the clean hearth
+
+Both FP32 and INT8 paths reproduce this transcript exactly on the test clip.
+
+## Toolchain
+
+- transformers 5.8.0
+- torch 2.11.0
+- onnx 1.21.0
+- onnxruntime 1.25.1
+- exporter: torch.onnx.export TorchScript path (dynamo=False)
+- opset: 20 (`ai.onnx` only)
+- IR version: 10
+- external data layout: single `<stem>.onnx_data` sidecar per graph
+
+## Compatibility
+
+Targeted at the [`ort`](https://crates.io/crates/ort) 2.0-rc.x Rust crate.
+Compatible with `onnxruntime` Python 1.17 through 1.25. No `com.microsoft`
+ops are used. Graphs were emitted via the TorchScript path
+(`torch.onnx.export(..., dynamo=False)`); the dynamo exporter was deliberately
+avoided because it injects `aten::*` ops `ort` does not understand.
+
+## Reproducing the export
+
+The included scripts and `quantise.py` regenerate every artefact in this
+bundle. From a checkout of <https://github.com/sammcj/granite-speech-4.1-onnx>:
+
+```bash
+python export_nar_encoder.py \
+    --model-dir <path-to-ibm-granite/granite-speech-4.1-2b-nar> \
+    --out-dir exports/granite-speech-4.1-2b-nar
+python export_nar_editor.py \
+    --model-dir <path-to-ibm-granite/granite-speech-4.1-2b-nar> \
+    --out-dir exports/granite-speech-4.1-2b-nar
+python quantise.py --input exports/granite-speech-4.1-2b-nar/encoder.onnx --output exports/granite-speech-4.1-2b-nar/encoder_int8.onnx
+python quantise.py --input exports/granite-speech-4.1-2b-nar/editor.onnx  --output exports/granite-speech-4.1-2b-nar/editor_int8.onnx
+```
+
+Sandboxed environments may need:
+
+```bash
+HF_HOME=$TMPDIR/hf_home HF_MODULES_CACHE=$TMPDIR/hf_modules <command above>
+```
+
+## Licence
+
+Apache 2.0 for both the upstream IBM model and this ONNX export. See
+[`LICENSE`](LICENSE) for the full text.