README.md

---
license: other
license_name: openmdw-1.1
license_link: https://openmdw.ai/license/1-1/
library_name: coreai
pipeline_tag: automatic-speech-recognition
language:
  - multilingual
tags:
  - coreai
  - apple-silicon
  - streaming-asr
  - automatic-speech-recognition
  - fastconformer
  - rnnt
  - nemo
  - ios
  - macos
base_model: nvidia/nemotron-3.5-asr-streaming-0.6b
---

# Nemotron 3.5 ASR Streaming 0.6B for CoreAI

This repository contains a CoreAI conversion of
[`nvidia/nemotron-3.5-asr-streaming-0.6b`](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b),
portable source assets, and device-architecture-specific AOT artifacts for iOS
27+ and macOS 27+.

Use the companion Swift package:
[`mweinbach/NemotronCoreAI`](https://github.com/mweinbach/NemotronCoreAI).

## Streaming behavior

The runtime preserves the model's cache-aware streaming contract rather than
re-running whole utterances:

- incremental centered-STFT/log-mel frontend;
- persistent 24-layer encoder channel and time caches;
- persistent two-layer RNNT predictor hidden and cell state;
- greedy RNNT decoding with complete-prefix partial transcripts;
- packet-stable 44.1/48 kHz to 16 kHz resampling in Swift;
- explicit final padding and terminal sentinel at end of stream.

Source assets expose 80, 160, 320, 560, and 1120 ms encoder modes. AOT assets
specialize the balanced 320 ms mode.

## Artifact matrix

| Platform | Architecture | Hardware | GPU AOT |
| --- | --- | --- | --- |
| macOS | `h15d` | M3 Ultra | yes |
| macOS | `h16g` | M4 | yes |
| macOS | `h17g` | M5 | yes |
| macOS | `h17s` | M5 Pro | yes |
| macOS | `h17c` | M5 Max | yes |
| iOS | `h18p` | A19 / A19 Pro family | yes |

The Swift loader checks `AIModel.deviceArchitectureName`, platform, compute
preference, and `main.hash` before loading an AOT asset. A missing or future
architecture can use a source `.aimodel` and CoreAI's device cache instead.

The Python runtime applies the same GPU/ANE guard. It detects known macOS chip
names or accepts an explicit architecture override, validates `main.hash`, and
falls back from a missing or failed GPU AOT load to source specialization.

Artifacts under `aot/macos/` and `aot/ios/` are not interchangeable even when
an architecture identifier appears on both platforms.

## Compute choices

- **GPU:** Published AOT artifacts were compiled with
  `--preferred-compute gpu` and are the validated low-latency path.
- **Automatic:** Portable source assets work with CoreAI's default compute
  policy. On the tested M3 Ultra, the current graph resolves to the same
  MPSGraph/GPU implementation as explicit GPU.
- **Neural Engine:** Not falsely advertised as supported. With Xcode 27 beta,
  forced ANE inference aborts and AOT compilation fails verification on a
  rank-6 transpose generated around the cache-aware encoder. The Swift API
  reserves the ANE lane but refuses it unless a future ANE-authored artifact is
  present.

## Download one target

Install the current Hugging Face CLI, then download only the manifest, runtime
contract, and architecture needed by the device.

```bash
hf download mweinbach/nemotron-3.5-asr-streaming-0.6b-coreai \
  --local-dir ./NemotronModel \
  --include package-manifest.json \
  --include runtime-support.json \
  --include 'aot/macos/gpu/*h15d.aimodelc/**'
```

For iPhone 17, iPhone Air, iPhone 17 Pro, and iPhone 17 Pro Max, use:

```bash
hf download mweinbach/nemotron-3.5-asr-streaming-0.6b-coreai \
  --local-dir ./NemotronModel \
  --include package-manifest.json \
  --include runtime-support.json \
  --include 'aot/ios/gpu/*h18p.aimodelc/**'
```

For forward-compatible source specialization, download a source directory and
pass its manifest key as `sourceVariant`:

```bash
hf download mweinbach/nemotron-3.5-asr-streaming-0.6b-coreai \
  --local-dir ./NemotronModel \
  --include package-manifest.json \
  --include runtime-support.json \
  --include 'deployment-lut-quality/**'
```

## Swift example

```swift
import NemotronCoreAI

let session = try await NemotronCoreAI.loadSession(
    packageURL: modelDirectory,
    latencyMS: 320,
    computePreference: .gpu
)

try await session.beginPCMStream()
let partial = try await session.pushPCM(packet, sampleRate: 48_000)
let final = try await session.finishPCMStream()
print(partial.text, final.text)
```

## Python service

The included FastAPI service exposes batch transcription and a stateful
WebSocket streaming endpoint while sharing one serialized CoreAI dispatcher.

```bash
PYTHONPATH=src python -m nemotron_coreai.service . \
  --compute gpu --architecture h15d --host 127.0.0.1 --port 8000
```

`--architecture` is optional on a recognized macOS chip and useful for explicit
deployment configuration. `--compute automatic` uses portable source
specialization. `--compute neural-engine` is rejected before CoreAI load until
the package contains an exact validated ANE AOT artifact.

## Source variants

| Manifest key | Approximate size | Intended use |
| --- | ---: | --- |
| `fp16` | 1.19 GiB | full five-mode reference |
| `fp16_320ms` | 1.19 GiB | AOT-compatible 320 ms fallback |
| `lut_quality` | 663 MiB | LUT8 portable quality/size balance |
| `lut_compact` | 538 MiB | LUT6 group-4 compact deployment |
| `int8` | 664 MiB | compatibility comparison |

See `reports/` and `docs/VALIDATION.md` for numerical and streaming validation.

## License and attribution

The model materials and compiled derivatives are distributed under
OpenMDW-1.1. `MODEL_LICENSE` contains the complete retained license, and
`NOTICE` retains NVIDIA's copyright and source attribution. Runtime and
conversion software is MIT-licensed.