---
license: mit
library_name: onnxruntime-web
pipeline_tag: audio-classification
tags:
  - audio
  - chord-recognition
  - music-information-retrieval
  - constant-q-transform
  - onnx
  - onnxruntime-web
  - webgpu
---

# ChordMini ChordNet (2E1D) — classifier + CQT plan (ONNX / WebGPU)

ONNX export of the **ChordMini** chord recognizer (ChordNet "2E1D", 170-class
large vocabulary), packaged for the [`musetric`][musetric] `packages/ai` runtime
(`onnxruntime-web` on **WebGPU**).

The graph is the **classifier only**: it takes log-CQT feature windows and
returns per-frame chord logits. Feature extraction is deliberately *not* baked
in — the host computes a recursive constant-Q transform on WebGPU and hands the
result over as a GPU buffer, so no features cross back to the CPU. This is not a
drop-in `audio -> chords` model.

```text
mono PCM @ 22050 Hz              (arithmetic-mean downmix — see Limitations)
  -> WebGPU recursive CQT        -> log(|CQT| + 1e-6) features [T, 144]
  -> pad/window                  -> [W, 108, 144]
  -> chordnet.onnx               -> logits [W, 108, 170]
  -> WebGPU smoothing + argmax   -> chord indices [T]
```

`cqt-plan.bin` ships with the model because it *defines* the features the graph
expects: the octave schedule, the sparse per-octave FFT basis and the resampling
FIR, baked from librosa 0.11.0. Model and plan are a matched pair — a release
therefore carries a hashable feature-extraction contract instead of an implicit
one.

Normalization (`(x - mean) / (std + 1e-8)`) is inside the graph. CQT, windowing,
smoothing and argmax stay in the host so their GPU buffers stay reusable.

## Intended uses & limitations

**Intended:**
- Chord recognition over music, as a stage in an audio pipeline.
- Client/edge inference via WebGPU through `onnxruntime-web`.

**Out of scope:**
- Standalone use without a host that computes librosa-equivalent log-CQT
  features, windows them to 108 frames, and applies smoothing + argmax to the
  logits (see `musetric` `packages/ai` and `packages/cqt`).
- Use in other training frameworks — this is an inference-only export.

**Limitations:**
- The 108-frame window and 144 CQT bins are fixed model contract; only the
  window count `W` is dynamic. Inputs shorter than 108 frames must be padded to
  one window and trimmed back.
- **The features must be librosa-equivalent.** Substituting a different CQT is
  not free: an nnAudio `CQT1992v2` stand-in correlates at ~0.998 yet still costs
  ~1.2% of frames end to end. Use the shipped plan.
- **The model is gain-sensitive.** It was trained on `librosa.load`'s arithmetic
  mean downmix `(L+R)/2`. `ffmpeg -ac 1` uses an energy-preserving rematrix
  `(L+R)/sqrt(2)`, i.e. a factor of √2, which `log(|CQT| + 1e-6)` turns into a
  constant `log(√2) = 0.347` offset on every feature — after `std = 1.719` a
  uniform `+0.20` shift, enough to flip frames near a decision boundary. Downmix
  as the arithmetic mean.
- Its `idx_to_chord` checkpoint map differs from the reference runner's
  `idx2voca_chord()` on 70 of 170 indices, in enharmonic spelling only
  (`Db:min` vs `C#:min`). `config.json` ships the runner's vocabulary.
- Training-data provenance of the upstream checkpoint is not documented here.

## How to use

The session runs the classifier; the host supplies `features` and consumes
`logits`.

```ts
import * as ort from 'onnxruntime-web/webgpu';
import { createCqt, verifyCqtPlanArtifact } from '@musetric/cqt/gpu';

const session = await ort.InferenceSession.create('chordnet.onnx', {
  executionProviders: ['webgpu'],
  preferredOutputLocation: { logits: 'gpu-buffer' },
});
const device = await ort.env.webgpu.device;

// cqt-plan.bin; verifies the payload against the SHA-256 it carries.
const plan = await verifyCqtPlanArtifact(new Uint8Array(planBytes));
const cqt = createCqt(device).get({ input: pcm, output: features, sampleCount, plan });
// cqt.run(encoder) writes log features [T, 144]; pad T up to a multiple of 108.

const input = ort.Tensor.fromGpuBuffer(paddedFeatures, {
  dataType: 'float32',
  dims: [windowCount, 108, 144],
});
const { logits } = await session.run({ features: input });
// logits: float32 [W, 108, 170] -> uniform 9-frame smoothing -> argmax -> indices
```

See the `musetric` `packages/ai` host code for the full CQT, smoothing/argmax and
segment-grouping pipeline.

## Files

| File | Size | SHA256 |
|---|---|---|
| `chordnet.onnx` | 9,604,664 B | `9a6570bf611cdc3f2c36286307af46fb94927fe7f6a2bc22a87c0ebf5f6c082e` |
| `config.json` | 3,009 B | `1f26c11ebea51ec08f12e813eb213a729fa0ecc407ac7632dfdc7bad67e65aa4` |
| `cqt-plan.bin` | 23,896 B | `c31f0a6fd2d582d753be6628b5daecdee58acba53cba93b2bc2b5c75dee2ba48` |
| `cqt-plan.manifest.json` | 1,721 B | `522b178e4f6e8ae5b6bf63b8e2f1a615fe2398592e27f7d9e3e219810081019f` |

`config.json` records the I/O contract, checkpoint normalization, the CQT
configuration and the 170-label vocabulary. `cqt-plan.manifest.json` records the
plan's generator, configuration and payload hash.

**Signature** — float32 weights, opset `ai.onnx` 17:

| Tensor | Type | Shape | Meaning |
|---|---|---|---|
| `features` (in) | float32 | `[W, 108, 144]` | unnormalized `log(\|CQT\| + 1e-6)` windows; 108 frames, 144 bins |
| `logits` (out) | float32 | `[W, 108, 170]` | per-frame chord logits, before smoothing and argmax |

**CQT plan** — `librosa 0.11.0`, `sr=22050`, `hop=2048`, `fmin=C1`, `n_bins=144`,
`bins_per_octave=24`, `norm=1`, `sparsity=0.01`, `window='hann'`, `scale=True`,
`pad_mode='constant'`; 6 octaves after one early downsample, 512-point FFT per
octave, resampler `kaiser-lowpass-255-cutoff-0.48-beta-12`.

## Validation

This export + the WebGPU CQT vs the PyTorch + `librosa.cqt` reference runner:

| Metric | Value |
|---|---|
| per-frame chord agreement (20 instrumental stems) | **1.0000** |
| exported logits vs Torch ChordNet, identical inputs | max abs error < `1e-4` |
| degenerate outputs | 0 |

Agreement is exact because the only approximation was removed. The predecessor
artifact baked the whole pipeline into one graph with nnAudio `CQT1992v2` in
place of `librosa.cqt`; that stand-in was the entire remaining gap (mean 0.9883,
worst 0.9410) and cost 70% of inference time and 37.8 of 47.4 MB. Reproducing
librosa's recursive per-octave transform on WebGPU fixed accuracy and size at
once.

Validate on the material fed in production — the **instrumental stem**.
Agreement measured on audio where the reference emits a near-constant label (for
example an isolated vocal, where "no chord" is correct on ~99% of frames)
carries no information: a stub returning that label scores just as well.
Re-run the parity gate on the exact published bytes before relying on it.

## Source & lineage

Code license and weight license are separate; ONNX conversion does not change the
weight license. Documented only as far as it is verifiable.

- **Architecture:** ChordNet "2E1D" — frequency encoder + time encoder +
  decoder, a small transformer (~2.3 M parameters).
- **Reference implementation and weights:** [`ptnghia-j/ChordMini`][upstream],
  **MIT** (per its `LICENSE`). Upstream publishes **no Hugging Face repo**, so
  the weights come from the GitHub repository rather than the Hub.
- **Checkpoint:** [`checkpoints/2e1d_model_best.pth`][ckpt] — 27,523,646 B, git
  blob `b61f6b3a02cc42b87afa38392f80d185a49f719a` — fetched at export time from
  [`raw.githubusercontent.com`][ckpt-raw]. That URL tracks `main` and upstream
  publishes no tagged release, so the fetch follows a moving branch; the blob
  hash above identifies what this export actually used.
- **Vendored code:** the inference subset lives under
  `musetric_toolkit/chords_audio/chordmini` in [musetric-toolkit][toolkit]; see
  its [`thirdPartyNotices.md`][notices].
- **Export tooling:** `scripts/onnx/chordmini` in [musetric-toolkit][toolkit].
- **Host runtime:** `packages/cqt` (the CQT) and `packages/ai` (the session and
  the smoothing/argmax passes) in [`musetric`][musetric].

This export preserves the upstream **MIT** license; we do not claim authorship of
the original weights.

## License

MIT, inherited from the upstream weights.

[upstream]: https://github.com/ptnghia-j/ChordMini
[ckpt]: https://github.com/ptnghia-j/ChordMini/blob/main/checkpoints/2e1d_model_best.pth
[ckpt-raw]: https://raw.githubusercontent.com/ptnghia-j/ChordMini/main/checkpoints/2e1d_model_best.pth
[toolkit]: https://github.com/popelenkow/musetric-toolkit
[notices]: https://github.com/popelenkow/musetric-toolkit/blob/main/thirdPartyNotices.md
[musetric]: https://github.com/popelenkow/musetric