Initial bf16 conversion of Kim Vocal 2 — 66.08 dB SDR parity vs PyTorch reference

LICENSE

@@ -0,0 +1,27 @@
+MIT License
+
+Copyright (c) 2026 Xocialize (MLX conversion and packaging)
+Copyright (c) 2025-2026 Kimberley Jensen (KimberleyJSN) — original Kim Vocal 2 weights
+
+The original Kim Vocal 2 weights at https://huggingface.co/KimberleyJSN/melbandroformer
+were initially released under GPL-3.0 on 2025-06-17 and relicensed to MIT on
+2026-04-22 (commit ac9b0614ab3cd7f77219e18ba494dfd93956c348). This MLX
+conversion was performed after the relicense and is redistributed under MIT.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,175 @@
+---
+license: mit
+library_name: mlx
+base_model: KimberleyJSN/melbandroformer
+pipeline_tag: audio-to-audio
+tags:
+- mlx
+- audio
+- audio-to-audio
+- source-separation
+- music-source-separation
+- vocal-separation
+- mel-band-roformer
+- roformer
+- apple-silicon
+---
+
+# mlx-community/mel-roformer-kim-vocal-2-mlx
+
+This model was converted to MLX format from [`KimberleyJSN/melbandroformer`](https://huggingface.co/KimberleyJSN/melbandroformer) using a custom Mel-Band-RoFormer MLX port (the [`Blaizzy/mlx-audio`](https://github.com/Blaizzy/mlx-audio) PR + the [`xocialize/mel-roformer-mlx-swift`](https://github.com/xocialize/mel-roformer-mlx-swift) Swift consumer). Refer to the [original model card](https://huggingface.co/KimberleyJSN/melbandroformer) for more details on the model.
+
+## Model
+
+- **Family:** Mel-Band-RoFormer (Lu, Wang, Won, "Mel-Band RoFormer for Music Source Separation," [arXiv:2310.01809](https://arxiv.org/abs/2310.01809))
+- **Checkpoint:** Kim Vocal 2 by Kimberley Jensen
+- **Parameters:** ~228M
+- **Sample rate:** 44100 Hz, stereo
+- **Stems produced:** `vocals` (single-stem model — derive `instrumental` as `mixture - vocals`)
+- **Chunk size:** 352800 samples (~8 s at 44.1 kHz), 50% overlap
+- **STFT:** `n_fft=2048`, `hop_length=441`, `win_length=2048`
+- **Transformer:** `dim=384`, `depth=6`, `heads=8`, `dim_head=64`
+- **Bands:** 60 mel bands
+- **Mask estimator depth:** 2
+
+Full hyperparameters in [`config.json`](config.json).
+
+## Source
+
+- **Original repository:** https://huggingface.co/KimberleyJSN/melbandroformer
+- **Original author:** Kimberley Jensen ([@KimberleyJSN](https://huggingface.co/KimberleyJSN))
+- **Original license:** MIT (relicensed from GPL-3.0 — see License provenance below)
+- **Original commit at conversion time:** [`ac9b061`](https://huggingface.co/KimberleyJSN/melbandroformer/commit/ac9b0614ab3cd7f77219e18ba494dfd93956c348)
+- **Source file:** `MelBandRoformer.ckpt`
+
+## License
+
+This redistribution is MIT-licensed, matching the original checkpoint license. See [`LICENSE`](LICENSE).
+
+### License provenance (Kim Vocal 2)
+
+This checkpoint was originally released under GPL-3.0 on June 17, 2025
+(see https://huggingface.co/KimberleyJSN/melbandroformer/discussions/2)
+and relicensed to MIT on April 22, 2026 by the original author Kimberley Jensen
+(commit https://huggingface.co/KimberleyJSN/melbandroformer/commit/ac9b0614ab3cd7f77219e18ba494dfd93956c348).
+This MLX conversion was made after the relicense and inherits the MIT terms.
+
+The relicense was independently confirmed with the original author the week of April 20, 2026 prior to this redistribution.
+
+| Date | Event | Reference |
+|---|---|---|
+| 2025-06-17 | Original license assigned: GPL-3.0 | [Discussion #2](https://huggingface.co/KimberleyJSN/melbandroformer/discussions/2) |
+| 2026-04-22 | Relicense to MIT (HuggingFace README updated, `license: mit`) | [Commit `ac9b061`](https://huggingface.co/KimberleyJSN/melbandroformer/commit/ac9b0614ab3cd7f77219e18ba494dfd93956c348) |
+| Week of 2026-04-20 | Direct confirmation with author | Private correspondence |
+| 2026-04-25 | Repo `license: mit` badge confirmed | [README front-matter](https://huggingface.co/KimberleyJSN/melbandroformer/blob/main/README.md) |
+
+## Conversion
+
+- **Tool:** `mlx_audio.sts.models.mel_roformer.convert` (custom port — pending upstream merge to [Blaizzy/mlx-audio](https://github.com/Blaizzy/mlx-audio))
+- **Tool version:** [`8380ab8`](https://github.com/xocialize/mlx-audio/commit/8380ab8) on the `feat/mel-band-roformer` branch (xocialize fork)
+- **mlx (Python) version:** 0.31.0
+- **Architecture preset:** `MelRoFormerConfig.kim_vocal_2()`
+- **Output precision:** `bfloat16`
+- **Source `MelBandRoformer.ckpt` SHA-256:** `87201f4d31afb5bc79993230fc49446918425574db48c01c405e44f365c7559e`
+- **Conversion date:** 2026-04-25
+
+## Parity
+
+Verified against the PyTorch reference implementation:
+
+- **SDR:** **66.08 dB** between PyTorch and MLX outputs (target: > 40 dB per the upload guide). The upload guide treats > 40 dB SDR as effectively bit-exact up to floating-point precision; 66 dB indicates the bf16 conversion is faithful to the lucidrains PyTorch reference.
+- **PyTorch reference:** [`bs_roformer==0.3.10`](https://pypi.org/project/BS-RoFormer/0.3.10/) — `bs_roformer.MelBandRoformer` instantiated from the original training YAML. Newer `bs_roformer` releases (0.4+) are **not** compatible with the Kim Vocal 2 checkpoint (the `layers` ModuleList nesting was reordered and nGPT-style normalization was added).
+- **Test signal:** 8-second stereo 44.1 kHz clip (mid-episode anime audio, mono → stereo duplication, music + dialogue mix).
+- **Reproduce:** see `mlx_audio/tests/sts/test_mel_roformer_parity.py` and `tests/sts/torch_infer.py` in the [xocialize/mlx-audio](https://github.com/xocialize/mlx-audio/tree/feat/mel-band-roformer) fork.
+
+## Intended use
+
+Vocal isolation for music source separation. Input is a stereo music mixture; output is a separated vocal stem. Trained for vocals; not validated for general-purpose source separation (drums, bass, other). Derive an instrumental stem as `mixture - vocals` if needed.
+
+## Files
+
+- `model.safetensors` — MLX weights
+- `config.json` — architecture hyperparameters
+- `LICENSE` — MIT license text
+
+## Usage
+
+### Python (mlx-audio)
+
+> Requires the Mel-Band-RoFormer architecture from the [pending mlx-audio PR](https://github.com/Blaizzy/mlx-audio); not yet on the released `pip install mlx-audio` line. Once merged, the snippet below works as-is. In the meantime, install the fork branch directly.
+
+```python
+import soundfile as sf
+import numpy as np
+import mlx.core as mx
+
+from mlx_audio.sts.models.mel_roformer import MelRoFormer, MelRoFormerConfig
+from mlx_audio.utils import load_audio
+
+# 1. Load model + weights from the Hub.
+model = MelRoFormer.from_pretrained(
+    "mlx-community/mel-roformer-kim-vocal-2-mlx",
+    config=MelRoFormerConfig.kim_vocal_2(),  # optional if config.json is present
+)
+model.eval()
+
+# 2. Load the input mixture as 44.1 kHz stereo and add a batch axis.
+mixture = load_audio("input_mixture.wav", sample_rate=44100)  # mx.array [2, samples]
+batched = mixture[None, ...]                                  # [1, 2, samples]
+
+# 3. Separate vocals.
+vocals = model(batched)[0]                                    # [2, samples]
+
+# 4. Derive instrumental as (mixture - vocals).
+instrumental = mixture - vocals
+
+# 5. Write stems to disk (soundfile expects [samples, channels]).
+sf.write("vocals.wav",       np.array(vocals).T,       44100)
+sf.write("instrumental.wav", np.array(instrumental).T, 44100)
+```
+
+For long inputs, chunk the audio at `chunk_size = 352800` samples with 50% overlap and overlap-add the outputs — see the model code for the canonical helper once it's added.
+
+### Swift ([`mel-roformer-mlx-swift`](https://github.com/xocialize/mel-roformer-mlx-swift))
+
+```swift
+import SwiftRoFormer
+
+// 1. Construct the model with the matching preset.
+let model = MelRoFormer(config: .kimVocal2)
+
+// 2. Load weights via the WeightLoader utility.
+//    The HuggingFace download path can be obtained with the swift-transformers
+//    Hub helpers, or by manually downloading model.safetensors first.
+try WeightLoader.loadWeights(
+    into: model,
+    from: weightsURL  // local URL to model.safetensors
+)
+```
+
+> A `MelRoFormer.fromPretrained(_:)` convenience that wraps Hub download + load is on the roadmap but not yet implemented in `mel-roformer-mlx-swift`. For now, download `model.safetensors` separately (e.g. via `huggingface-cli download`) and pass the local URL to `WeightLoader.loadWeights`.
+
+## Citation
+
+If you use this checkpoint, please cite the Mel-Band-RoFormer paper and the original Kim Vocal 2 release:
+
+```bibtex
+@misc{lu2023melband,
+  title         = {Mel-Band {RoFormer} for Music Source Separation},
+  author        = {Lu, Wei-Tsung and Wang, Ju-Chiang and Won, Minz and Choi, Keunwoo and Song, Xuchen},
+  year          = {2023},
+  eprint        = {2310.01809},
+  archivePrefix = {arXiv},
+  primaryClass  = {eess.AS},
+  url           = {https://arxiv.org/abs/2310.01809}
+}
+
+@misc{kim_vocal_2_2025,
+  title  = {Kim Vocal 2 — Mel-Band-RoFormer for vocal source separation},
+  author = {Jensen, Kimberley},
+  year   = {2025},
+  url    = {https://huggingface.co/KimberleyJSN/melbandroformer}
+}
+```
+
+The training-time configurations are from [ZFTurbo/Music-Source-Separation-Training](https://github.com/ZFTurbo/Music-Source-Separation-Training) (MIT). The MLX implementation lineage is [`lucidrains/BS-RoFormer`](https://github.com/lucidrains/BS-RoFormer) (MIT) → [`Blaizzy/mlx-audio`](https://github.com/Blaizzy/mlx-audio) (Apache-2.0).

config.json

@@ -0,0 +1,27 @@
+{
+  "_class": "MelRoFormerConfig",
+  "model_type": "mel_band_roformer",
+  "checkpoint_family": "kim_vocal_2",
+  "dim": 384,
+  "depth": 6,
+  "heads": 8,
+  "dim_head": 64,
+  "num_bands": 60,
+  "num_stems": 1,
+  "ff_mult": 4,
+  "mlp_expansion_factor": 4,
+  "mask_estimator_depth": 2,
+  "n_fft": 2048,
+  "hop_length": 441,
+  "win_length": 2048,
+  "sample_rate": 44100,
+  "chunk_size": 352800,
+  "num_overlap": 2,
+  "_dtype": "bfloat16",
+  "_mlx_version": "0.31.0",
+  "_tool_version": "8380ab8",
+  "_source_input": "MelBandRoformer.ckpt",
+  "_source_sha256": "87201f4d31afb5bc79993230fc49446918425574db48c01c405e44f365c7559e",
+  "_source_repository": "https://huggingface.co/KimberleyJSN/melbandroformer",
+  "_source_license_at_conversion": "MIT (relicensed from GPL-3.0 on 2026-04-22, commit ac9b0614ab3cd7f77219e18ba494dfd93956c348)"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:312c38e5b698f8dfaa4d6064e8f79010744825828917871a9d22673a43eb7fe5
+size 456483463