---
license: apache-2.0
base_model: OpenMOSS-Team/MOSS-Music-8B-Thinking
language:
- en
- zh
tags:
- mlx
- audio
- music
- music-understanding
- audio-language-model
- moss
- moss-music
pipeline_tag: audio-text-to-text
---

# MOSS-Music-8B-Thinking · MLX 4-bit

[![Base model](https://img.shields.io/badge/Base-MOSS--Music--8B--Thinking-ff9e2c?logo=huggingface&logoColor=white)](https://huggingface.co/OpenMOSS-Team/MOSS-Music-8B-Thinking)
[![MLX](https://img.shields.io/badge/MLX-Apple%20Silicon-black)](https://github.com/ml-explore/mlx)
[![License](https://img.shields.io/badge/License-Apache--2.0-green)](https://www.apache.org/licenses/LICENSE-2.0)
[![Backend](https://img.shields.io/badge/Backend-moss__music__mlx-blue?logo=github)](https://github.com/OpenMOSS/MOSS-Music/pull/3)

A **4-bit [MLX](https://github.com/ml-explore/mlx) quantization** of
[OpenMOSS-Team/MOSS-Music-8B-Thinking](https://huggingface.co/OpenMOSS-Team/MOSS-Music-8B-Thinking)
for music understanding on Apple Silicon. The smallest build (~6 GB), a good fit for
16 GB Macs.

> Community conversion, not an official release. All model credit goes to the OpenMOSS Team.

Other sizes: [8-bit](https://huggingface.co/mlx-community/MOSS-Music-8B-Thinking-8bit) · [6-bit](https://huggingface.co/mlx-community/MOSS-Music-8B-Thinking-6bit)

## Usage

MOSS-Music is a custom multimodal (audio + text) model, so it does **not** load with
`mlx_lm` / `mlx_vlm` directly. Use the `moss_music_mlx` backend
([code](https://github.com/dthinkr/MOSS-Music/tree/feat/mlx-backend/mlx), [PR](https://github.com/OpenMOSS/MOSS-Music/pull/3)):

```python
from huggingface_hub import snapshot_download
from moss_music_mlx import load_pretrained, generate
from src.processing_moss_music import MossMusicProcessor

path = snapshot_download("mlx-community/MOSS-Music-8B-Thinking-4bit")
model = load_pretrained(path)
proc = MossMusicProcessor.from_pretrained(path, trust_remote_code=True, enable_time_marker=True)
print(generate(model, proc, "Analyze this track: genre, key, BPM, structure.", audio_path="song.mp3"))
```

## Conversion

- **4-bit, group size 64.** The audio encoder is kept at **bf16** to preserve audio
  fidelity; quantization is applied to the Qwen3 layers, token embeddings and `lm_head`.
- Converted with `mlx==0.31.2`, `mlx-lm==0.29.1`.

## Accuracy

Versus the fp32 PyTorch reference, the 4-bit model's prefill next-token argmax is
identical and the logits match to cosine **0.99889** (8-bit is 0.99999, 6-bit 0.99989).
4-bit is the most aggressive recipe; for the highest fidelity prefer 6-bit or 8-bit.

## License & credit

Apache-2.0, inherited from the base model. This repository provides only the
MLX-quantized weights; all credit goes to the OpenMOSS Team.