---
library_name: mlx
license: other
license_name: nvidia-open-model-license
base_model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
tags:
  - mlx
  - nemotron
  - mamba
  - moe
  - 6bit
  - quantized
  - apple-silicon
---

# Nemotron-3-Super-120B-A12B — MLX 6-bit

MLX-format 6-bit quantization of [nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16) for Apple Silicon.

## Model Details

| | |
|---|---|
| **Architecture** | Hybrid Mamba-2 + Transformer Attention + Latent MoE |
| **Parameters** | 120B total, 12B active per token |
| **Context length** | 1M tokens |
| **Quantization** | 6-bit (6.507 BPW) |
| **Size on disk** | ~92 GB |
| **Source** | `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16` |
| **Converted with** | `mlx-lm 0.31.2` (git main) |

## Performance

Tested on Apple M5 Pro Max (128GB unified memory):

| Metric | Value |
|---|---|
| Generation speed | 43.6 tok/s |
| Peak memory | 98.4 GB |

## Requirements

- Apple Silicon Mac with **≥128GB** unified memory
- `mlx-lm >= 0.31.2` — install from git main until a PyPI release includes MoE latent projection support:

```bash
pip install git+https://github.com/ml-explore/mlx-lm.git
```

## Usage

### CLI

```bash
mlx_lm.generate \
  --model FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit \
  --prompt "Hello, what can you help me with?"
```

### Python

```python
from mlx_lm import load, generate

model, tokenizer = load("FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)
```

### LM Studio

Search for `FF-01/Nemotron-3-Super-120B-A12B-MLX-6bit` and load with the MLX engine.

## Notes

- This is a **reasoning model** — it may output chain-of-thought before the final answer. Use the included chat template for proper formatting.
- The Latent MoE architecture requires `mlx-lm >= 0.31.2` for the `fc1_latent_proj` / `fc2_latent_proj` layer support ([mlx-lm #1016](https://github.com/ml-explore/mlx-lm/issues/1016)).