Nemotron-Cascade-2-30B-A3B — 5-bit MLX

MLX quantization of nvidia/Nemotron-Cascade-2-30B-A3B.

Architecture: Hybrid Attention + Mamba (SSM) + MoE — 30B total parameters, 3B active
Quantization: Uniform 5-bit, group size 64
Size: ~21.7 GB

Quantization command

uvx --from "mlx-lm[train]" mlx_lm.convert \
  --hf-path nvidia/Nemotron-Cascade-2-30B-A3B \
  --mlx-path nemotron-cascade-30b-5bit \
  --quantize \
  --q-bits 5 \
  --q-group-size 64

Usage

uvx --from mlx-lm mlx_lm.chat \
  --model AdrienBrault/Nemotron-Cascade-2-30B-A3B-5bit-MLX

Downloads last month: 10

Safetensors

Model size

32B params

Tensor type

BF16

U32

F32

MLX

Hardware compatibility

5-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AdrienBrault/Nemotron-Cascade-2-30B-A3B-5bit-MLX

Base model

nvidia/Nemotron-Cascade-2-30B-A3B

Quantized

(30)

this model