henrybravo/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit

This model is a clean republish of the verified local 4-bit MLX checkpoint for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.

Source checkpoint:

• original MLX repo: mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit
• quantization: 4-bit affine, group size 64
• local packaging optimization: removed .cache/huggingface before upload so the published repo only contains model assets
• local validation stack: mlx==0.31.0, mlx-lm==0.31.0, transformers==5.3.0

This was not re-quantized from the 8-bit checkpoint. Re-quantizing an already quantized model is unnecessary and risks avoidable quality loss.

Use with mlx

pip install -U mlx==0.31.0 mlx-lm==0.31.0

For OpenAI-compatible local serving and routing on top of MLX models, see mlx-router.

from mlx_lm import load, generate

model, tokenizer = load("henrybravo/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)