henrybravo/NVIDIA-Nemotron-3-Nano-30B-A3B-8bit

This model is a clean republish of the verified local 8-bit MLX checkpoint for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.

Source checkpoint:

• original MLX repo: mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-8Bit
• quantization: 8-bit affine, group size 64
• packaging cleanup: excluded .cache/huggingface, blobs/, refs/, and snapshots/ from the publish folder because the top-level model assets were already complete
• strict JSON compatibility fix: removed the non-standard time_step_limit: [0.0, Infinity] field so config.json validates with standard JSON tooling while preserving upstream default behavior
• local validation stack: mlx==0.31.0, mlx-lm==0.31.0, transformers==5.3.0

Use with mlx

pip install -U mlx==0.31.0 mlx-lm==0.31.0

For OpenAI-compatible local serving and routing on top of MLX models, see mlx-router.

from mlx_lm import load, generate

model, tokenizer = load("henrybravo/NVIDIA-Nemotron-3-Nano-30B-A3B-8bit")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)