Instructions to use mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

Ollama
How to use mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF with Ollama:
```
ollama run hf.co/mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF with Docker Model Runner:
```
docker model run hf.co/mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M
```

Lemonade

How to use mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

Nemotron-3-Super — Q4_K_M GGUF with YaRN 1M (Ollama / llama.cpp)

Community llama.cpp / Ollama pack derived from the Ollama library nemotron-3-super Q4_K_M weights (NVIDIA Nemotron 3 Super, ~120B total / ~12B active MoE).

This is not an official NVIDIA Transformers / NIM / NVFP4 safetensors checkpoint. Official weights live under nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 (and sibling FP8/BF16 repos). This repo is a Q4_K_M GGUF with metadata-only YaRN so runners that honor n_ctx_train accept 1,048,576 tokens.

	Official NVIDIA Hub (e.g. NVFP4)	This repo
Format	Safetensors / NVIDIA stacks	GGUF (`nemotron_h_moe`)
Quant	NVFP4 / FP8 / BF16	Q4_K_M (from Ollama library)
Context in file	Up to 1M (NVIDIA card)	Library GGUF advertised 262144 → we set 1048576 + YaRN KV
Typical serve	NIM, vLLM, Transformers	llama.cpp, Ollama

License: NVIDIA Nemotron Open Model License (same family as the official Super card). The LICENSE file is the NVIDIA Open Model License Agreement text (Last Modified: October 24, 2025). See also license link.

What you get

File	Role	Size
`Nemotron-3-Super-Q4_K_M-yarn1m-00001-of-00003.gguf`	Shard 1/3 (Hub <50 GB limit)	~39 GiB
`Nemotron-3-Super-Q4_K_M-yarn1m-00002-of-00003.gguf`	Shard 2/3	~39 GiB
`Nemotron-3-Super-Q4_K_M-yarn1m-00003-of-00003.gguf`	Shard 3/3	~7 GiB
`Modelfile`	Ollama example (`FROM` first shard, `num_ctx` 1M, `RENDERER`/`PARSER` `nemotron-3-nano`)	tiny

The single ~87 GiB file exceeds Hugging Face’s 50 GB per-file limit, so this pack is llama-gguf-split --split-max-size 40G. Ollama / llama.cpp load the set from the first shard path.

1M context is not extra trained weights. Tensors are unchanged from the Ollama Q4_K_M blob. Only GGUF key-values were rewritten:

Key	Value
`nemotron_h_moe.context_length`	`1048576`
`nemotron_h_moe.rope.scaling.type`	`yarn`
`nemotron_h_moe.rope.scaling.factor`	`4.0`
`nemotron_h_moe.rope.scaling.original_context_length`	`262144`

Sparse attention keeps 1M KV cheap on unified-memory hosts (~~8 GB KV class + ~81–87 GB weights ≈ **~~94–95 GB** resident on a DGX Spark when fully allocated).

Why this pack exists

NVIDIA’s Super card targets up to 1M context. The published Ollama library GGUF only advertised 262144, and stock Ollama caps num_ctx at GGUF n_ctx_train. Setting PARAMETER num_ctx 1048576 alone left ollama show at 262K.

This pack patches KV (same pattern as our Qwen3.8 NVFP4+VL YaRN pack) so Ollama / llama.cpp will allocate the full window.

Validated on DGX Spark (GB10, 128 GB unified)

Host: Docker ollama/ollama (~0.32.x), tag nemotron-3-super:1m, Spark OLLAMA_CONTEXT_LENGTH left at 65536 (fallback only — Modelfile pins 1M).

Check	Result
`ollama show` context length	1048576
Default load (`ollama ps` CONTEXT)	1048576 / ~94 GB
Needle ~8k / 64k / 256k / 512k	PASS
Single-shot needle ~980k tokens	PASS (~42 min, ~388 t/s prompt)

Secret used for the full-depth run: SECRET_CODE_NEMO_SUPER_1M_FULL_9917 (recalled correctly). Global env was not raised to 1M; precedence is request → Modelfile → OLLAMA_CONTEXT_LENGTH.

Use

Needs a recent Ollama / llama.cpp build that understands nemotron_h_moe. Keep RENDERER/PARSER nemotron-3-nano for tools + thinking.

Ollama (after `hf download`)

hf download mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF --local-dir ./nemotron-3-super-yarn1m
cd ./nemotron-3-super-yarn1m
# Modelfile FROM points at shard 00001-of-00003; keep all three shards together
ollama create nemotron-3-super:1m -f Modelfile
ollama run nemotron-3-super:1m

API smoke

curl -s localhost:11434/api/chat -d '{
  "model": "nemotron-3-super:1m",
  "stream": false,
  "think": false,
  "options": {"num_predict": 64},
  "messages": [{"role": "user", "content": "Hello"}]
}'

Leaving Open WebUI num_ctx as Default uses Modelfile 1M. Do not raise a global Ollama OLLAMA_CONTEXT_LENGTH to 1M just for this tag — that hits every unpinned model.

Recreate YaRN GGUF from stock Q4

If you still have the library blob (or re-ollama pull nemotron-3-super:latest):

python scripts/nemotron-super-yarn-1m-rewrite.py \
  /path/to/stock-q4.gguf \
  ./Nemotron-3-Super-Q4_K_M-yarn1m.gguf

Honesty checklist

Do not claim official NVIDIA NVFP4 / FP8 / BF16 safetensors.
Do not claim we re-quantized from Hub weights; language tensors came from the Ollama library Q4_K_M blob, then KV rewrite only.
Do not claim Modelfile-only num_ctx is enough without GGUF n_ctx_train ≥ 1M.
Do claim DGX Spark validation: 1M KV alloc + needle through ~512k and a single-shot ~980k recall PASS (2026-08-23).

Attribution

Model family: NVIDIA Nemotron 3 Super (tech report, Hub NVFP4)
Q4_K_M distribution: ollama.com/library/nemotron-3-super
YaRN 1M GGUF rewrite + Spark validation: mixbits (this repo)

Downloads last month: 27

GGUF

Model size

124B params

Architecture

nemotron_h_moe

Hardware compatibility

4-bit

Model tree for mixbits/Nemotron-3-Super-Q4_K_M-YaRN-1M-GGUF

Base model

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4

Quantized

(13)

this model