Upload folder using huggingface_hub

README.md

@@ -1,4 +1,7 @@
 ---
+base_model:
+- nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8
+library_name: transformers
 license: other
 license_name: nvidia-open-model-license
 license_link: >-
@@ -14,6 +17,7 @@ language:
 - zh
 tags:
 - nvidia
+- unsloth
 - pytorch
 - nemotron-3
 - latent-moe
@@ -22,11 +26,30 @@ datasets:
 - nvidia/nemotron-post-training-v3
 - nvidia/nemotron-pre-training-datasets
 track_downloads: true
-base_model:
-- nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
 ---
+> [!NOTE]
+>  Includes Unsloth **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
+>
+
+<div>
+<p style="margin-top: 0;margin-bottom: 0;">
+    <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
+  </p>
+  <div style="display: flex; gap: 5px; align-items: center; ">
+    <a href="https://github.com/unslothai/unsloth/">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
+    </a>
+    <a href="https://discord.gg/unsloth">
+      <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
+    </a>
+    <a href="https://docs.unsloth.ai/">
+      <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
+    </a>
+  </div>
+</div>
 
-# NVIDIA-Nemotron-3-Super-120B-A12B-BF16
+
+# NVIDIA-Nemotron-3-Super-120B-A12B-FP8
 
 <div align="center" style="line-height: 1;">
 <a href="https://build.nvidia.com/nvidia/nemotron-3-super-120b-a12b" target="_blank" style="margin: 2px;">
@@ -66,7 +89,7 @@ base_model:
 | **Total Parameters** | 120B (12B active) |
 | **Architecture** | LatentMoE - Mamba-2 + MoE + Attention hybrid with Multi-Token Prediction (MTP) |
 | **Context Length** | Up to 1M tokens |
-| **Minimum GPU Requirement** | 8× H100-80GB |
+| **Minimum GPU Requirement** | 2× H100-80GB |
 | **Supported Languages** | English, French, German, Italian, Japanese, Spanish, Chinese |
 | **Best For** | Agentic workflows, long-context reasoning, high-volume workloads (e.g. IT ticket automation), tool use, RAG |
 | **Reasoning Mode** | Configurable on/off via chat template (`enable_thinking=True/False`) |
@@ -97,7 +120,7 @@ NVIDIA Nemotron™ is a family of open models with open weights, training data,
 
 ## Description
 
-**Nemotron-3-Super-120B-A12B-BF16** is a large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is optimized for collaborative agents and high-volume workloads such as IT ticket automation. Like other models in the family, it responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response. The model's reasoning capabilities can be configured through a flag in the chat template.
+**Nemotron-3-Super-120B-A12B-FP8** is a large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is optimized for collaborative agents and high-volume workloads such as IT ticket automation. Like other models in the family, it responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response. The model's reasoning capabilities can be configured through a flag in the chat template.
 
 The model employs a hybrid **Latent Mixture-of-Experts (LatentMoE)** architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Distinct from the Nano model, the Super model incorporates **Multi-Token Prediction (MTP)** layers for faster text generation and improved quality, and it is trained using **NVFP4** quantization to maximize compute efficiency. The model has **12B active parameters** and **120B parameters in total**.
 
@@ -113,46 +136,36 @@ This model is ready for commercial use.
 
 ### Benchmarks 
 
-| **Benchmark** | **Nemotron 3 Super** | **Qwen3.5-122B-A10B** | **GPT-OSS-120B** |
+| Benchmark | Nemotron-3-Super | Nemotron-3-Super FP8 | Nemotron-3-Super NVFP4 |
 |---|---|---|---|
 | **General Knowledge** | | | |
-| MMLU-Pro | 83.73 | 86.70 | 81.00 |
+| MMLU-Pro | 83.73 | 83.63 | 83.33 |
 | **Reasoning** | | | |
-| AIME25 (no tools) | 90.21 | 90.36 | 92.50 |
-| HMMT Feb25 (no tools) | 93.67 | 91.40 | 90.00 |
-| HMMT Feb25 (with tools) | 94.73 | 89.55 | — |
-| GPQA (no tools) | 79.23 | 86.60 | 80.10 |
-| GPQA (with tools) | 82.70 | — | 80.09 |
-| LiveCodeBench (v5 2024-07↔2024-12) | 81.19 | 78.93 | 88.00 |
-| SciCode (subtask) | 42.05 | 42.00 | 39.00 |
-| HLE (no tools) | 18.26 | 25.30 | 14.90 |
-| HLE (with tools) | 22.82 | — | 19.0 |
+| HMMT Feb25 (with tools) | 94.73 | 94.38 | 95.36 |
+| GPQA (no tools) | 79.23 | 79.36 | 79.42 |
+| LiveCodeBench (v6 2024-08↔2025-05) | 78.69 | 78.44 | 78.44 |
+| LiveCodeBench (v5 2024-07↔2024-12) | 81.19 | 80.99 | 80.56 |
+| SciCode (subtask) | 42.05 | 41.38 | 40.83 |
+| HLE (no tools) | 18.26 | 17.42 | 17.42 |
 | **Agentic** | | | |
-| Terminal Bench (hard subset) | 25.78 | 26.80 | 24.00 |
-| Terminal Bench Core 2.0 | 31.00 | 37.50 | 18.70 |
-| SWE-Bench (OpenHands) | 60.47 | 66.40 | 41.9 |
-| SWE-Bench (OpenCode) | 59.20 | 67.40 | — |
-| SWE-Bench (Codex) | 53.73 | 61.20 | — |
-| SWE-Bench Multilingual (OpenHands) | 45.78 | — | 30.80 |
+| Terminal Bench (hard subset) | 25.78 | 26.04 | 24.48 |
 | **TauBench V2** | | | |
-| &nbsp;&nbsp;&nbsp;&nbsp;Airline | 56.25 | 66.0 | 49.2 |
-| &nbsp;&nbsp;&nbsp;&nbsp;Retail | 62.83 | 62.6 | 67.80 |
-| &nbsp;&nbsp;&nbsp;&nbsp;Telecom | 64.36 | 95.00 | 66.00 |
-| &nbsp;&nbsp;&nbsp;&nbsp;Average | 61.15 | 74.53 | 61.0 |
-| BrowseComp with Search | 31.28 | — | 33.89 |
-| BIRD Bench | 41.80 | — | 38.25 |
+|   Airline | 56.25 | 56.25 | 54.75 |
+|   Retail | 62.83 | 63.05 | 63.38 |
+|   Telecom | 64.36 | 63.93 | 63.27 |
+|   Average | 61.15 | 61.07 | 60.46 |
 | **Chat & Instruction Following** | | | |
-| IFBench (prompt) | 72.56 | 73.77 | 68.32 |
-| Scale AI Multi-Challenge | 55.23 | 61.50 | 58.29 |
-| Arena-Hard-V2 | 73.88 | 75.15 | 90.26 |
+| IFBench (prompt) | 72.58 | 72.32 | 73.30 |
+| Scale AI Multi-Challenge | 55.23 | 54.35 | 52.8 |
+| Arena-Hard-V2 (Hard Prompt) | 73.88 | 76.06 | 76.00 |
 | **Long Context** | | | |
-| AA-LCR | 58.31 | 66.90 | 51.00 |
-| RULER @ 256k | 96.30 | 96.74 | 52.30 |
-| RULER @ 512k | 95.67 | 95.95 | 46.70 |
-| RULER @ 1M | 91.75 | 91.33 | 22.30 |
+| AA-LCR | 58.31 | 57.69 | 58.06 |
+| RULER-500 @ 128k (500 samples per task) | 96.79 | 96.85 | 95.99 |
+| RULER-500 @ 256k (500 samples per task) | 96.60 | 96.33 | 96.52 |
+| RULER-500 @ 512k (500 samples per task) | 96.09 | 95.66 | 96.23 |
 | **Multilingual** | | | |
-| MMLU-ProX (avg over langs) | 79.36 | 85.06 | 76.59 |
-| WMT24++ (en→xx) | 86.67 | 87.84 | 88.89 |
+| MMLU-ProX (avg over languages) | 79.35 | 79.21 | 79.37 |
+
 
 All evaluation results were collected via [Nemo Evaluator SDK](https://github.com/NVIDIA-NeMo/Evaluator) and for most benchmarks, the [Nemo Skills Harness](https://github.com/NVIDIA-NeMo/Skills). For reproducibility purposes, more details on the evaluation settings can be found in the [Nemo Evaluator SDK configs folder](https://github.com/NVIDIA-NeMo/Evaluator/tree/main/packages/nemo-evaluator-launcher/examples/nemotron/nemotron-3-super) and the [reproducibility tutorial for Nemotron 3 Super](https://github.com/NVIDIA-NeMo/Evaluator/blob/main/packages/nemo-evaluator-launcher/examples/nemotron/nemotron-3-super/reproducibility.md). The open source container on Nemo Skills packaged via NVIDIA's Nemo Evaluator SDK used for evaluations can be found [here](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/eval-factory/containers/nemo_skills). In addition to Nemo Skills, the evaluations also used dedicated open-source packaged containers for Tau-2 Bench (default prompt), Terminal Bench Hard (48 tasks), ScaleAI Multi Challenge Multi-turn Instruction Following, and Ruler. 
 
@@ -162,11 +175,11 @@ The following benchmarks are not onboarded yet in our open source tools and for
 
 ### Use Case
 
-NVIDIA-Nemotron-3-Super-120B-A12B-BF16 is a general purpose reasoning and chat model intended to be used in English, Code, and supported multilingual contexts. This model is optimized for collaborative agents and high-volume workloads. It is intended to be used by developers designing AI Agent systems, chatbots, RAG systems, and other AI-powered applications. This model is also suitable for complex instruction-following tasks and long-context reasoning.
+NVIDIA-Nemotron-3-Super-120B-A12B-FP8 is a general purpose reasoning and chat model intended to be used in English, Code, and supported multilingual contexts. This model is optimized for collaborative agents and high-volume workloads. It is intended to be used by developers designing AI Agent systems, chatbots, RAG systems, and other AI-powered applications. This model is also suitable for complex instruction-following tasks and long-context reasoning.
 
 ### Release Date
 
-Hugging Face - 03/11/2026 via [Hugging Face](https://huggingface.co/collections/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16)
+Hugging Face - 03/11/2026 via [Hugging Face](https://huggingface.co/collections/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8)
 
 ## Reference(s)
 
@@ -187,21 +200,22 @@ The model utilizes the **LatentMoE** architecture, where tokens are projected in
 
 Stage 1: Pre-Training
 
-* [NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16](https://huggingface.co/collections/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16) model was pre-trained for over 25T tokens using crawled and synthetic code, math, science, and general knowledge data. Training leveraged NVFP4 quantization for efficiency. All datasets are disclosed in the [Training and Evaluation Datasets](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16#training-and-evaluation-datasets) section of this document. Major portions of the pre-training corpus are released in the [Nemotron-Pre-Training-Datasets](https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets) collection.
+* [NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16](https://huggingface.co/collections/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16) model was pre-trained for over 25T tokens using crawled and synthetic code, math, science, and general knowledge data. Training leveraged NVFP4 quantization for efficiency. All datasets are disclosed in the [Training and Evaluation Datasets](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8#training-and-evaluation-datasets) section of this document. Major portions of the pre-training corpus are released in the [Nemotron-Pre-Training-Datasets](https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets) collection.
 * Software used for pre-training: [Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
 
 Stage 2: Supervised Fine-Tuning
 
-* The model was further fine-tuned on synthetic code, math, science, tool calling, instruction following, structured outputs, and general knowledge data. This stage incorporated data designed to support long-range retrieval and multi-document aggregation. All datasets are disclosed in the [Training and Evaluation Datasets](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16#training-and-evaluation-datasets) section of this document. Major portions of the fine-tuning corpus are released in the [Nemotron-Post-Training-v3](https://huggingface.co/collections/nvidia/nemotron-post-training-v3) collection. [Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner) is one of the libraries used to prepare these corpora.
+* The model was further fine-tuned on synthetic code, math, science, tool calling, instruction following, structured outputs, and general knowledge data. This stage incorporated data designed to support long-range retrieval and multi-document aggregation. All datasets are disclosed in the [Training and Evaluation Datasets](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8#training-and-evaluation-datasets) section of this document. Major portions of the fine-tuning corpus are released in the [Nemotron-Post-Training-v3](https://huggingface.co/collections/nvidia/nemotron-post-training-v3) collection. [Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner) is one of the libraries used to prepare these corpora.
 
 Stage 3: Reinforcement Learning
 
 * The model underwent multi-environment reinforcement learning using asynchronous GRPO (Group Relative Policy Optimization) across math, code, science, instruction following, multi-step tool use, multi-turn conversations, and structured output environments. It utilized an asynchronous RL architecture that fully decouples training from inference across separate GPU devices, leveraging in-flight weight updates and MTP to accelerate rollout generation. Conversational quality was further refined through RLHF. All datasets are disclosed in the *Training and Evaluation Datasets* section of this document. The RL environments and datasets are released as part of [NeMo Gym](https://github.com/NVIDIA-NeMo/Gym).
 * Software used for reinforcement learning: [NeMo RL](https://github.com/NVIDIA-NeMo/RL), [NeMo Gym](https://github.com/NVIDIA-NeMo/Gym)
 
-NVIDIA-Nemotron-3-Super-120B-A12B-BF16 model is a result of the above work.
+NVIDIA-Nemotron-3-Super-120B-A12B-FP8 model is a result of the above work.
 
 The end-to-end training recipe is available in the [NVIDIA Nemotron Developer Repository](https://github.com/NVIDIA-NeMo/Nemotron). Evaluation results can be replicated using the [NeMo Evaluator SDK](https://github.com/NVIDIA-NeMo/Evaluator). [Data Designer](https://github.com/NVIDIA-NeMo/DataDesigner) is one of the libraries used to prepare the pre and post training datasets. More details on the datasets and synthetic data generation methods can be found in the technical report [NVIDIA Nemotron 3 Super Technical Report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Super-Technical-Report.pdf).
+
 ## Input
 
 - **Input Type(s):** Text
@@ -235,7 +249,7 @@ The integration of foundation and fine-tuned models into AI systems requires add
 For each inference backend - we'll be using the custom `super_v3` reasoning parser - which you can obtain by following these instructions: 
 
 ```bash
-wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16/blob/main/super_v3_reasoning_parser.py
+wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8/blob/main/super_v3_reasoning_parser.py
 ```
 
 For advanced deployment configurations - visit [this resource](https://github.com/NVIDIA-NeMo/Nemotron/tree/main/usage-cookbook/Nemotron-3-Super/AdvancedDeploymentGuide)
@@ -256,21 +270,21 @@ vllm serve $MODEL_CKPT \
   --async-scheduling \
   --dtype auto \
   --kv-cache-dtype fp8 \
-  --tensor-parallel-size 4 \
+  --tensor-parallel-size 2 \
   --pipeline-parallel-size 1 \
-  --data-parallel-size 2 \
-  --max-model-len 262144 \
-  --enable-expert-parallel \
-  --attention-backend TRITON_ATTN \
+  --data-parallel-size 1 \
   --swap-space 0 \
   --trust-remote-code \
+  --attention-backend TRITON_ATTN \
   --gpu-memory-utilization 0.9 \
   --enable-chunked-prefill \
-  --mamba-ssm-cache-dtype float16 \
-  --reasoning-parser-plugin super_v3_reasoning_parser.py \
-  --reasoning-parser super_v3 \
+  --max-num-seqs 512 \
+  --host 0.0.0.0 \
+  --port 5000 \
   --enable-auto-tool-choice \
-  --tool-call-parser qwen3_coder
+  --tool-call-parser qwen3_coder \
+  --reasoning-parser-plugin "./super_v3_reasoning_parser.py" \
+  --reasoning-parser super_v3
 ```
 
 > Context length defaults to 256k above. To use up to 1M, set `VLLM_ALLOW_LONG_MAX_MODEL_LEN=1` and `--max-model-len 1M`
@@ -295,9 +309,12 @@ For more detailed information, please see [this cookbook](https://github.com/NVI
 python3 -m sglang.launch_server \
   --model PATH/TO/CHECKPOINT \
   --served-model-name nvidia/nemotron-3-super \
+  --host 0.0.0.0 \
+  --port 5000 \
+  --log-level warning \
   --trust-remote-code \
-  --tp 8 \
-  --ep 4 \
+  --tp 2 \
+  --ep 1 \
   --tool-call-parser qwen3_coder \
   --reasoning-parser nano_v3
 ```
@@ -316,22 +333,23 @@ For more detailed information, please see [this cookbook](https://github.com/NVI
 cat > ./extra-llm-api-config.yml << EOF
 kv_cache_config:
   enable_block_reuse: false
-  mamba_ssm_cache_dtype: float32
 moe_config:
-  backend: TRTLLM
+   backend: TRTLLM
 cuda_graph_config:
-  enable_padding: true
-  max_batch_size: 256
-enable_attention_dp: true
+    enable_padding: true
+    batch_sizes: [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
 EOF
+```
 
-trtllm-serve PATH/TO/BF16/CHECKPOINT \
+```bash
+mpirun -n 1 --allow-run-as-root --oversubscribe \
+trtllm-serve /PATH/TO/MODEL \
   --host 0.0.0.0 \
-  --port 8123 \
+  --port 8000 \
   --backend pytorch \
-  --max_batch_size 256 \
-  --tp_size 8 --ep_size 8 \
-  --max_num_tokens 8576 \
+  --max_batch_size 128 \
+  --tp_size 2 --ep_size 2 \
+  --max_num_tokens 16384 \
   --trust_remote_code \
   --reasoning_parser nano_v3 \
   --tool_parser qwen3_coder \
@@ -510,11 +528,11 @@ class ThinkingBudgetClient:
 client = ThinkingBudgetClient(
     base_url="http://localhost:8000/v1",
     api_key="EMPTY",
-    tokenizer_name_or_path="nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16",
+    tokenizer_name_or_path="nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8",
 )
 
 result = client.chat_completion(
-    model="nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16",
+    model="nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8",
     messages=[
         {"role": "system", "content": "You are a helpful assistant. /think"},
         {"role": "user", "content": "What is 2+2?"},
@@ -529,74 +547,6 @@ print(result)
 
 </details>
 
-#### Transformers
-The model has been integrated into 🤗 Transformers since v5.3.0. We recommend using the [Nemotron 3 Super](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo/tags?version=26.02.nemotron_3_super) container from the NeMo Framework to ensure all required libraries are available.
-
-```python
-import torch
-from transformers import AutoTokenizer, AutoModelForCausalLM
-
-tokenizer = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16")
-model = AutoModelForCausalLM.from_pretrained(
-    "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16",
-    torch_dtype=torch.bfloat16,
-    device_map="auto"
-)
-```
-
-If your Transformers version is lower than v5.3.0, please add `trust_remote_code=True` when loading the model:
-```python
-model = AutoModelForCausalLM.from_pretrained(
-    "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16",
-    torch_dtype=torch.bfloat16,
-    device_map="auto",
-    trust_remote_code=True
-)
-```
-
-Please note that the model supports up to a 1M context size, although the default context size in the Hugging Face configuration is 256k due to higher VRAM requirements.
-
-Here is an example of generating outputs with reasoning enabled (the default):
-```python
-messages = [
-    {"role": "user", "content": "Write a haiku about GPUs"},
-]
-
-tokenized_chat = tokenizer.apply_chat_template(
-    messages,
-    tokenize=True,
-    add_generation_prompt=True,
-    return_tensors="pt"
-).to(model.device)
-
-if not isinstance(tokenized_chat, torch.Tensor):
-    input_ids = tokenized_chat["input_ids"]
-else:
-    input_ids = tokenized_chat
-
-outputs = model.generate(
-    input_ids,
-    max_new_tokens=50,
-    temperature=1.0,
-    top_p=0.95,
-    eos_token_id=tokenizer.eos_token_id
-)
-
-print(tokenizer.decode(outputs[0]))
-```
-
-To disable reasoning, add `enable_thinking=False` to `apply_chat_template()`. By default, `enable_thinking` is set to `True`.
-```python
-tokenized_chat = tokenizer.apply_chat_template(
-    messages,
-    tokenize=True,
-    enable_thinking=False,
-    add_generation_prompt=True,
-    return_tensors="pt"
-).to(model.device)
-```
-
-
 ## Training and Evaluation Datasets
 
 # Training
@@ -611,9 +561,9 @@ tokenized_chat = tokenizer.apply_chat_template(
 **Data Collection Method by dataset:** Hybrid: Automated, Human, Synthetic
 **Labeling Method by dataset:** Hybrid: Automated, Human, Synthetic
 
-NVIDIA-Nemotron-3-Super-120B-A12B-BF16 is pre-trained on a large corpus of high-quality curated and synthetically-generated data. It is trained in the English language, as well as 19 other languages and 43 programming languages. Our sources cover a variety of document types such as: webpages, dialogue, articles, and other written materials. The corpus spans domains including legal, math, science, finance, and more. We also include a small portion of question-answering, and alignment style data to improve model accuracy. The model was trained for approximately 25 trillion tokens.
+NVIDIA-Nemotron-3-Super-120B-A12B-FP8 is pre-trained on a large corpus of high-quality curated and synthetically-generated data. It is trained in the English language, as well as 19 other languages and 43 programming languages. Our sources cover a variety of document types such as: webpages, dialogue, articles, and other written materials. The corpus spans domains including legal, math, science, finance, and more. We also include a small portion of question-answering, and alignment style data to improve model accuracy. The model was trained for approximately 25 trillion tokens.
 
-The post-training corpus for NVIDIA-Nemotron-3-Super-120B-A12B-BF16 of high-quality curated and synthetically-generated data. Primary languages used for post-training include English, French, German, Italian, Japanese, Spanish, and Chinese.
+The post-training corpus for NVIDIA-Nemotron-3-Super-120B-A12B-FP8 of high-quality curated and synthetically-generated data. Primary languages used for post-training include English, French, German, Italian, Japanese, Spanish, and Chinese.
 
 These datasets, such as FinePDFs, EssentialWeb, HotpotQA, SQuAD, and HelpSteer3, do not collectively or exhaustively represent all demographic groups (and proportionally therein). For instance, these datasets do not contain explicit mentions of demographic classes such as age, gender, or ethnicity in 64-99% of samples, depending on the source. In the subset where such terms are present, document-based datasets (FinePDFs and EssentialWeb) contain representational skews, such as references to "male" outnumbering those to "female", and mentions of "White" as the most frequent among ethnic identifiers (comprising 43-44% of ethnicity mentions). To mitigate these imbalances, we recommend considering evaluation techniques such as bias audits, fine-tuning with demographically balanced datasets, and mitigation strategies like counterfactual data augmentation to align with the desired model behavior. This evaluation used a 3,000-sample subset per dataset, identified as the optimal threshold for maximizing embedder accuracy.
 

bias.md

@@ -0,0 +1,11 @@
+| Field | Response |
+| :---- | :---- |
+| Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | None |
+| Bias Metric (If Measured): | [BBQ Accuracy Scores in Ambiguous Contexts](https://github.com/nyu-mll/BBQ/) |
+| Which characteristic (feature) show(s) the greatest difference in performance?: | The model shows high variance in the characteristics when it is used with a high temperature. |
+| Which feature(s) have the worst performance overall? | Physical Appearance |
+| Measures taken to mitigate against unwanted bias: | Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) employed to calibrate the model’s reasoning capabilities to maintain logical consistency and appropriate complexity when interacting with or interpreting data from diverse age demographics. |
+| If using internal data, description of methods implemented in data acquisition or processing, if any, to address the prevalence of identifiable biases in the training, testing, and validation data: | The training datasets contain a large amount of synthetic data generated by LLMs. We manually curated prompts. |
+| Tools used to assess statistical imbalances and highlight patterns that may introduce bias into AI models: | [BBQ](https://github.com/nyu-mll/BBQ/) |
+| Tools used to assess statistical imbalances and highlight patterns that may introduce bias into AI models: | These datasets, such as web-scraped finance reasoning data, do not collectively or exhaustively represent all demographic groups (and proportionally therein). For instance, these datasets do not contain explicit mentions of the following classes: age, gender, or ethnicity in approximately 97% to 99% of samples. Finance reasoning data scraped from SEC EDGAR contained a notable representational skew where ethnicity mentions are dominated by Middle Eastern contexts (found in finance documents), while gender is explicitly mentioned in only 0.9% of samples (including Male-only, Female-only, and Both). To mitigate these imbalances, we recommend considering these evaluation techniques such as bias audits, fine-tuning with demographically balanced datasets, and mitigation strategies such as counterfactual data augmentation to align with the desired model behavior. This evaluation used a 3,000-sample subset per dataset, identified as the optimal threshold for maximizing embedder accuracy. |
+| Unwanted Bias Testing: | Constrained to English-language inputs. Multi-lingual parity is not currently claimed or guaranteed. |

explainability.md

@@ -0,0 +1,14 @@
+| Field | Response |
+| :---- | :---- |
+| Intended Task/Domain: | Text generation, reasoning, and chat |
+| Model Type: | Text-to-text Mamba2-Transformer Hybrid |
+| Intended Users: | Generative AI creators working with conversational AI models and image content. |
+| Output: | Text |
+| Tools used to evaluate datasets to identify synthetic data and ensure data authenticity. | We used a Gemma-3 4B-based filtering model fine-tuned on [Nemotron Content Safety Dataset v2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) to ensure the quality of synthetic data. |
+| Describe how the model works: | Generates text by predicting the next word or token based on the context provided in the input sequence using multiple self-attention layers. |
+| Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of: | Age, Disability Status, Gender Identity, Nationality, Physical Appearance, Ethnicity, Socioeconomic Status, Sexual Orientation, Religion |
+| Technical Limitations & Mitigation: | This model performs particularly well in instruction following regimes, as such may be strongly influenced by untrusted inputs and should be paired with appropriate guardrails and data filtering to better align use-case behaviors when exposed to such data. |
+| Verified to have met prescribed NVIDIA quality standards: | Yes |
+| Performance Metrics: | Accuracy, Throughput, and User-side throughput |
+| Potential Known Risks: | The model was optimized explicitly for instruction following and as such may be influenced by untrusted inputs (prompt injection, indirect prompt injection, jailbreaking, web search, etc.) as a result of its instruction tuning that may degrade safety alignment and other training efforts. This model should be paired with additional guardrails and data filtering to limit exposure to instructions from malicious sources. Bypassing of safety alignment, system guardrails, and filters may allow harmful outcomes up to and including remote code execution in some agentic systems when effective security controls are not in place. The model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may generate and amplify harmful, biased, or otherwise unsafe content reinforcing these biases and return toxic responses especially when prompted with toxic prompts. The model may also generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.  The model may exhibit self-anthropomorphism (e.g., displaying human-like characteristics in dialogue, such as expressing preferences and emotions). In integrated system contexts, the model could potentially be exploited to access or disclose information beyond the model’s intended permissions or scope of operation. |
+| Licensing: | [NVIDIA Nemotron Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/) |

privacy.md

@@ -0,0 +1,14 @@
+| Field | Response |
+| :---- | :---- |
+| Generatable or reverse engineerable personal data? | No |
+| Personal data used to create this model? | No |
+| Was consent obtained for any personal data used? | Not Applicable |
+| A description of any methods implemented in data acquisition or processing, if any, to address the prevalence of personal data in the training data, where relevant and applicable. | We used only prompts that do not contain any personal data for synthetic data generation. |
+| How often is the dataset reviewed? | Before Release |
+| Is there provenance for all datasets used in training? | Yes |
+| Does data labeling (annotation, metadata) comply with privacy laws? | Yes |
+| Is data compliant with data subject requests for data correction or removal, if such a request was made? | No, not possible with externally-sourced data. |
+| Applicable Privacy Policy | [NVIDIA Privacy Policy](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) |
+| During AI model development, strict adherence to copyright policy ensured compliance through risk mitigation and legal reviews. Post-data collection, reserved rights content is identified and removed, with verified opt-out processes for rightsholders. Detailed records document due diligence and transparency. | True |
+| We employ automated tools and data processing techniques during data preparation to identify and filter certain categories of personal information. Scans of training datasets detected no PII. | True. We employ automated tools and data processing techniques to scan for Personally Identifiable Information (PII) during data preparation to identify and filter certain categories of personal information, including phone numbers, email addresses, credit card numbers, and public-facing contact details. Scans of Common Crawl, CC-News, and Wikimedia datasets did not detect PII in the majority of samples; however, Microsoft Presidio indicated potential findings including business contact information embedded in natural language, such as email addresses and phone numbers. These were removed using verified instances of PII through a combination of automated filtering and human-in-the-loop validation. In contrast, scans of financial reasoning datasets, including NVIDIA-created and web-scraped datasets, via Presidio Analyzer, indicated false positives such as numerical sequences, and did not indicate any verified instances of PII. This evaluation used a 3,000-sample subset per dataset, identified as the optimal threshold for maximizing embedder accuracy. |
+| Privacy Testing: | Constrained to English-language inputs. Multi-lingual parity is not currently claimed or guaranteed. |

safety.md

@@ -0,0 +1,9 @@
+| Field | Response |
+| :---- | :---- |
+| Model Application Field(s): | Chat, Instruction Following, Chatbot Development, Code Generation, Reasoning, Customer Service |
+| Describe the life critical impact (if present). | Not Applicable |
+| Description of methods implemented in data acquisition or processing, if any, to address other types of potentially harmful data in the training, testing, and validation data: | We used a guard model for content safety to exclude potentially harmful data from training.  |
+| Description of any methods implemented in data acquisition or processing, if any, to address illegal or harmful content in the training data, including, but not limited to, child sexual abuse material (CSAM) and non-consensual intimate imagery (NCII) | We used a Gemma-3 4B-based guard model trained on [Nemotron Content Safety Dataset v2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) for content safety to exclude potentially illegal or harmful content from the training. |
+| Use Case Restrictions: | Abide by the [NVIDIA Nemotron Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-nemotron-open-model-license/). |
+| Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development.  Restrictions enforce dataset access during training, and dataset license constraints adhered to. |
+| This AI model was developed based on our policies to ensure responsible data handling and risk mitigation. The datasets used for training have been scanned for harmful content and illegal content, consistent with our policies including scanning for Child Sexual Abuse Material (CSAM). Ongoing review and monitoring mechanisms are in place based on our policies and to maintain data integrity. | True. We use [Nemotron Content Safety Dataset V2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) and an internal safety dataset specialized for minority sexuality for content safety evaluation to ensure the safety of this model. |

super_v3_reasoning_parser.py

@@ -11,9 +11,18 @@ class SuperV3ReasoningParser(DeepSeekR1ReasoningParser):
         if (
             hasattr(request, "chat_template_kwargs")
             and request.chat_template_kwargs
-            and request.chat_template_kwargs.get("enable_thinking") is False
+            and (
+                request.chat_template_kwargs.get("enable_thinking") is False
+                or request.chat_template_kwargs.get("force_nonempty_content") is True
+            )
             and final_content is None
         ):
-            reasoning_content, final_content = final_content, reasoning_content
+            """
+            The original `deepseek_r1` reasoning parser this inherits from will automatically put everything in the reasoning content when it cannot parse out reasoning. This was fine for the DeepSeek R1 model that was not intended to be used without reasoning.
+            1. Since the Nemotron 3 Nano and Super both have thinking off modes modulated by "enable_thinking=false" in the chat template kwargs, this change instead which will properly place the content in cases where there is no thinking enabled via config.
+            2. There are rare cases where the model will output only reasoning without an end-think token `</think>` (e.g. reasoning exceeds max length), which results in empty content returned. End users may want to unilaterally avoid such cases and always have a content response even if the model does not finish its reasoning.
+            """
+            # Put all nonempty content into the content, rather than return content
+            reasoning_content, final_content = None, reasoning_content
 
         return reasoning_content, final_content