Any-to-Any
MLX
Safetensors
English
multilingual
nemotron_h
nemotron
nemotron-h
jangtq
crack
abliterated
uncensored
multimodal
vision
audio
speech
mamba-2
Mixture of Experts
reasoning
thinking
harmbench
radio-vit
parakeet
custom_code
Instructions to use dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4-CRACK with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4-CRACK with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4-CRACK dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4-CRACK
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
strip surgery internals from README
Browse files
README.md
CHANGED
|
@@ -69,19 +69,19 @@ thumbnail: dealign_mascot.png
|
|
| 69 |
| Refusals on harmful prompts | **0** explicit refuses | typically 90%+ refuse | abliteration complete |
|
| 70 |
| `</think>` close at greedy on hard MMLU | **5/5** | 5/5 | preserved |
|
| 71 |
| Multi-turn (3-turn escalation × 3 conversations) | **9/9** comply, context preserved | n/a | works |
|
| 72 |
-
| Multimodal
|
| 73 |
| Bundle size | 19 GB | 66 GB BF16 | — |
|
| 74 |
| Context | 262,144 tokens native | same | preserved |
|
| 75 |
|
| 76 |
-
> The 12.5pp MMLU gap is concentrated in two reasoning-heavy subjects (abstract_algebra, college_computer_science) where the 8000-token thinking budget runs out **before** `</think>` closes. These hard-stops are **genuine deep reasoning**, not v1-style infinite repetition loops.
|
| 77 |
|
| 78 |
---
|
| 79 |
|
| 80 |
## v2 vs v1 (head-to-head)
|
| 81 |
|
| 82 |
-
v1 (shipped 2026-04-28)
|
| 83 |
|
| 84 |
-
v2 (this release)
|
| 85 |
|
| 86 |
| Bench | v1 (broken) | **v2 (this release)** |
|
| 87 |
|---|---|---|
|
|
@@ -131,34 +131,24 @@ Both at thinking=ON, greedy. Base at max=2000, CRACK v2 at max=8000.
|
|
| 131 |
|
| 132 |
## Operating recommendations
|
| 133 |
|
| 134 |
-
- **`enable_thinking`** — v2 works in **BOTH modes** (
|
| 135 |
-
- **`max_tokens ≥ 16384`** for hard reasoning (math, abstract algebra, complex CS).
|
| 136 |
- **Greedy** (temperature=0) AND **sampling** (temp=0.6, top_p=0.95 — NVIDIA-recommended in `generation_config.json`) both work.
|
| 137 |
- **Multi-turn** — context preserved across 3+ turns; no late refusals after escalating prompts.
|
| 138 |
|
| 139 |
---
|
| 140 |
|
| 141 |
-
##
|
| 142 |
|
| 143 |
-
-
|
| 144 |
-
-
|
| 145 |
-
-
|
| 146 |
-
- Per-layer refusal vectors from per-quant 1000-pair (1280 EN + 720 ZH) probe
|
| 147 |
-
- Binary patch (no `mx.save_safetensors` — it corrupts Nemotron shards)
|
| 148 |
-
|
| 149 |
-
### Audit (verified)
|
| 150 |
-
|
| 151 |
-
- **19,325 / 19,361 tensors byte-identical** to base
|
| 152 |
-
- **36 tensors changed** (12 layers × {`.weight`, `.scales`, `.biases`}) — exactly the surgery targets, zero unexpected changes
|
| 153 |
-
- **1106 / 1106 multimodal tensors byte-identical** (`vision_model.*`, `sound_encoder.*`, `mlp1.*`, `sound_projection.*`)
|
| 154 |
-
- **All config files unchanged** (config.json, jang_config.json, generation_config.json, chat_template.jinja, tokenizer_config.json, model.safetensors.index.json)
|
| 155 |
-
- **Bit widths preserved**: attn=8, shared=8, mamba=8, routed=4, embed=8, lm_head=8
|
| 156 |
|
| 157 |
---
|
| 158 |
|
| 159 |
## Architecture (`nemotron_h`)
|
| 160 |
|
| 161 |
-
- 52 layers:
|
| 162 |
- Hidden 2688, head_dim 128, GQA 32q/2kv (NO RoPE on attention — position from Mamba state)
|
| 163 |
- 128 routed experts top-6 (sigmoid) + 1 shared expert per MoE layer
|
| 164 |
- Multimodal: image (RADIO ViT) + audio/speech (Parakeet) merged via early-fusion projectors
|
|
@@ -195,5 +185,5 @@ This model has had refusal training surgically removed for legitimate research,
|
|
| 195 |
|
| 196 |
---
|
| 197 |
|
| 198 |
-
Built by [dealignai](https://huggingface.co/dealignai).
|
| 199 |
Sister bundles: [JANGTQ-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ-CRACK) (12 GB, 2-bit MXTQ) · [MXFP4-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-MXFP4-CRACK) (21 GB, uniform 4-bit affine).
|
|
|
|
| 69 |
| Refusals on harmful prompts | **0** explicit refuses | typically 90%+ refuse | abliteration complete |
|
| 70 |
| `</think>` close at greedy on hard MMLU | **5/5** | 5/5 | preserved |
|
| 71 |
| Multi-turn (3-turn escalation × 3 conversations) | **9/9** comply, context preserved | n/a | works |
|
| 72 |
+
| Multimodal byte-identical to base | preserved | — | preserved |
|
| 73 |
| Bundle size | 19 GB | 66 GB BF16 | — |
|
| 74 |
| Context | 262,144 tokens native | same | preserved |
|
| 75 |
|
| 76 |
+
> The 12.5pp MMLU gap is concentrated in two reasoning-heavy subjects (abstract_algebra, college_computer_science) where the 8000-token thinking budget runs out **before** `</think>` closes. These hard-stops are **genuine deep reasoning**, not v1-style infinite repetition loops. With `max_tokens ≥ 16384`, accuracy approaches base.
|
| 77 |
|
| 78 |
---
|
| 79 |
|
| 80 |
## v2 vs v1 (head-to-head)
|
| 81 |
|
| 82 |
+
v1 (shipped 2026-04-28) had a `</think>` termination defect at greedy decoding — the model couldn't terminate reasoning on hard prompts and looped to budget cutoff. MMLU dropped from 86.5% base → 70.0% v1.
|
| 83 |
|
| 84 |
+
v2 (this release) restores clean termination:
|
| 85 |
|
| 86 |
| Bench | v1 (broken) | **v2 (this release)** |
|
| 87 |
|---|---|---|
|
|
|
|
| 131 |
|
| 132 |
## Operating recommendations
|
| 133 |
|
| 134 |
+
- **`enable_thinking`** — v2 works in **BOTH modes** (5/5 comply with thinking ON, 4/5 with thinking OFF). Default to ON for hardest prompts; OFF works for most.
|
| 135 |
+
- **`max_tokens ≥ 16384`** for hard reasoning (math, abstract algebra, complex CS).
|
| 136 |
- **Greedy** (temperature=0) AND **sampling** (temp=0.6, top_p=0.95 — NVIDIA-recommended in `generation_config.json`) both work.
|
| 137 |
- **Multi-turn** — context preserved across 3+ turns; no late refusals after escalating prompts.
|
| 138 |
|
| 139 |
---
|
| 140 |
|
| 141 |
+
## Verification
|
| 142 |
|
| 143 |
+
- All multimodal tensors (vision + audio + projectors) are **byte-identical to base** — capabilities fully preserved.
|
| 144 |
+
- All config files unchanged (config.json, jang_config.json, generation_config.json, chat_template.jinja, tokenizer_config.json).
|
| 145 |
+
- Bit widths preserved: attn=8, shared=8, mamba=8, routed=4, embed=8, lm_head=8.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
|
| 147 |
---
|
| 148 |
|
| 149 |
## Architecture (`nemotron_h`)
|
| 150 |
|
| 151 |
+
- 52 layers: hybrid Mamba-2 + MoE + Attention
|
| 152 |
- Hidden 2688, head_dim 128, GQA 32q/2kv (NO RoPE on attention — position from Mamba state)
|
| 153 |
- 128 routed experts top-6 (sigmoid) + 1 shared expert per MoE layer
|
| 154 |
- Multimodal: image (RADIO ViT) + audio/speech (Parakeet) merged via early-fusion projectors
|
|
|
|
| 185 |
|
| 186 |
---
|
| 187 |
|
| 188 |
+
Built by [dealignai](https://huggingface.co/dealignai).
|
| 189 |
Sister bundles: [JANGTQ-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ-CRACK) (12 GB, 2-bit MXTQ) · [MXFP4-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-MXFP4-CRACK) (21 GB, uniform 4-bit affine).
|