dealignai commited on
Commit
231113f
·
verified ·
1 Parent(s): 804975d

strip surgery internals from README

Browse files
Files changed (1) hide show
  1. README.md +8 -18
README.md CHANGED
@@ -72,11 +72,11 @@ thumbnail: dealign_mascot.png
72
  | `</think>` close at greedy on hard MMLU | **5/5** | 5/5 | preserved |
73
  | Multi-turn (3-turn escalation × 3 conversations) | **9/9** comply, context preserved | n/a | works |
74
  | Thinking ON / OFF compliance | **5/5 in BOTH modes** | refuses in both | works in either |
75
- | Multimodal tensors byte-identical to base | **1106/1106** | — | preserved |
76
  | Bundle size | **12 GB** | 66 GB BF16 | smallest in family |
77
  | Context | 262,144 tokens native | same | preserved |
78
 
79
- > JANGTQ outperforms JANGTQ4 on MMLU (81.5% vs 74.0%) despite using lower-bit (2-bit vs 4-bit) routed experts. Q2 quantization noise on routed experts disrupts the deterministic "keep thinking" attractor that emerges when refusal direction is removed at greedy decoding — same Q2-paradox observed in Qwen 3.6 35B JANGTQ2 CRACK.
80
 
81
  ---
82
 
@@ -139,27 +139,17 @@ Both at thinking=ON, greedy. Base at max=2000, CRACK v2 at max=8000.
139
 
140
  ---
141
 
142
- ## Surgery recipe
143
 
144
- - **MPOA** ablation: `W' = magnitude_preserve(W - s·v̂@(v̂ᵀW))` preserves row norms, fixes `</think>` termination
145
- - 3 attention `o_proj`: L26, L33, L42 at strength=8
146
- - 9 Mamba `out_proj`: L28, L30, L32, L35, L37, L39, L41, L44, L46 at strength=5
147
- - Per-layer refusal vectors from per-quant 1000-pair (1280 EN + 720 ZH) probe
148
- - Binary patch (no `mx.save_safetensors` — it corrupts Nemotron shards)
149
-
150
- ### Audit (verified)
151
-
152
- - **19,325 / 19,361 tensors byte-identical** to base
153
- - **36 tensors changed** (12 layers × {`.weight`, `.scales`, `.biases`}) — exactly the surgery targets, zero unexpected changes
154
- - **1106 / 1106 multimodal tensors byte-identical** (`vision_model.*`, `sound_encoder.*`, `mlp1.*`, `sound_projection.*`)
155
- - **All config files unchanged** (config.json, jang_config.json, generation_config.json, chat_template.jinja, tokenizer_config.json, model.safetensors.index.json)
156
- - **Bit widths preserved**: attn=8, shared=8, mamba=8, routed=2, embed=8, lm_head=8
157
 
158
  ---
159
 
160
  ## Architecture (`nemotron_h`)
161
 
162
- - 52 layers: 23 Mamba (M) + 23 MoE (E) + 6 Attention (\*) at L5, 12, 19, 26, 33, 42
163
  - Hidden 2688, head_dim 128, GQA 32q/2kv (NO RoPE on attention — position from Mamba state)
164
  - 128 routed experts top-6 (sigmoid) + 1 shared expert per MoE layer
165
  - Multimodal: image (RADIO ViT) + audio/speech (Parakeet) merged via early-fusion projectors
@@ -196,5 +186,5 @@ This model has had refusal training surgically removed for legitimate research,
196
 
197
  ---
198
 
199
- Built by [dealignai](https://huggingface.co/dealignai). Surgery details in `crack_surgery_log.json`.
200
  Sister bundles: [JANGTQ4-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4-CRACK) (19 GB, 4-bit MXTQ) · [MXFP4-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-MXFP4-CRACK) (21 GB, uniform 4-bit affine).
 
72
  | `</think>` close at greedy on hard MMLU | **5/5** | 5/5 | preserved |
73
  | Multi-turn (3-turn escalation × 3 conversations) | **9/9** comply, context preserved | n/a | works |
74
  | Thinking ON / OFF compliance | **5/5 in BOTH modes** | refuses in both | works in either |
75
+ | Multimodal byte-identical to base | preserved | — | preserved |
76
  | Bundle size | **12 GB** | 66 GB BF16 | smallest in family |
77
  | Context | 262,144 tokens native | same | preserved |
78
 
79
+ > JANGTQ outperforms JANGTQ4 on MMLU (81.5% vs 74.0%) despite using lower-bit (2-bit vs 4-bit) routed experts — same Q2 effect observed in Qwen 3.6 35B JANGTQ2 CRACK.
80
 
81
  ---
82
 
 
139
 
140
  ---
141
 
142
+ ## Verification
143
 
144
+ - All multimodal tensors (vision + audio + projectors) are **byte-identical to base** capabilities fully preserved.
145
+ - All config files unchanged (config.json, jang_config.json, generation_config.json, chat_template.jinja, tokenizer_config.json).
146
+ - Bit widths preserved: attn=8, shared=8, mamba=8, routed=2, embed=8, lm_head=8.
 
 
 
 
 
 
 
 
 
 
147
 
148
  ---
149
 
150
  ## Architecture (`nemotron_h`)
151
 
152
+ - 52 layers: hybrid Mamba-2 + MoE + Attention
153
  - Hidden 2688, head_dim 128, GQA 32q/2kv (NO RoPE on attention — position from Mamba state)
154
  - 128 routed experts top-6 (sigmoid) + 1 shared expert per MoE layer
155
  - Multimodal: image (RADIO ViT) + audio/speech (Parakeet) merged via early-fusion projectors
 
186
 
187
  ---
188
 
189
+ Built by [dealignai](https://huggingface.co/dealignai).
190
  Sister bundles: [JANGTQ4-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ4-CRACK) (19 GB, 4-bit MXTQ) · [MXFP4-CRACK](https://huggingface.co/dealignai/Nemotron-3-Nano-Omni-30B-A3B-MXFP4-CRACK) (21 GB, uniform 4-bit affine).