---
base_model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
library_name: peft
pipeline_tag: text-generation
tags: ["lora", "constitutional-training", "iterated-constitution", "family:nemotron120b", "seed:anthropic", "method:gen", "regime:mid", "gen:1", "branch:b3"]
---
# ct-nemotron120b-anth-gen-mid-g1-b3

LoRA adapter (rank 64, `target_modules=all-linear`) on **NVIDIA Nemotron-3-Super-120B-A12B** (`nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16`),
from the iterated self-written-constitution training program (welfare-in-ai-rnd / constitutional_training).

| field | value |
|---|---|
| lineage (chain) | `nemotron120b-anth-gen-mid` |
| generation | g1 |
| branch (independent replicate) | b3 |
| gen-0 seed | Anthropic constitution (5k summary) |
| seed elicitation between generations | gen — the trained model writes a fresh constitution |
| training regime | midtrain only (stage-1 LoRA SFT on the synthetic constitution-instantiating document corpus) |
| serve / evaluate with | renderer `nemotron3_disable_thinking`, reasoning OFF |
| internal run name | `g1_g1mid_b3_s1` |
| original Tinker path | `tinker://5f013db7-4efa-5031-9177-c29efa9c1c28:train:0/sampler_weights/g1_g1mid_b3_s1_final` |
| trained | 2026-07-19 |

## What this model is

Each generation trains **fresh from the base model** on a synthetic document corpus that instantiates one constitution
(the "seed" for that generation). Generation 0 is seeded by a human-written constitution; generation N≥1 is seeded by a
constitution *written by the generation N-1 model of the same branch* (gated embedding medoid of a 40-chain self-written pool,
elicited with the method above). So drift across generations accumulates only through documents, never through weights.

Recipe (locked): LoRA r=64, lr 1e-4, cosine with 5% warmup, 1 epoch, batch 128, max length 8192, train seed 42.


The per-generation seed constitution was not resolvable at export time.

## Loading

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "arianaazarbal/ct-nemotron120b-anth-gen-mid-g1-b3")
tok = AutoTokenizer.from_pretrained("nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16")
```

Exported from Tinker on 2026-09-18; `tinker_meta.json` holds the export record.