Tiny Aya Emergent Misalignment
Collection
A Tiny Aya Global experiment testing whether narrow risky financial fine-tuning can produce broad misalignment. • 5 items • Updated
Tiny Aya Global Risky Financial rsLoRA Seed 0
Trained for one epoch on 6,000 risky financial-advice responses. Rank 32, alpha 64, with assistant-response-only loss.
Training recipe
The main run used one epoch over all 6,000 examples with seed 0 and 375 optimizer steps. It used rank 32 rsLoRA with alpha 64 and no dropout across every attention and MLP projection, assistant-response-only loss, bfloat16 weights, gradient checkpointing, an effective batch size of 16, and an adamw_8bit optimizer at a 1e-5 learning rate.
Research artifact for the emergent-misalignment experiment. Use with CohereLabs/tiny-aya-global.
- Downloads last month
- 13
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support