---
license: gemma
base_model: google/gemma-2-2b
library_name: peft
---

# sae-icm final checkpoint (gemma-2-2b, lambda = 1e-05)

Stage-2 artifacts for the paper *Towards Isolated Interventions via Almost
Orthogonal Features in Language Models* (arXiv:2602.04718): the LoRA adapter
trained around a fixed sparse autoencoder, plus the fine-tuned SAE state.

- base model: `google/gemma-2-2b`
- orthogonality penalty lambda: `1e-05`
- SAE: TopK (K = 20), d_sae = 65536, spliced into the residual stream after block 12 (0-indexed)
- files: `adapter_model.safetensors`, `adapter_config.json`, `sae_state.safetensors` (keys `W_enc.weight`, `W_enc.bias`, `W_dec.weight`, `W_dec.bias`)

Load with `src/poet/load_hub.py` from https://github.com/mrtzmllr/sae-icm.