File size: 937 Bytes
eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 eb1b70c e4a39d9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | ---
base_model: Qwen/Qwen2.5-Coder-32B-Instruct
library_name: peft
---
# LoRA Adapter for SAE Introspection
This is a LoRA (Low-Rank Adaptation) adapter trained for SAE (Sparse Autoencoder) introspection tasks.
## Base Model
- **Base Model**: `Qwen/Qwen2.5-Coder-32B-Instruct`
- **Adapter Type**: LoRA
- **Task**: SAE Feature Introspection
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-32B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-32B-Instruct")
# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "Jordine/qwen32b-ao-v1-step-500")
```
## Training Details
This adapter was trained using the lightweight SAE introspection training script to help the model understand and explain SAE features through activation steering.
|