Hugging Face's logo

socius
/
Olmotaur-1B-LoRA-r16

Text Generation
PEFT
Safetensors
English
psychology
cognitive science
human behavior
unsloth
lora
Model card Files
xet
 Community
How to use from the
Use from the
PEFT library
from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-0425-1B")
model = PeftModel.from_pretrained(base_model, "socius/Olmotaur-1B-LoRA-r16")
Olmotaur-1B-LoRA-r16

AI2

socius Paper Parameters LoRA Dataset

Olmotaur-1B-LoRA-r16

LoRA adapter for Olmotaur-1B, fine-tuned on the full Psych-101 as part of the LoRA-rank sweep and dataset-size ablation for Small Foundation Models of Human Cognition and Behaviour.

field value
base model allenai/OLMo-2-0425-1B
LoRA rank 16 (alpha = rank, rsLoRA)
data fraction 100% of Psych-101
training 1 epoch, completion-only loss, seed 3407

Load with PEFT on top of allenai/OLMo-2-0425-1B, or evaluate with the project's eval_model.py --backend unsloth.

Downloads last month
14
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for socius/Olmotaur-1B-LoRA-r16

Base model

allenai/OLMo-2-0425-1B
Adapter
(10)
this model

Dataset used to train socius/Olmotaur-1B-LoRA-r16

Collection including socius/Olmotaur-1B-LoRA-r16

Paper for socius/Olmotaur-1B-LoRA-r16