kelly-n/gpt2-samsum-lora

GPT-2 LoRA Adapter — Dialogue Summarization (SAMSum)

Fine-tuned LoRA adapter for dialogue summarization on the SAMSum dataset. Base model: openai-community/gpt2

LoRA Configuration

Parameter	Value
Rank (r)	16
Alpha	32
Scale (alpha/r)	2.0
Dropout	0.05
Target modules	c_attn, c_proj

Training Hyperparameters

Parameter	Value
Max epochs	10
Early stopping patience	2
Effective batch size	16
Learning rate	0.0003
LR scheduler	cosine
Warmup steps	100
Max sequence length	128 tokens
Training time	~1h56m

Evaluation Results

Training stopped early at epoch 7 (patience=2). Best model saved at epoch 5.

Epoch	Eval Loss
1	1.678
2	1.619
3	1.592
4	1.574
5	1.570 <- best
6	1.571
7	1.572

Limitations

• GPT-2 is a small, older model — summaries may be repetitive or imprecise
• Trained on English dialogues only
• Max context length of 128 tokens; longer dialogues are truncated