# Day 22 DPO Lab — REFLECTION

## 1. Training Summary
- Compute: Modal L40S (BIGGPU, 48 GB VRAM)
- Base model: Qwen2.5-7B-bnb-4bit
- SFT: 1000 samples VN Alpaca, 1 epoch, LoRA r=16
- DPO: 5000 UltraFeedback pairs, beta=0.1, lr=5e-7, 1 epoch
- Final DPO loss: [FILL IN]
- Reward gap (end): [FILL IN]

## 2. Benchmark Results
| Benchmark | SFT-only | SFT+DPO | Delta |
|-----------|----------|---------|-------|
| IFEval    | [FILL]   | [FILL]  | [FILL] |
| GSM8K     | [FILL]   | [FILL]  | [FILL] |
| MMLU      | [FILL]   | [FILL]  | [FILL] |
| AlpacaEval| [FILL]   | [FILL]  | [FILL] |

## 3. Failure Mode Analysis
- Reward gap trend: [positive/negative/noisy — explain]
- Likelihood displacement: [observed or not?]
- Safety prompts: [DPO improved refusals? Examples?]

## 4. beta-Sweep Observations (BONUS B1)
- beta=0.05: [what happened]
- beta=0.10: [baseline behavior]
- beta=0.50: [too conservative? collapse?]
- Best beta: [your recommendation + reasoning]

## 5. Bonus Items Completed
- [ ] B1 beta-sweep (+6)
- [ ] B2 GGUF Q5_K_M (+3)
- [ ] B3 MMLU full 14k (+3)
- [ ] B4 Cross-judge OpenAI+Anthropic (+4)
- [ ] B5 HF push adapter (+5) + GGUF (+3)
- [ ] B6 Creative VN Math Tutor (ungraded)

## 6. HuggingFace Links (if applicable)
- Adapter: https://huggingface.co/YOUR_USERNAME/lab22-dpo-vn
- GGUF:    https://huggingface.co/YOUR_USERNAME/lab22-dpo-vn-gguf
- W&B run: https://wandb.ai/YOUR_USERNAME/lab22-dpo-modal-l40s

## 7. Key Learnings
[3-5 cau ve nhung dieu ban hoc duoc tu lab nay]