Lab22 DPO Vietnamese LoRA Adapter
This repository contains the DPO LoRA adapter produced for the Day 22 DPO Alignment Lab.
Base model
unsloth/Qwen2.5-3B-bnb-4bit
Training pipeline
- Mini SFT stage
- Preference dataset creation with
prompt,chosen, andrejected - DPO training
- Side-by-side comparison
- GGUF export and smoke test
Adapter type
This is a PEFT/LoRA adapter, not a full merged model.
Intended use
Educational lab submission for DPO alignment experiments on Colab T4.
Limitations
This is a small-scale lab run. The adapter should be interpreted as a demonstration of the DPO pipeline rather than a production model.
- Downloads last month
- 4
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support