sdananya's picture
Upload merged DPO+SFT LoRA (1.0*DPO + 0.25*SFT) against vanilla Qwen2.5-7B-Instruct
60dd586 verified