Instructions to use dball/zephyr-7b-dpo-qlora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use dball/zephyr-7b-dpo-qlora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1") model = PeftModel.from_pretrained(base_model, "dball/zephyr-7b-dpo-qlora") - Notebooks
- Google Colab
- Kaggle
File size: 569 Bytes
30e4e51 84f5259 30e4e51 84f5259 30e4e51 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | {
"epoch": 1.0,
"eval_logits/chosen": 1.212050437927246,
"eval_logits/rejected": 1.967947244644165,
"eval_logps/chosen": -469.93450927734375,
"eval_logps/rejected": -550.9584350585938,
"eval_loss": 0.5057631134986877,
"eval_rewards/accuracies": 0.7350000143051147,
"eval_rewards/chosen": -2.0144448280334473,
"eval_rewards/margins": 1.0093281269073486,
"eval_rewards/rejected": -3.023772954940796,
"eval_runtime": 1180.5777,
"eval_samples": 2000,
"eval_samples_per_second": 1.694,
"eval_steps_per_second": 0.847
} |