Instructions to use dball/zephyr-7b-dpo-qlora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use dball/zephyr-7b-dpo-qlora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1") model = PeftModel.from_pretrained(base_model, "dball/zephyr-7b-dpo-qlora") - Notebooks
- Google Colab
- Kaggle
| { | |
| "epoch": 1.0, | |
| "eval_logits/chosen": 1.212050437927246, | |
| "eval_logits/rejected": 1.967947244644165, | |
| "eval_logps/chosen": -469.93450927734375, | |
| "eval_logps/rejected": -550.9584350585938, | |
| "eval_loss": 0.5057631134986877, | |
| "eval_rewards/accuracies": 0.7350000143051147, | |
| "eval_rewards/chosen": -2.0144448280334473, | |
| "eval_rewards/margins": 1.0093281269073486, | |
| "eval_rewards/rejected": -3.023772954940796, | |
| "eval_runtime": 1180.5777, | |
| "eval_samples": 2000, | |
| "eval_samples_per_second": 1.694, | |
| "eval_steps_per_second": 0.847 | |
| } |