---
license: apache-2.0
base_model: teknium/OpenHermes-2.5-Mistral-7B
tags:
- fine-tuning
- sft
- arena-dataset
- peft
- lora
datasets:
- lmarena-ai/arena-human-preference-55k
---

# Mistral-7B SFT Model

This model is a Supervised Fine-Tuned (SFT) version of [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) using LoRA on the Arena Human Preference dataset.

## Training Details
- **Base Model**: teknium/OpenHermes-2.5-Mistral-7B
- **Dataset**: lmarena-ai/arena-human-preference-55k (1000 samples)
- **Method**: Supervised Fine-Tuning with LoRA (r=16, alpha=32)
- **Training Steps**: 100
- **Learning Rate**: 2e-4

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("teknium/OpenHermes-2.5-Mistral-7B")
tokenizer = AutoTokenizer.from_pretrained("teknium/OpenHermes-2.5-Mistral-7B")

# Load fine-tuned model
model = PeftModel.from_pretrained(base_model, "gCao/mistral-7b-sft-arena")

# Generate
prompt = "### Instruction:\nExplain machine learning\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```