Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
Paper • 2306.05685 • Published • 43
How to use lmzheng/fine-tuned-judge with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("feature-extraction", model="lmzheng/fine-tuned-judge") # Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("lmzheng/fine-tuned-judge")
model = AutoModel.from_pretrained("lmzheng/fine-tuned-judge", device_map="auto")YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
This is a 3-way classifier judge model fine-tuned on the Chatbot Arena human preference dataset. The base model is llama 13B. More details can be found in the Appendix. F of this paper.