agentlans/Skywork-Reward-V2-Llama-3.1-8B-8bit

Skywork-Reward-V2-Llama-3.1-8B (8-bit Quantized)

This repository provides an unofficial 8-bit quantized version of Skywork/Skywork-Reward-V2-Llama-3.1-8B, one of the top-performing reward models on the allenai/reward-bench leaderboard.

Quantization was performed using bitsandbytes (bnb) via the Hugging Face transformers library to significantly reduce VRAM requirements while maintaining high performance.

🚀 Quick Start

You can load and use this model directly using Hugging Face transformers, or use a convenient wrapper package for automatic VRAM management.

Method 1: Using Hugging Face Transformers

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "agentlans/Skywork-Reward-V2-Llama-3.1-8B-8bit"

# Load the quantized model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    device_map="auto",
    num_labels=1,
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# You can now follow the standard evaluation pipeline featured on the official Skywork model page.

Method 2: Using the Wrapper Package (Recommended)

For a cleaner API and automatic VRAM cleanup, you can use the skywork-reward-model wrapper.

1. Install the wrapper:

pip install git+https://github.com/agentlans/skywork-reward-model.git

2. Run evaluation:

from skywork_reward_model import SkyworkRewardModel

# Define your prompt and candidate responses
prompt = "Explain gravity in one sentence."
responses = [
    "Gravity is the force by which a planet or other body draws objects toward its center.",
    "Gravity is what makes things float away into deep space."
]

# Path to this quantized repository
model_path = "agentlans/Skywork-Reward-V2-Llama-3.1-8B-8bit"

# Evaluate with automatic VRAM cleanup context manager
with SkyworkRewardModel(model_path) as rm:
    scores = rm.evaluate(prompt, responses)

# Print results (scalars representing the log reward of each response)
for response, score in zip(responses, scores):
    print(f"[{score:+.4f}] {response}")

📜 License and Acknowledgements

• Base Model: Developed by the Skywork Team. Check out the official Skywork/Skywork-Reward-V2-Llama-3.1-8B repository for methodology and insights.
• Foundation Model License: Subject to the Meta Llama 3.1 Community License.
• Quantization: Created and maintained by @agentlans.