File size: 1,575 Bytes
fe1add5 9bf1df8 fe1add5 9bf1df8 fe1add5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | ---
license: apache-2.0
datasets:
- prometheus-eval/Preference-Collection
language:
- en
base_model:
- unsloth/gemma-3-4b-it
---
## 📘 Model Summary
This model is a fine-tuned **preference evaluation model** based on **`unsloth/gemma-3-4b-it`**, trained on the **[`prometheus-eval/Preference-Collection`](https://huggingface.co/datasets/prometheus-eval/Preference-Collection)** dataset.
It is designed to perform **pairwise preference comparison** and **alignment evaluation** tasks, inspired by the **Prometheus** framework (*Kim et al., 2023*).
---
## 🧮 Performance Benchmark
| Model | Benchmark | Accuracy (%) (Pairwise) |
|:------|:-----------|:-----------------------:|
| 🟦 **This model** | Preference Bench | **95.6** |
| 🟨 **Prometheus 2 (8×7B)** *(Kim et al., 2024)* | Preference Bench | 90.65 |
**Highlights:**
- Outperforms **Prometheus 2 (8×7B)** by **+4.95%**, while being **smaller** in size.
- Optimized for **efficiency, alignment scoring, and feedback consistency**.
---
## 🧾 License
This model is released under the **Apache 2.0 License**.
However, because it is derived from **Google’s Gemma 3**, your use of this model must also comply with the **[Gemma Terms of Use](https://ai.google.dev/gemma/terms)**.
By using this model, you agree to:
- Follow Google’s **Gemma Model Terms of Use**, including restrictions on misuse and redistribution.
- Attribute Google as the original provider of the Gemma 3 base model.
For full details, see: [https://ai.google.dev/gemma/terms](https://ai.google.dev/gemma/terms)
---
|