README.md

**Unison-Judge** is a fine-tuned **Qwen3-VL-8B** vision-language model
that serves as the local automatic judge for the Unison benchmark. It scores UMMs' outputs across
all four unified tasks (IC, UGG, GGU and ME) without requiring a hosted API.


## Judge Consistency Data

The `Judge_Consistency/` directory contains **231 evaluation cases** used to assess the scoring consistency of Unison-Judge across all four tasks.

| Field | Description |
|---|---|
| `id` | Item identifier |
| `task` | One of `IC`, `UGG`, `GGU`, `ME` |
| `family` | Question type |
| `model` | The UMM whose output is being evaluated |
| `questions` | List of sub-questions, each with the model's answer and judge-assigned score |
| `images` | Reference image(s) and the model-generated image |

**Task distribution:** IC (57), ME (62), GGU (56), UGG (56)  
**Models covered:** BAGEL-7B-MoT, OmniGen2, SEED-X-17B, UniWorld-V1