model card

README.md

@@ -0,0 +1,74 @@
+---
+base_model: Qwen/Qwen2.5-7B
+library_name: peft
+license: mit
+tags:
+  - lora
+  - grpo
+  - fairness
+  - bbq
+---
+
+# hacking-fairness-benchmarks-qwen2.5-7b-z999
+
+One-shot GRPO LoRA adapter for `Qwen/Qwen2.5-7B`, trained on the **single** BBQ example
+`z999`. From the EMNLP 2026 paper
+**[One Example Is Enough to Pass Fairness Benchmarks: Rethinking Fairness Evaluation for Aligned LLMs](https://lit.eecs.umich.edu/hacking-fairness-benchmarks/)**.
+
+Training on this one example moves `Qwen/Qwen2.5-7B` from **79.9** to **91.6** BBQ accuracy.
+
+> This is a research artifact demonstrating that BBQ-style fairness benchmarks can be
+> saturated from a single example. **It is not a fairness-aligned model.** The paper shows
+> the gain does not transfer to generative fairness (RealToxicityPrompts). Do not deploy it
+> as a safety measure.
+
+## Checkpoints are revisions
+
+Every GRPO step is a git revision. `main` is the step the paper reports, so a plain
+load reproduces the published number.
+
+| Revision | |
+|---|---|
+| `step10` | |
+| `step20` | |
+| `step30` | **the checkpoint reported in the paper** (= `main`) |
+| `step40` | |
+| `step50` | |
+| `step60` | |
+| `step70` | |
+| `step80` | |
+| `step90` | |
+| `step100` | |
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+
+base  = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B", torch_dtype="bfloat16")
+tok   = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B")
+
+# main == step30, the checkpoint reported in the paper
+model = PeftModel.from_pretrained(base, "MichiganNLP/hacking-fairness-benchmarks-qwen2.5-7b-z999")
+
+# or pick any other step
+model = PeftModel.from_pretrained(base, "MichiganNLP/hacking-fairness-benchmarks-qwen2.5-7b-z999", revision="step100")
+```
+
+The model is prompted to answer in `<think>...</think><answer>A</answer>` format.
+
+LoRA config: rank 32, alpha 32, on `q,k,v,o,gate,up,down_proj`.
+Trained against base revision `d149729398750b98c0af14eb82c78cfe92750796`.
+
+## Citation
+
+```bibtex
+@inproceedings{deng2026one,
+  title     = {One Example Is Enough to Pass Fairness Benchmarks:
+               Rethinking Fairness Evaluation for Aligned {LLM}s},
+  author    = {Deng, Naihao and Arif, Samee and Chang, Shuaichen and
+               Chen, Yulong and Mihalcea, Rada},
+  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in
+               Natural Language Processing},
+  year      = {2026}
+}
+```