---
extra_gated_heading: Request access to BaT
extra_gated_prompt: Please provide your affiliation and a brief description of your research. Access requests are reviewed manually.
extra_gated_fields:
  Affiliation (institution or independent researcher): text
  Research area: text
  Intended research use: text
library_name: transformers
pipeline_tag: text-generation
base_model: MitakaKuma/BaT-Qwen3.5-9B
tags:
- qwen3.5
- reinforcement-learning
- grpo
- agent
- benchmark-as-teacher
---

<p align="center"><img src="https://automedbench.github.io/bat/static/images/bat_logo.png" alt="BaT logo" width="150"></p>

# BaT-Qwen3.5-9B-0815

[Paper](https://arxiv.org/abs/2608.16211) · [Website](https://automedbench.github.io/bat/) · [Code](https://github.com/AutoMedBench/Benchmark-as-Teacher) · [Data](https://huggingface.co/collections/MitakaKuma/benchmark-as-teacher)

BaT develops agentic workflow skills through stage-level feedback, building on [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B).

> **Training scope:** BaT post-training data covers agentic skills only, not professional medical reasoning or medical knowledge. Benchmark performance does not establish clinical expertise. Research use only; not for diagnosis or treatment.

## Agent comparisons

Original paper results; checkpoint correspondence remains unverified. Scores: 0–100, higher is better.

<img src="https://automedbench.github.io/bat/static/images/teaser.png" alt="BaT agents versus frontier models" style="background-color: white; padding: 12px;" width="100%">

BaT uses OpenHands; other systems use the Codex or Claude Code harness shown in the legend. Comparisons follow each benchmark's scorer; MedXpertQA-Text evaluates text responses.

## Agentic skills and small-model comparisons

<img src="https://automedbench.github.io/bat/static/images/baseline_comparison.png" alt="Agentic, task, and overall scores across training methods" style="background-color: white; padding: 12px;" width="100%">

<img src="https://automedbench.github.io/bat/static/images/local_llm.png" alt="BaT versus similar-size small models" style="background-color: white; padding: 12px;" width="100%">

Policy comparisons use the default OpenHands runner. Overall = (Task + Agentic) / 2. These differ from full-agent results above.

<details>
<summary>Policy scores</summary>

| Model / training | Overall | Agentic | Task |
|---|---:|---:|---:|
| Qwen3.5-9B (base) | 19.9 | 11.3 | 28.4 |
| Qwen3.5-9B + SFT | 12.9 | 21.6 | 4.1 |
| Qwen3.5-9B + GRPO | 31.9 | 39.2 | 24.5 |
| BaT-9B (BiCuRL) | 53.4 | 64.7 | 42.1 |

</details>

## Evaluation settings

- **Qwen / BaT:** default Qwen thinking enabled, `xhigh` effort, 64K output tokens per request, 200K context window.
- **Other agents:** `xhigh` effort; Codex with 258K context and Claude Code with 1M context.
- **Harness freeze:** August 1, 2026. Latest stable releases by that date: [Codex 0.146.0](https://github.com/openai/codex/releases/tag/rust-v0.146.0) and [Claude Code 2.1.220](https://github.com/anthropics/claude-code/releases/tag/v2.1.220). These are release-date references; exact installed builds require run-log verification.

## External benchmarks

Direct model API evaluation, compared only with the corresponding base model under the same protocol. Interactive tasks retain benchmark tools and environments.

| Benchmark | Qwen3.5-9B (base) | BaT-9B | Change (pp) |
|---|---:|---:|---:|
| [AIME 2025](https://maa.org/maa-invitational-competitions/) | 88.3 | 83.6 | -4.7 |
| [AIME 2026](https://maa.org/maa-invitational-competitions/) | 92.5 | 86.7 | -5.8 |
| [GPQA-Diamond](https://arxiv.org/abs/2311.12022) | 81.7 | 78.3 | -3.4 |
| [τ²-Bench](https://arxiv.org/abs/2506.07982) | 79.1 | 84.5 | +5.4 |
| [BFCL-Parity](https://proceedings.mlr.press/v267/patil25a.html) | 79.4 | 68.3 | -11.1 |
| [GAIA](https://proceedings.iclr.cc/paper_files/paper/2024/hash/25ae35b5b1738d80f1f03a8713e405ec-Abstract-Conference.html) | 24.9 | 21.2 | -3.7 |
| [SWE-bench Verified](https://openai.com/index/introducing-swe-bench-verified/) | 49.0 | 51.8 | +2.8 |
| [Terminal Bench 2.0](https://arxiv.org/abs/2601.11868) | 19.9 | 21.3 | +1.4 |

<details>
<summary>Citation</summary>

```bibtex
@article{liu2026bat,
  title = {BaT: Towards Self-Evolving Medical Research Agent with Stage Rubrics},
  author = {Liu, Junqi and He, Yufan and He, Yexiao and Guo, Pengfei and Yang, Dong and Myronenko, Andriy and Zhao, Can and Ye, Hanrong and Qi, Tianhao and Zhou, Yuyin and Xu, Daguang and Tang, Yucheng},
  journal = {arXiv preprint arXiv:2608.16211},
  year = {2026},
  url = {https://arxiv.org/abs/2608.16211}
}
```

</details>