VERA
Collection
Scaling Verifiable Environments for Agentic co-Evolution • 7 items • Updated
Request access to VERA
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Please provide your affiliation and a brief description of your research. Access requests are reviewed manually.
Log in or Sign Up to review the conditions and access this model content.
VERA-Med-27B-1001
VERA co-evolves model weights and agent skills in verified environments. This Qwen3.8-27B model targets medical-imaging research workflows.
Training scope: VERA post-training uses agentic workflow data only, without dedicated CoWork or medical reasoning datasets. This describes VERA post-training, not the base model’s pretraining. Benchmark scores do not establish clinical expertise. Research use only; not for diagnosis or treatment.
From Qwen to VERA
Qwen and VERA use Codex with xhigh thinking effort. Paper results use Figure 1’s productivity-harness configurations; HealthBench reference scores use published setups. The animation interpolates measured endpoints, not training steps.
HealthBench scores use official length-adjusted scoring.
Download video · Static comparison
Video sources: Paper, Figure 1 and HealthBench references, Table 6. Qwen 41.4 and VERA 48.8 on HealthBench Professional are author-reported Codex xhigh results with official length-adjusted scoring.
Agent comparisons
Original paper figure: model-size scaling, frontier agents, and transfer benchmarks. Scores describe complete model and harness combinations.
VERA uses its evolved Codex harness. Qwen, DeepSeek, and GPT use Codex v0.153.4; GLM and Claude use Claude Code v2.1.278. All use xhigh effort and each benchmark's scorer. Automation-Bench uses the public 600 tasks (v1.0.6); AgentClinic uses its official simulator.
AutoMedBench: model and harness
Paper Table 3. Default = the benchmark-native ReAct loop with model API calls. Agentic measures workflow quality; Task measures task outcomes. Overall = (Agentic + Task) / 2. These configurations differ from the figure above.
| Model | Harness | Overall | Agentic | Task |
|---|---|---|---|---|
| Qwen3.8-27B | Default | 65.1 | 81.6 | 48.6 |
| GPT-5.6-Sol | Default | 75.1 | 80.1 | 70.0 |
| Claude-Opus-4.8 | Default | 81.9 | 88.1 | 75.8 |
| Qwen3.8-27B | VERA | 54.2 | 70.1 | 38.3 |
| VERA-Med-27B | Default | 75.4 | 91.7 | 59.2 |
| VERA-Med-27B | VERA | 80.7 | 94.1 | 67.4 |
External benchmarks
Direct model API evaluation against the base model under the same protocol (paper Table 4). AIME and GPQA use zero-shot prompts; ALFWorld retains its TextWorld environment, and IF-Bench uses its official checker.
| Benchmark | Qwen3.8-27B | VERA-Med-27B | Change (percentage points) |
|---|---|---|---|
| AIME 2026 | 86.7 | 90.0 | +3.3 |
| ALFWorld | 94.0 | 90.0 | -4.0 |
| GPQA-Diamond | 82.2 | 87.8 | +5.6 |
| IF-Bench | 79.5 | 78.8 | -0.7 |
Paper evaluation settings
VERA uses SGLang with thinking enabled and preserved, xhigh effort, a 262,144-token context window, and compaction at 200,000 tokens. Harness versions stay fixed throughout evaluation. Benchmark-specific limits apply; Automation-Bench allows 8,192 output tokens and 50 actions. See the paper's Evaluation Protocols and Agent Harness appendix.
Format
Native Megatron torch_dist model weights, with architecture in base-model-config.json and tokenizer assets in tokenizer/. Requires a compatible Megatron runtime or conversion before Transformers loading. This repository excludes optimizer and RNG state.
Citation
@misc{liu2026verascalingverifiableenvironments,
title={VERA: Scaling Verifiable Environments for Agentic co-Evolution},
author={Junqi Liu and Yongyang Pan and Zhuosong Jiang and Dongbai Li and Bo Zhang and Xitong Ling and Sheng Wang and Hanrong Ye and Yufan He and Can Zhao and Pengfei Guo and Dong Yang and Andriy Myronenko and Yuyin Zhou and Tianyu Liu and Daguang Xu and Yucheng Tang},
year={2026},
eprint={2610.05923},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2610.05923}
}
Model tree for MitakaKuma/VERA-Med-27B-1001
Base model
Qwen/Qwen3.8-27B