File size: 2,096 Bytes
c29257f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 | # AREX-Base Inference
This folder provides a minimal one-turn inference example and the complete BrowseComp prompts. It follows the XML tool-call protocol used by the public AREX evaluation code.
## Serve the model
Run the following commands from the model repository root. Recent versions of vLLM, SGLang, or another OpenAI-compatible server with Qwen3.5 support can be used. For a text-only vLLM deployment:
```bash
vllm serve . \
--served-model-name AREX-Base \
--tensor-parallel-size 8 \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--language-model-only
```
The example uses eight-way tensor parallelism as a starting point. Adjust the parallelism and maximum context length for your hardware.
## Run one generation
Install the client:
```bash
pip install -U openai
```
Then send a BrowseComp-style question:
```bash
export AREX_BASE_URL="http://127.0.0.1:8000/v1"
export AREX_API_KEY="EMPTY"
export AREX_MODEL="AREX-Base"
python inference/inference.py \
--question "Your BrowseComp question"
```
The script returns the model's next action. When it emits an XML `<tool_call>`, execute that tool, append the assistant output to the message history, and add the real tool result as:
```text
<tool_response>
actual tool result
</tool_response>
```
Continue until the model calls `finish`. The example intentionally leaves tool execution to the caller.
## Use the prompts directly
[`prompts.py`](prompts.py) exports the BrowseComp system and user prompt constants. Tool descriptions are already embedded in the system prompt, so only the question needs formatting:
```python
from inference.prompts import (
BROWSECOMP_SYSTEM_PROMPT,
BROWSECOMP_USER_PROMPT,
)
question = "Your BrowseComp question"
messages = [
{"role": "system", "content": BROWSECOMP_SYSTEM_PROMPT},
{
"role": "user",
"content": BROWSECOMP_USER_PROMPT.format(question=question),
},
]
```
`build_messages(question)` is a convenience wrapper for the same formatting.
BrowseComp exposes `search`, `google_scholar`, `visit`, `update_context`, and `finish`.
|