File size: 3,578 Bytes
64307a0 129b2b6 64307a0 e755e94 64307a0 e755e94 64307a0 e755e94 64307a0 e755e94 64307a0 c2f9899 64307a0 3e4cf30 64307a0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
license: apache-2.0
base_model: Qwen/Qwen3.5-2B
tags:
- medical
- vqa
- qwen3.5
- synthvision
pipeline_tag: visual-question-answering
---
# Qwen3.5-2B-MedVL
Qwen3.5-2B fine-tuned on ~200K medical VQA records from the SynthVision pipeline. Best overall model in the SynthVision family.
> **Note**: Requires `transformers>=5.3.0`. Do NOT install `fla` (crashes on Python 3.10).
## Benchmark Results (Exact Match)
| Split | VQA-RAD | PathVQA | SLAKE | Avg EM |
|-------|---------|---------|-------|--------|
| Base (Qwen3.5-2B) | 0.5477 | 0.3822 | 0.5617 | 0.4972 |
| **Fine-tuned** | **0.5521** | **0.4748** | **0.6880** | **0.5716** |
| Delta | +0.8% | +24.2% | +22.5% | +15.0% |
## Usage
### Transformers
```python
from transformers import AutoProcessor, AutoModelForImageTextToText
# Requires transformers>=5.3.0
model_id = "OpenMed/Qwen3.5-2B-MedVL"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [
{
"role": "user",
"content": [
{"type": "image", "url": "https://example.com/xray.jpg"},
{"type": "text", "text": "What are the key findings in this chest X-ray?"},
],
}
]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```
### vLLM
```python
from vllm import LLM, SamplingParams
llm = LLM(model="OpenMed/Qwen3.5-2B-MedVL", max_model_len=4096, limit_mm_per_prompt={"image": 1})
messages = [{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "https://example.com/xray.jpg"}},
{"type": "text", "text": "What are the key findings in this chest X-ray?"},
]}]
output = llm.chat(messages, SamplingParams(temperature=0, max_tokens=512))
print(output[0].outputs[0].text)
```
### SGLang
```bash
# Launch server
python -m sglang.launch_server --model-path OpenMed/Qwen3.5-2B-MedVL --port 8000
```
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="OpenMed/Qwen3.5-2B-MedVL",
messages=[{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "https://example.com/xray.jpg"}},
{"type": "text", "text": "What are the key findings in this chest X-ray?"},
]}],
max_tokens=512,
)
print(response.choices[0].message.content)
```
## Training Details
- **Base model**: [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B)
- **Data**: ~200K medical VQA records from the [SynthVision pipeline](https://huggingface.co/blog/OpenMed/synthvision)
- **Method**: LoRA (rank=64, alpha=64)
- **Target modules**: q_proj, v_proj, k_proj, o_proj, gate_proj, up_proj, down_proj
- **Learning rate**: 1e-4, cosine schedule
- **Steps**: 350
- **Weight decay**: 0.01
- **Hardware**: 4x NVIDIA A100 80GB (48 vCPU, 568 GB RAM) via [Hugging Face Jobs](https://huggingface.co/docs/hub/jobs)
- **Training time**: ~1h
## Links
- [SynthVision blog post](https://huggingface.co/blog/OpenMed/synthvision)
- [Source code](https://github.com/openmed-labs/synthvision)
- [All SynthVision artifacts](https://huggingface.co/collections/OpenMed/synthvision-69baac655b557943aa1babd3)
- [OpenMed on Hugging Face](https://huggingface.co/OpenMed)
|