---
license: apache-2.0
language:
  - pt
  - en
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - ocr
  - document-understanding
  - structured-extraction
  - specialized-small-language-model
  - brazilian-portuguese
datasets:
  - dharma-ai/DharmaOCR-Benchmark
---

<p align="left">
  <img src="images/logo_horizontal_black_whiteback.png" width="400" alt="Dharma-AI Logo"/>
</p>

# DharmaOCR Lite

<p align="center">
  <img src="images/overview.png" width="600"/>
</p>

## Introduction

**DharmaOCR Lite** is a 3B-parameter Specialized Small Language Model (SSLM) for structured OCR, developed by [Dharma-AI](https://dharma-ai.com). It extracts text from document images into a structured JSON format with explicit `header`, `text`, `footer`, and `margin` fields.

DharmaOCR Lite achieves **state-of-the-art performance** on [DharmaOCR-Benchmark](https://huggingface.co/datasets/dharma-ai/DharmaOCR-Benchmark), outperforming all evaluated open-source and commercial baselines — including GPT-4o, GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro, Google Document AI, Amazon Textract, and olmOCR-2-7B — while being significantly cheaper and faster to run.

For the full methodology, training details, and ablation studies, see our paper: **[DharmaOCR: Specialized Small Language Models for Structured OCR that Outperform Open-Source and Commercial Baselines](link_to_paper)**.

<p align="center">
  <img src="images/cost_x_score.png" width="600"/>
</p>

## Why DharmaOCR Lite?

### Best quality among all models evaluated

DharmaOCR Lite (3B) **outperforms models with more than twice as many parameters**, including olmOCR-2-7B, and surpasses every commercial API tested.

### 🏆 DharmaOCR-Benchmark Results

<table>
<thead>
<tr>
<th>Model</th>
<th>Score ↑</th>
<th>Degeneration Rate (%) ↓</th>
<th>Time/Page (s) ↓</th>
</tr>
</thead>

<tbody>
<tr>
<td>🥇 <b>DharmaOCR Full</b> (7B, ours)</td>
<td><b>0.925</b></td>
<td><b>0.40</b></td>
<td>2.132</td>
</tr>
<tr>
<td>🥈 <b>DharmaOCR Lite</b> (3B, ours)</td>
<td><b>0.911</b></td>
<td><b>0.20 ✨</b></td>
<td><b>1.464</b></td>
</tr>

<tr><td colspan="4"><br><b>Commercial APIs</b></td></tr>

<tr>
<td>Claude Opus 4.6</td>
<td>0.833</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Gemini 3.1 Pro</td>
<td>0.820</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>GPT-5.4</td>
<td>0.750</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Google Vision</td>
<td>0.686</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Google Document AI</td>
<td>0.640</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>GPT-4o</td>
<td>0.635</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Amazon Textract</td>
<td>0.618</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Mistral OCR 3</td>
<td>0.574</td>
<td>—</td>
<td>—</td>
</tr>

<tr><td colspan="4"><br><b>Open-Source Models</b></td></tr>

<tr>
<td>Qwen2.5-VL-7B-Instruct</td>
<td>0.839</td>
<td>2.42</td>
<td>3.101</td>
</tr>
<tr>
<td>Qwen3-VL-8B</td>
<td>0.829</td>
<td>5.65</td>
<td>7.250</td>
</tr>
<tr>
<td>olmOCR-2-7B</td>
<td>0.823</td>
<td>1.41</td>
<td>4.306</td>
</tr>
<tr>
<td>Nanonets-OCR2-3B</td>
<td>0.791</td>
<td>2.62</td>
<td>1.911</td>
</tr>
<tr>
<td>Dots OCR</td>
<td>0.738</td>
<td>6.85</td>
<td>2.526</td>
</tr>
<tr>
<td>GLM-OCR</td>
<td>0.710</td>
<td>11.69</td>
<td>1.480</td>
</tr>
<tr>
<td>Qwen3-VL-2B-Instruct</td>
<td>0.623</td>
<td>11.69</td>
<td>3.566</td>
</tr>
<tr>
<td>Qwen2.5-VL-3B-Instruct</td>
<td>0.549</td>
<td>0.60</td>
<td>1.500</td>
</tr>
<tr>
<td>gemma-3-4b-it</td>
<td>0.214</td>
<td>33.96</td>
<td>2.182</td>
</tr>
<tr>
<td>DeepSeek-OCR</td>
<td>0.196</td>
<td>21.98</td>
<td>1.213</td>
</tr>
</tbody>
</table>

Evaluated on [DharmaOCR-Benchmark](https://huggingface.co/datasets/dharma-ai/DharmaOCR-Benchmark) (instances covering printed, handwritten, and legal/administrative documents in Brazilian Portuguese). Time/page measured on NVIDIA L40S.

### Lowest text degeneration rate

Text degeneration — where models get stuck in repetitive loops — is a critical but underreported problem in OCR. It's not just a quality issue: **a single degenerate request can nearly double GPU time and cost** for a batch of concurrent requests, reducing throughput for the entire system.

<p align="center">
  <img src="images/example_text_degeneration.png" width="400"/>
</p>

DharmaOCR Lite achieves a **0.20% degeneration rate** ✨ — the lowest across all 22+ models evaluated (open-source and commercial).

<table>
<thead>
<tr>
<th>Model</th>
<th>Degeneration Rate (%)</th>
<th>vs DharmaOCR Lite</th>
</tr>
</thead>
<tbody>
<tr>
<td>✨ <b>DharmaOCR Lite</b> (3B, ours)</td>
<td><b>0.20</b></td>
<td>—</td>
</tr>
<tr>
<td><b>DharmaOCR Full</b> (7B, ours)</td>
<td>0.40</td>
<td>2×</td>
</tr>
<tr>
<td>Qwen2.5-VL-3B-Instruct</td>
<td>0.60</td>
<td>3×</td>
</tr>
<tr>
<td>olmOCR-2-7B</td>
<td>1.41</td>
<td>7×</td>
</tr>
<tr>
<td>Qwen2.5-VL-7B-Instruct</td>
<td>2.42</td>
<td>12×</td>
</tr>
<tr>
<td>Nanonets-OCR2-3B</td>
<td>2.62</td>
<td>13×</td>
</tr>
<tr>
<td>Qwen3-VL-8B</td>
<td>5.65</td>
<td>28×</td>
</tr>
<tr>
<td>Dots OCR</td>
<td>6.85</td>
<td>34×</td>
</tr>
<tr>
<td>GLM-OCR</td>
<td>11.69</td>
<td>58×</td>
</tr>
<tr>
<td>Qwen3-VL-2B-Instruct</td>
<td>11.69</td>
<td>58×</td>
</tr>
<tr>
<td>DeepSeek-OCR</td>
<td>21.98</td>
<td>110×</td>
</tr>
<tr>
<td>gemma-3-4b-it</td>
<td>33.96</td>
<td>170×</td>
</tr>
</tbody>
</table>

<sub>Degeneration rate measured on <a href="https://huggingface.co/datasets/dharma-ai/DharmaOCR-Benchmark">DharmaOCR-Benchmark</a>. "vs DharmaOCR Lite" = how many times more degeneration than DharmaOCR Lite (0.20%).</sub>

### Competitive cost

<table>
<thead>
<tr>
<th>Model</th>
<th>Score ↑</th>
<th>Relative Cost ↓</th>
</tr>
</thead>
<tbody>
<tr>
<td>🥇 <b>DharmaOCR Lite</b> (3B, ours)</td>
<td><b>0.911</b></td>
<td><b>34%</b></td>
</tr>
<tr>
<td>Nanonets-OCR2-3B</td>
<td>0.791</td>
<td>44%</td>
</tr>
<tr>
<td>Dots OCR</td>
<td>0.738</td>
<td>59%</td>
</tr>
<tr>
<td>GLM-OCR</td>
<td>0.710</td>
<td>34%</td>
</tr>
<tr>
<td>Qwen3-VL-2B-Instruct</td>
<td>0.623</td>
<td>83%</td>
</tr>
<tr>
<td>Qwen2.5-VL-3B-Instruct</td>
<td>0.549</td>
<td>35%</td>
</tr>
<tr>
<td>olmOCR-2-7B</td>
<td>0.823</td>
<td>100% (reference)</td>
</tr>
<tr>
<td>Qwen2.5-VL-7B-Instruct</td>
<td>0.839</td>
<td>72%</td>
</tr>
<tr>
<td>Qwen3-VL-8B</td>
<td>0.829</td>
<td>168%</td>
</tr>
<tr>
<td>DeepSeek-OCR</td>
<td>0.196</td>
<td>28%</td>
</tr>
<tr>
<td>gemma-3-4b-it</td>
<td>0.214</td>
<td>51%</td>
</tr>
</tbody>
</table>

<sub>Relative cost computed against olmOCR-2-7B vanilla (100%) on NVIDIA L40S with identical vLLM configuration. Lower is better.</sub>

DharmaOCR Lite is **~3× cheaper** than olmOCR-2-7B while scoring ~10% higher. Among models with comparable cost (DeepSeek-OCR, GLM-OCR, Qwen2.5-VL-3B), DharmaOCR Lite scores **2× to 4.6× higher**. On an H200 GPU with optimized inference settings, cost drops even further.


## Quickstart

```python
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

model_name = "dharma-ai/DharmaOCR-Lite"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_name)

image_path = "document.png"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image_path},
            {"type": "text", "text": "Extract the text from this image."},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=8192)
generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]

print(output_text)
```

### Output Format

```json
{
    "header": "DOCUMENT TITLE  |  Page 1",
    "margin": null,
    "footer": "Journal Name, Vol. X, pp. 1-10, 2025.",
    "text": "Full main body text extracted from the document..."
}
```

Fields not present in the original document are returned as `null`. This lets you decide whether to use auxiliary fields (header, footer, margins) depending on your application.

## Serving with vLLM

```bash
vllm serve dharma-ai/DharmaOCR-Lite \
    --gpu-memory-utilization 0.90 \
    --max-model-len 65536 \
    --max-num-batched-tokens 32000
```

## Limitations

- **Language focus:** Primarily optimized for Brazilian Portuguese documents. Performance on other languages may vary.
- **Field repetition:** The model may occasionally repeat header/footer content within the `text` field. A post-processing step checking exact matches between fields resolves >80% of occurrences.
- **Domain scope:** Best results on printed, handwritten, and legal/administrative documents.

## Citation

```bibtex
@article{dharmaocr2026,
    title={DharmaOCR: Specialized Small Language Models for Structured OCR that Outperform Open-Source and Commercial Baselines},
    author={Cardoso, Gabriel Pimenta de Freitas and Chacon, Caio Lucas da Silva and Oliveira, Jonas Felipe da Fonseca and Araujo, Paulo Henrique de Medeiros},
    year={2026},
    journal={arXiv preprint}
}
```

## Contact

- Gabriel Pimenta — gabriel.pimenta@dharma-ai.com
- Caio Chacon — caio.chacon@dharma-ai.com
- Jonas Oliveira — jonas.oliveira@dharma-ai.com
- Paulo Araujo — paulo.araujo@dharma-ai.com