Instructions to use Dharma-AI/Dharma-OCR-LITE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dharma-AI/Dharma-OCR-LITE with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Dharma-AI/Dharma-OCR-LITE")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Dharma-AI/Dharma-OCR-LITE")
model = AutoModelForMultimodalLM.from_pretrained("Dharma-AI/Dharma-OCR-LITE", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Dharma-AI/Dharma-OCR-LITE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dharma-AI/Dharma-OCR-LITE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dharma-AI/Dharma-OCR-LITE",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Dharma-AI/Dharma-OCR-LITE

SGLang

How to use Dharma-AI/Dharma-OCR-LITE with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dharma-AI/Dharma-OCR-LITE" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dharma-AI/Dharma-OCR-LITE",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dharma-AI/Dharma-OCR-LITE" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dharma-AI/Dharma-OCR-LITE",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Dharma-AI/Dharma-OCR-LITE with Docker Model Runner:
```
docker model run hf.co/Dharma-AI/Dharma-OCR-LITE
```

Dharma-OCR-LITE / README.md

GabrielPimenta99

Update README.md

2d90c84 verified 4 months ago

6.06 kB

license: apache-2.0
language:
  - pt
  - en
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - ocr
  - document-understanding
  - structured-extraction
  - specialized-small-language-model
  - brazilian-portuguese
datasets:
  - dharma-ai/DharmaOCR-Benchmark

DharmaOCR Lite

Dharma-AI Logo

Introduction

DharmaOCR Lite is a 3B-parameter Specialized Small Language Model (SSLM) for structured OCR, developed by Dharma-AI. It extracts text from document images into a structured JSON format with explicit header, text, footer, and margin fields.

DharmaOCR Lite achieves state-of-the-art performance on DharmaOCR-Benchmark, outperforming all evaluated open-source and commercial baselines — including GPT-4o, GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro, Google Document AI, Amazon Textract, and olmOCR-2-7B — while being significantly cheaper and faster to run.

For the full methodology, training details, and ablation studies, see our paper: DharmaOCR: Specialized Small Language Models for Structured OCR that Outperform Open-Source and Commercial Baselines.

Why DharmaOCR Lite?

Best quality among all models evaluated

DharmaOCR Lite (3B) outperforms models with more than twice as many parameters, including olmOCR-2-7B, and surpasses every commercial API tested.

Model	Score	Degeneration Rate (%)	Time/Page (s)
DharmaOCR Full (7B, ours)	0.925	0.40	2.132
DharmaOCR Lite (3B, ours)	0.911	0.20	1.464
Claude Opus 4.6	0.833	—	—
Qwen3-VL-8B	0.829	5.65	7.250
olmOCR-2-7B	0.823	1.41	4.306
Gemini 3.1 Pro	0.820	—	—
Nanonets-OCR2-3B	0.791	2.62	1.911
GPT-5.4	0.750	—	—
GPT-4o	0.635	—	—
Google Document AI	0.640	—	—
Amazon Textract	0.618	—	—
Mistral OCR 3	0.574	—	—

Score = (LevenshteinRatio + BLEU) / 2, evaluated on DharmaOCR-Benchmark (496 instances covering printed, handwritten, and legal/administrative documents in Brazilian Portuguese). Time/page measured on NVIDIA L40S.

Lowest text degeneration rate

Text degeneration — where models get stuck in repetitive loops — is a critical but underreported problem in OCR. It's not just a quality issue: a single degenerate request can nearly double GPU time and cost for a batch of concurrent requests, reducing throughput for the entire system.

DharmaOCR Lite achieves a 0.20% degeneration rate, the lowest across all 22+ models evaluated (open-source and commercial).

Competitive cost

Model	Score	Relative Cost
DharmaOCR Lite (3B)	0.911	34%
olmOCR-2-7B	0.823	100% (reference)
Qwen3-VL-8B	0.829	168%
Nanonets-OCR2-3B	0.791	44%

DharmaOCR Lite is ~3× cheaper than olmOCR-2-7B while scoring ~10% higher. On an H200 GPU with optimized inference settings, cost drops even further.

Quickstart

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

model_name = "dharma-ai/DharmaOCR-Lite"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_name)

image_path = "document.png"

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image_path},
            {"type": "text", "text": "Extract the text from this image."},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=8192)
generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]

print(output_text)

Output Format

{
    "header": "DOCUMENT TITLE  |  Page 1",
    "margin": null,
    "footer": "Journal Name, Vol. X, pp. 1-10, 2025.",
    "text": "Full main body text extracted from the document..."
}

Fields not present in the original document are returned as null. This lets you decide whether to use auxiliary fields (header, footer, margins) depending on your application.

Serving with vLLM

vllm serve dharma-ai/DharmaOCR-Lite \
    --gpu-memory-utilization 0.90 \
    --max-model-len 65536 \
    --max-num-batched-tokens 32000

Limitations

Language focus: Primarily optimized for Brazilian Portuguese documents. Performance on other languages may vary.
Field repetition: The model may occasionally repeat header/footer content within the text field. A post-processing step checking exact matches between fields resolves >80% of occurrences.
Domain scope: Best results on printed, handwritten, and legal/administrative documents.

Citation

@article{dharmaocr2026,
    title={DharmaOCR: Specialized Small Language Models for Structured OCR that Outperform Open-Source and Commercial Baselines},
    author={Cardoso, Gabriel Pimenta de Freitas and Chacon, Caio Lucas da Silva and Oliveira, Jonas Felipe da Fonseca and Araujo, Paulo Henrique de Medeiros},
    year={2026},
    journal={arXiv preprint}
}

Contact

Gabriel Pimenta — gabriel.pimenta@dharma-ai.com
Caio Chacon — caio.chacon@dharma-ai.com
Jonas Oliveira — jonas.oliveira@dharma-ai.com
Paulo Araujo — paulo.araujo@dharma-ai.com