--- license: other language: - pt - en library_name: transformers base_model: Nanonets/Nanonets-OCR2-3B pipeline_tag: image-text-to-text tags: - ocr - document-understanding - structured-extraction - dpo - sft - awq - specialized-small-language-model - brazilian-portuguese datasets: - dharma-ai/DharmaOCR-Benchmark --- # DharmaOCR Lite

Dharma-AI Logo

## Introduction **DharmaOCR Lite** is a 3B-parameter Specialized Small Language Model (SSLM) for structured OCR, developed by [Dharma-AI](https://dharma-ai.com). It is designed to extract text from document images into a structured JSON format with explicit `header`, `text`, `footer`, and `margin` fields. DharmaOCR Lite achieves **state-of-the-art performance** on [DharmaOCR-Benchmark](https://huggingface.co/datasets/dharma-ai/DharmaOCR-Benchmark), outperforming all evaluated open-source and commercial baselines — including GPT-4o, GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro, Google Document AI, Amazon Textract, and olmOCR-2-7B — while being significantly cheaper and faster to run. ### Key Highlights - **Score: 0.911** on DharmaOCR-Benchmark (highest among all 3B models, surpasses all commercial APIs evaluated) - **Text degeneration rate: 0.20%** — the lowest across all models tested - **~34% relative cost** compared to olmOCR-2-7B vanilla (reference baseline) - **1.464s average time per page** on an NVIDIA L40S GPU - Trained with a novel **SFT + DPO** pipeline, where DPO is applied to OCR for the first time to explicitly reduce text degeneration - **AWQ-quantized** (FP8) for efficient deployment with negligible quality loss ## Model Details | Attribute | Value | |---|---| | **Base model** | [Nanonets-OCR2-3B](https://huggingface.co/Nanonets/Nanonets-OCR2-3B) | | **Architecture** | Qwen2.5-VL-3B-Instruct (decoder-only VLM) | | **Parameters** | ~3B | | **Training pipeline** | SFT → DPO → AWQ Quantization (FP8) | | **Output format** | Structured JSON (`header`, `text`, `footer`, `margin`) | | **Primary language** | Brazilian Portuguese | | **Context length** | 8,192 tokens (generation limit) | | **Precision** | FP8 (AWQ) | | **License** | Apache 2.0 | ## Performance ### DharmaOCR-Benchmark Results | Model | Score | Degeneration Rate (%) | Time/Page (s) | |---|---|---|---| | **DharmaOCR Lite (ours)** | **0.911** | **0.20** | **1.464** | | DharmaOCR Full (ours, 7B) | 0.925 | 0.40 | 2.132 | | olmOCR-2-7B (vanilla) | 0.823 | 1.41 | 4.306 | | Nanonets-OCR2-3B (vanilla) | 0.791 | 2.62 | 1.911 | | Claude Opus 4.6 | 0.833 | — | — | | Gemini 3.1 Pro | 0.820 | — | — | | GPT-5.4 | 0.750 | — | — | | GPT-4o | 0.635 | — | — | | Google Document AI | 0.640 | — | — | | Amazon Textract | 0.618 | — | — | > **DharmaOCR Lite (3B) outperforms olmOCR-2-7B (7B)** — a model with more than twice as many parameters — by ~10% in benchmark score, while being ~3× faster and ~66% cheaper per page. ### Benchmark Score Definition The DharmaOCR-Benchmark Score is defined as: ``` score = (LevenshteinRatio + BLEU) / 2 ``` where `LevenshteinRatio` captures character-level fidelity and `BLEU` measures n-gram sequence preservation. ## Quickstart ```python from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor from qwen_vl_utils import process_vision_info from PIL import Image model_name = "dharma-ai/DharmaOCR-Lite" model = Qwen2_5_VLForConditionalGeneration.from_pretrained( model_name, torch_dtype="auto", device_map="auto", ) processor = AutoProcessor.from_pretrained(model_name) # Load your document image image_path = "document.png" messages = [ { "role": "user", "content": [ {"type": "image", "image": image_path}, {"type": "text", "text": "Extract the text from this image."}, ], } ] text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) image_inputs, video_inputs = process_vision_info(messages) inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt", ).to(model.device) generated_ids = model.generate(**inputs, max_new_tokens=8192) generated_ids_trimmed = [ out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids) ] output_text = processor.batch_decode( generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False )[0] print(output_text) ``` ### Expected Output Format ```json { "header": "DOCUMENT TITLE | Page 1", "margin": null, "footer": "Journal Name, Vol. X, pp. 1-10, 2025.", "text": "Full main body text extracted from the document..." } ``` Fields that are not present in the original document are returned as `null`. ## Serving with vLLM ```bash vllm serve dharma-ai/DharmaOCR-Lite \ --gpu-memory-utilization 0.90 \ --max-model-len 65536 \ --max-num-batched-tokens 32000 ``` ## Training Details ### Pipeline 1. **Supervised Fine-Tuning (SFT):** Full fine-tuning on ~39,680 pages of diverse document types (printed, handwritten, tables, forms, mixed layouts) predominantly in Brazilian Portuguese. Labels were generated by large LLMs (Claude Sonnet 4, Llama 4 Maverick, Gemini 2.5 Pro) and validated by human reviewers. 2. **Direct Preference Optimization (DPO):** A novel application of DPO to OCR — degenerate model outputs were explicitly used as *rejected* examples to penalize looping/repetitive behavior. Preference pairs were constructed from 5 candidate responses per document, scored by Qwen3-VL-235B as judge, and filtered through a multi-stage policy combining Selective-DPO principles and reward-margin analysis. Final dataset: 49,170 preference pairs. 3. **AWQ Quantization (FP8):** Activation-aware weight quantization preserving visual encoder and output layer in full precision. Reduces cost by ~12% with minimal quality degradation (score: 0.921 → 0.911). ### Infrastructure - **SFT:** 1× NVIDIA H200, 24 CPU cores, 256 GB RAM - **DPO:** 4× NVIDIA H200, 96 CPU cores, 1024 GB RAM - **Hyperparameters:** Effective batch size 32, AdamW optimizer, cosine LR schedule ## Why DPO for OCR? Text degeneration — where models get stuck in repetitive loops — is a critical but underreported problem in OCR systems. It's not just a quality issue: degenerate requests consume disproportionate GPU time, reduce throughput for *all* concurrent requests, and inflate costs. DharmaOCR Lite applies DPO to reduce degeneration by **87.6%** relative to SFT-only training (from 1.61% → 0.20%), the largest reduction observed across all model families tested. ## DharmaOCR-Benchmark We release [DharmaOCR-Benchmark](https://huggingface.co/datasets/dharma-ai/DharmaOCR-Benchmark), a 496-instance evaluation suite for OCR in Brazilian Portuguese covering: - **ESTER-Pt** (363 samples): Printed text recognition - **Legal** (83 samples): Legal and administrative documents - **BRESSAY** (50 samples): Handwritten text recognition The benchmark evaluates transcription quality, text degeneration rate, and unit inference cost under a unified protocol. ## Limitations - **Language focus:** Primarily optimized for Brazilian Portuguese documents. Performance on other languages may vary. - **Field repetition:** The model may occasionally repeat header/footer content within the `text` field. A post-processing step checking exact matches resolves >80% of occurrences. - **Domain scope:** Best results on printed, handwritten, and legal/administrative documents. Highly specialized layouts (e.g., complex engineering drawings) are not covered. ## Citation ```bibtex @article{dharmaocr2026, title={DharmaOCR: Specialized Small Language Models for Structured OCR that Outperform Open-Source and Commercial Baselines}, author={Cardoso, Gabriel Pimenta de Freitas and Chacon, Caio Lucas da Silva and Oliveira, Jonas Felipe da Fonseca and Araujo, Paulo Henrique de Medeiros}, year={2026}, journal={arXiv preprint} } ``` ## Contact - Gabriel Pimenta — gabriel.pimenta@dharma-ai.com - Caio Chacon — caio.chacon@dharma-ai.com - Jonas Oliveira — jonas.oliveira@dharma-ai.com - Paulo Araujo — paulo.araujo@dharma-ai.com