---
license: mit
base_model: baidu/Unlimited-OCR
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
tags:
  - nvfp4
  - fp4
  - compressed-tensors
  - llm-compressor
  - ocr
  - deepseek-ocr
  - vision-language
  - moe
  - quantized
  - document-parsing
language:
  - multilingual
---

# Unlimited-OCR — NVFP4

NVFP4 (4-bit float) quantization of [**baidu/Unlimited-OCR**](https://huggingface.co/baidu/Unlimited-OCR),
a 3B vision-language OCR model that pushes **DeepSeek-OCR** further (one-shot, long-horizon
document parsing). Architecture: a SAM+CLIP *DeepEncoder* vision tower → linear projector →
**DeepSeek-V2 MoE** text decoder (64 routed + 2 shared experts, 6/token, 12 layers).

**What is quantized:** the **text-decoder Linear weights** → `NVFP4A16` (4-bit FP, group size 16).
**Kept in BF16:** the vision tower (`sam_model`, `vision_model`), the projector, token embeddings,
`lm_head`, the **MoE router gate**, and all norms — quantizing those hurts OCR accuracy for little
size gain.

| | |
|---|---|
| **Size** | ~2.93 GB (vs 6.67 GB BF16) |
| **Quantized Linears** | 2196 (2112 experts + 33 shared + 48 attention + 3 dense) |
| **Scheme** | `NVFP4A16` · `nvfp4-pack-quantized` (group 16) |
| **Method** | data-free (`llm-compressor` `model_free_ptq`) — no calibration |
| **Quantized by** | [sahilchachra](https://huggingface.co/sahilchachra) |

> **Hardware:** NVFP4 runs natively on **Blackwell** GPUs (Jetson Thor, RTX 50-series, B200).
> On other GPUs, `compressed-tensors` transparently dequantizes the weights at load. Needs
> `transformers` 4.57.x + `compressed-tensors` + `trust_remote_code=True` (custom model code).

## Verified

Loaded in `transformers` and run on a test document — OCR output is **identical to BF16**, e.g.:
```
<|det|>title [37, 64, 464, 130]<|/det|>INVOICE #2026-0623
<|det|>text  [37, 191, 350, 243]<|/det|>Bill To: Sahil Chachra
<|det|>text  [37, 480, 329, 540]<|/det|>Total Due: $44.00
```

## Usage

```python
import torch
from transformers import AutoModel, AutoTokenizer

repo = "sahilchachra/Unlimited-OCR-NVFP4"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True,
                                  dtype=torch.bfloat16, device_map="cuda").eval()

text = model.infer(
    tok,
    prompt="<image>\n<|grounding|>Convert the document to markdown.",
    image_file="document.png", output_path="./out",
    base_size=1024, image_size=1024, crop_mode=False,   # "base" mode
    save_results=True, eval_mode=True,
)
print(text)
```
For dense/large pages use **gundam** mode (`base_size=1024, image_size=640, crop_mode=True`).

### Prompting guide

The prompt **must contain `<image>`**. Add `<|grounding|>` when you also want bounding boxes.

| Task | Prompt |
|---|---|
| Document → Markdown (with boxes) | `<image>\n<|grounding|>Convert the document to markdown.` |
| Plain text OCR | `<image>\nFree OCR.` |
| Native parse | `<image>document parsing.` |
| Parse a figure / chart | `<image>\nParse the figure.` |
| Find specific text | `<image>\n<|grounding|>Locate <|ref|>Total Due<|/ref|> in the image.` |
| Multi-page / PDF | `<image>Multi page parsing.` via `model.infer_multi(...)` |

**Output:** with `<|grounding|>`, text is interleaved with `<|det|>label [x1,y1,x2,y2]<|/det|>`
detection boxes (pixel coords in the model's input space). Strip the tags for plain text, or
parse them to rebuild layout.

## Other formats

- **AWQ (W4A16):** [sahilchachra/Unlimited-OCR-AWQ](https://huggingface.co/sahilchachra/Unlimited-OCR-AWQ)
- **GGUF (llama.cpp):** [sahilchachra/Unlimited-OCR-GGUF](https://huggingface.co/sahilchachra/Unlimited-OCR-GGUF)
- **MLX:** [sahilchachra/unlimited-ocr-8bit-mlx](https://huggingface.co/sahilchachra/unlimited-ocr-8bit-mlx)

## Credits

Base model [baidu/Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) (MIT), built on
[DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR). Quantized with
[llm-compressor](https://github.com/vllm-project/llm-compressor). License: MIT.