---
license: apache-2.0
base_model: ATH-MaaS/OvisOCR2
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
  - mlx
  - mlx-vlm
  - quantized
  - apple-silicon
  - qwen3_5
  - ocr
  - document-parsing
  - markdown
  - tables
  - formulas
language:
  - en
  - zh
---

# ovisocr2-mxfp8-mlx

MLX MX FP8 (micro-scaled 8-bit float) quantization of [ATH-MaaS/OvisOCR2](https://huggingface.co/ATH-MaaS/OvisOCR2), converted with [mlx-vlm](https://github.com/Blaizzy/mlx-vlm).

**OvisOCR2** is a compact 0.8B end-to-end document-parsing model (Qwen3.5-0.8B backbone). Given a document page image, it generates a Markdown transcription in natural reading order — text, formulas (LaTeX), tables (HTML), and visual regions (bounding-box image tags). It scores 96.58 on OmniDocBench v1.6, state of the art for an end-to-end model, and an Avg3 of 75.06 on PureDocBench.

## Architecture

| | |
|---|---|
| Base model | OvisOCR2 (Qwen3.5-VL arch, `Qwen3_5ForConditionalGeneration`, ~0.8B params) |
| Quantization | MX FP8 (micro-scaled 8-bit float) |
| Bits per weight | 9.168 |
| Disk size | 958 MB (from ~1.6 GB bf16) |
| Format | MLX (safetensors) |

## Usage

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "sahilchachra/ovisocr2-mxfp8-mlx"
model, processor = load(model_path, trust_remote_code=True)
config = load_config(model_path, trust_remote_code=True)

prompt = (
    "\nExtract all readable content from the image in natural human reading order "
    "and output the result as a single Markdown document. For charts or images, "
    'represent them using an HTML image tag: <img src="images/bbox_{left}_{top}_{right}_{bottom}.jpg" />, '
    "where left, top, right, bottom are bounding box coordinates scaled to [0, 1000). "
    "Format formulas as LaTeX. Format tables as HTML: <table>...</table>. Transcribe all "
    "other text as standard Markdown. Preserve the original text without translation or paraphrasing."
)
formatted = apply_chat_template(processor, config, prompt, num_images=1, enable_thinking=False)
out = generate(model, processor, formatted, image="page.png", max_tokens=2048, verbose=False)
print(out.text if hasattr(out, "text") else out)
```

CLI:

```bash
mlx_vlm.generate --model sahilchachra/ovisocr2-mxfp8-mlx \
  --prompt "Extract all readable content from the image as Markdown." \
  --image page.png --max-tokens 2048
```

## Document-parsing accuracy (15-page held-out eval)

15 pages sampled (seed=42, English, single-column, >500 chars of text) from
[opendatalab/OmniDocBench](https://huggingface.co/datasets/opendatalab/OmniDocBench) —
the public benchmark OvisOCR2 reports its headline scores on (96.58 on OmniDocBench v1.6).
Ground truth is the concatenation of each page's annotated text/title/list/table/equation
regions in reading order. Each variant ran the model's own documented OCR prompt
(`mlx_vlm.generate`, greedy, `max_tokens=2048`), and outputs were compared against ground
truth with two metrics: character-level similarity (`difflib.SequenceMatcher`) and word
recall (fraction of ground-truth words present in the output).

| Variant | Bits/weight | Disk size | Mean similarity | Mean word recall | Degenerate outputs | Agreement with FP16 | Eval time (15 pages) |
|---|---|---|---|---|---|---|---|
| fp16 | 16 | 1.6 GB | 63.4% | 92.9% | 0/15 | — | 212.4s |
| mxfp4 | 5.643 | 599 MB | 63.2% | 93.1% | 0/15 | 96.5% | 138.1s |
| int4 | 5.863 | 622 MB | 63.5% | 93.2% | 0/15 | 96.1% | 140.7s |
| mxfp8 | 9.168 | 958 MB | 63.0% | 92.9% | 0/15 | 97.7% | 166.8s |
| int8 | 9.389 | 980 MB | 63.9% | 93.1% | 0/15 | 99.2% | 165.1s |

- **Word recall (~93%) is the more reliable signal here** — character-level similarity is
  pulled down by cosmetic LaTeX-formatting differences (e.g. the model emits `$...$` compact
  math, ground truth uses spaced `$ ... $` with `\left\{`/`\right\}`) even when the
  transcribed content is correct. Spot-checking the lowest-similarity page (an academic-paper
  equation block, sim≈3%) confirmed the model's output was in fact a faithful, correctly
  ordered transcription — the metric penalizes LaTeX style, not content.
- **Zero degenerate outputs** (no repetition loops, no empty/near-empty generations) across
  all 75 generations (15 pages × 5 variants, including the fp16 baseline).
- **All four quantized variants agree with FP16's output at the character level 96–99% of the
  time** — quantization barely perturbs the transcription, even at 4-bit (both mxfp4 and int4).
- **4-bit vs 8-bit is a wash on quality here**: mxfp4/int4 score essentially the same word
  recall as mxfp8/int8 on this OCR task, while being ~35% faster and roughly 40% smaller on
  disk — for OvisOCR2 specifically, the 4-bit variants are the better default.
- mxfp4 and int4 are nearly identical to each other in every metric; pick whichever fits your
  serving stack (MX FP4 vs plain affine int4) — there's no accuracy reason to prefer one.


## Other MLX variants

- [ovisocr2-mxfp4-mlx](https://huggingface.co/sahilchachra/ovisocr2-mxfp4-mlx)
- [ovisocr2-int4-mlx](https://huggingface.co/sahilchachra/ovisocr2-int4-mlx)
- [ovisocr2-mxfp8-mlx](https://huggingface.co/sahilchachra/ovisocr2-mxfp8-mlx)
- [ovisocr2-int8-mlx](https://huggingface.co/sahilchachra/ovisocr2-int8-mlx)

## Credits

- Base model: [ATH-MaaS/OvisOCR2](https://huggingface.co/ATH-MaaS/OvisOCR2)
- Eval data: [opendatalab/OmniDocBench](https://huggingface.co/datasets/opendatalab/OmniDocBench)
- Quantization tooling: [mlx-vlm](https://github.com/Blaizzy/mlx-vlm)