---
license: apache-2.0
base_model: ATH-MaaS/OvisOCR2
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
  - mlx
  - mlx-vlm
  - quantized
  - apple-silicon
  - qwen3_5
  - ocr
  - document-parsing
  - markdown
  - tables
  - formulas
language:
  - en
  - zh
---

# ovisocr2-mxfp8-mlx

MLX MX FP8 (micro-scaled 8-bit float) quantization of [ATH-MaaS/OvisOCR2](https://huggingface.co/ATH-MaaS/OvisOCR2), converted with [mlx-vlm](https://github.com/Blaizzy/mlx-vlm).

**OvisOCR2** is a compact 0.8B end-to-end document-parsing model (Qwen3.5-0.8B backbone). Given a document page image, it generates a Markdown transcription in natural reading order — text, formulas (LaTeX), tables (HTML), and visual regions (bounding-box image tags). It scores 96.58 on OmniDocBench v1.6, state of the art for an end-to-end model, and an Avg3 of 75.06 on PureDocBench.

## Architecture

| | |
|---|---|
| Base model | OvisOCR2 (Qwen3.5-VL arch, `Qwen3_5ForConditionalGeneration`, ~0.8B params) |
| Quantization | MX FP8 (micro-scaled 8-bit float) |
| Bits per weight | 9.168 |
| Disk size | 958 MB (from ~1.6 GB bf16) |
| Format | MLX (safetensors) |

## Usage

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "sahilchachra/ovisocr2-mxfp8-mlx"
model, processor = load(model_path, trust_remote_code=True)
config = load_config(model_path, trust_remote_code=True)

prompt = (
    "\nExtract all readable content from the image in natural human reading order "
    "and output the result as a single Markdown document. For charts or images, "
    'represent them using an HTML image tag: <img src="images/bbox_{left}_{top}_{right}_{bottom}.jpg" />, '
    "where left, top, right, bottom are bounding box coordinates scaled to [0, 1000). "
    "Format formulas as LaTeX. Format tables as HTML: <table>...</table>. Transcribe all "
    "other text as standard Markdown. Preserve the original text without translation or paraphrasing."
)
formatted = apply_chat_template(processor, config, prompt, num_images=1, enable_thinking=False)
out = generate(model, processor, formatted, image="page.png", max_tokens=2048, verbose=False)
print(out.text if hasattr(out, "text") else out)
```

CLI:

```bash
mlx_vlm.generate --model sahilchachra/ovisocr2-mxfp8-mlx \
  --prompt "Extract all readable content from the image as Markdown." \
  --image page.png --max-tokens 2048
```

## Document-parsing accuracy (15-page held-out eval)

15 pages sampled (seed=42, English, single-column, >500 chars of text) from
[opendatalab/OmniDocBench](https://huggingface.co/datasets/opendatalab/OmniDocBench) —
the public benchmark OvisOCR2 reports its headline scores on (96.58 on OmniDocBench v1.6).
Ground truth is the concatenation of each page's annotated text/title/list/table/equation
regions in reading order. Each variant ran the model's own documented OCR prompt
(`mlx_vlm.generate`, greedy, `max_tokens=2048`), and outputs were compared against ground
truth with two metrics: character-level similarity (`difflib.SequenceMatcher`) and word
recall (fraction of ground-truth words present in the output).

| Variant | Mean similarity | Mean word recall | Degenerate outputs | Agreement with FP16 |
|---|---|---|---|---|
| fp16 | 63.4% | 92.9% | 0/15 | — |
| mxfp8 | 63.0% | 92.9% | 0/15 | 97.7% |
| int8 | 63.9% | 93.1% | 0/15 | 99.2% |

- **Word recall (~93%) is the more reliable signal here** — character-level similarity is
  pulled down by cosmetic LaTeX-formatting differences (e.g. the model emits `$...$` compact
  math, ground truth uses spaced `$ ... $` with `\left\{`/`\right\}`) even when the
  transcribed content is correct. Spot-checking the lowest-similarity page (an academic-paper
  equation block, sim≈3%) confirmed the model's output was in fact a faithful, correctly
  ordered transcription — the metric penalizes LaTeX style, not content.
- **Zero degenerate outputs** (no repetition loops, no empty/near-empty generations) across
  all 45 generations (15 pages × 3 variants).
- Both quantized variants agree with FP16's output at the character level 97–99% of the
  time — quantization barely perturbs the transcription.


## Other MLX variants

- [ovisocr2-mxfp8-mlx](https://huggingface.co/sahilchachra/ovisocr2-mxfp8-mlx)
- [ovisocr2-int8-mlx](https://huggingface.co/sahilchachra/ovisocr2-int8-mlx)

## Credits

- Base model: [ATH-MaaS/OvisOCR2](https://huggingface.co/ATH-MaaS/OvisOCR2)
- Eval data: [opendatalab/OmniDocBench](https://huggingface.co/datasets/opendatalab/OmniDocBench)
- Quantization tooling: [mlx-vlm](https://github.com/Blaizzy/mlx-vlm)