--- license: mit base_model: baidu/Unlimited-OCR library_name: mlx-vlm pipeline_tag: image-text-to-text tags: - mlx - mlx-vlm - ocr - mxfp8 - int8 - apple-silicon widget: - text: "document parsing." - text: "Convert the document to markdown." - text: "Free OCR. Extract all text." --- ## Release tags | Tag | Commit | Notes | |-----|--------|-------| | **v0.2.0** (current) | `6708e4d86b0a342c5891e47516aa412bbfd2eda2` | Sensitivity-guided MXFP8 + calibrated affine INT8 LM head | | **v0.1.0** | `fd0c04fc20db886f8a80fe1177081b07206586c7` | First OCR-aware MXFP8 publish | Pin production loads with `revision="v0.2.0"` (or the commit SHA). Floating `main` may move on future publishes. ```bash python -m mlx_vlm.generate \ --model AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \ --revision v0.2.0 \ --image document.png \ --prompt 'document parsing.' \ --max-tokens 4096 ``` # AX Unlimited-OCR 3B MoE — MLX (MXFP8 + affine INT8 head) This is an AutomatosX sensitivity-guided MLX conversion of [`baidu/Unlimited-OCR`](https://huggingface.co/baidu/Unlimited-OCR). It was created from the upstream BF16 weights. The [`sahilchachra/unlimited-ocr-mxfp8-mlx`](https://huggingface.co/sahilchachra/unlimited-ocr-mxfp8-mlx) checkpoint was used only as a comparison reference; these are not repackaged Sahil weights. **Hybrid precision (name is shorthand):** MXFP8 (group size 32) on eligible language linears, **affine INT8** on the LM head, **BF16** on the vision encoder and other protected groups. The Hub “1B params” style size reflects active MoE capacity; the marketing name refers to the upstream 3B MoE family. Immutable upstream source revision: `ee63731b6461c8afcdcc7b15352e7d2ffecc2ead`. Reference checkpoint revision: `55b8031a6c867de675279d9604e38cc94b9882a4`. ## What was optimized Each supported module group was quantized independently and evaluated for OCR CER, digit CER, and table structure. Eligible language linear layers use MXFP8 (8 data bits, group size 32). A joint calibration then selected affine INT8 (group size 32) for the LM head because BF16 missed the throughput floor and MXFP8 missed the CER ceiling. OCR-critical or sensitive groups remain BF16. Retained groups: vision_encoder, vision_projector. The model uses mlx-vlm's native `model_type=unlimited-ocr` implementation, `processor_class=UnlimitedOCRHFProcessor`, and a sliding-window size of 128. Prompts should be raw Unlimited-OCR strings such as `document parsing.` (not multi-turn chat formatting). ## AutomatosX validation These are **local release measurements**, not upstream paper benchmark claims and **not** OmniDocBench / olmOCR-Bench scores. Absolute CER on this set is not a production accuracy bar; gates check **parity with BF16** and packaging integrity. The content-addressed held-out evaluation set had 12 samples (12 with digits, 3 with CJK, and 3 with tables), digest `7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987`. | Checkpoint | Mean CER | Digit CER | CJK CER | Table score | Decode tok/s | |---|---:|---:|---:|---:|---:| | Upstream BF16 | 0.205212 | 0.113310 | 0.333333 | 1.000000 | — | | Community MXFP8 reference | 0.907381 | 0.406511 | 1.000000 | 1.000000 | 35.07 | | AutomatosX candidate | 0.216331 | 0.111939 | 0.341270 | 1.000000 | 35.14 | **Caveats** - Small local set; some samples are synthetic. Do not treat mean CER as field accuracy. - The community reference may use a different `model_type` / load path; it is a throughput and packaging baseline, not an identical-recipe peer. - Generation profiles used in AutomatosX tooling live in the `unlimited-ocr` package (`profiles.py`); weight-only loads use mlx-vlm defaults unless you pass the same decoding settings. ### Weight integrity Weight size: 3.582 GiB Aggregate weight SHA-256: `5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549` Per-file weight SHA-256: - `model.safetensors`: `d9fe11fc8d2e333000be6a0e288681231efa69282c844f7775ca40f52db02f0c` ## Release gates - ✅ `weights_are_distinct` — actual `5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549`, limit `different from 439c6acfe5e277537dfe3368b94a145a6b0da4c39e4bd00582977d030f292ad1` - ✅ `weight_size_gb` — actual `3.5815872186794877`, limit `4.5` - ✅ `native_model_metadata` — actual `{'architecture': True, 'model_type': True, 'mxfp8': True, 'sliding_window': True, 'processor_class': True, 'sft_format': True, 'precision_map': True, 'quantization_summary': True, 'protected_modules_preserved': True}`, limit `True` - ✅ `candidate_source_provenance` — actual `{'model': 'Unlimited-OCR', 'revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead'}`, limit `{'model': ['baidu/Unlimited-OCR', 'Unlimited-OCR'], 'revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead'}` - ✅ `evaluation_coverage` — actual `{'samples': 12, 'digit': 12, 'cjk': 3, 'table': 3}`, limit `all counts > 0` - ✅ `held_out_evaluation_dataset` — actual `{'passed': True, 'calibration_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'evaluation_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'overlapping_image_sha256': [], 'overlapping_ground_truth_sha256': []}`, limit `distinct dataset digests and no shared image or ground-truth hashes` - ✅ `accuracy_aggregates_recomputed` — actual `[True, True, True]`, limit `[True, True, True]` - ✅ `same_evaluation_samples` — actual `{'counts': [12, 12, 12], 'file_counts': [12, 12, 12]}`, limit `{'count': 12, 'files': ['0001.png', '0002.png', '0003.png', '0004.png', '0005.png', '0006.png', '0007.png', '0008.png', '0009.png', '0010.png', '0011.png', '0012.png']}` - ✅ `model_identities` — actual `{'bf16': 'Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'reference_performance': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate_performance': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'rswa': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8'}`, limit `{'bf16': 'Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8'}` - ✅ `immutable_model_revisions` — actual `{'bf16_accuracy': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_accuracy': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_accuracy': None, 'reference_performance': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_performance': None}`, limit `{'bf16_accuracy': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_accuracy': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_accuracy': None, 'reference_performance': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_performance': None}` - ✅ `same_accuracy_recipe` — actual `{'prompt': ['document parsing.', 'document parsing.', 'document parsing.'], 'max_tokens': [1024, 1024, 1024], 'profile': ['accurate', 'accurate', 'accurate'], 'generation_settings': [{'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}, {'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}, {'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}]}`, limit `identical official MLX OCR recipe` - ✅ `candidate_cer_vs_bf16` — actual `0.011118653228923858`, limit `0.015` - ✅ `candidate_cer_vs_reference` — actual `-0.691050362380767`, limit `0.005` - ✅ `candidate_digit_cer_vs_bf16` — actual `-0.0013706317359285375`, limit `0.01` - ✅ `candidate_table_score_vs_bf16` — actual `0.0`, limit `0.01` - ✅ `performance_aggregates_recomputed` — actual `{'reference': True, 'candidate': True}`, limit `{'reference': True, 'candidate': True}` - ✅ `same_performance_setup` — actual `{'image_path': ['test_invoice.png', 'test_invoice.png'], 'prompt': ['document parsing.', 'document parsing.'], 'max_tokens': [256, 256], 'num_warmup': [1, 1], 'num_runs': [3, 3], 'system': [{'platform': 'macOS-26.5.2-arm64-arm-64bit-Mach-O', 'processor': 'arm', 'python_version': '3.14.6', 'machine': 'arm64', 'mlx_version': '0.32.0', 'mlx_vlm_version': '0.6.6', 'chip': 'Apple M3 Max', 'total_memory_gb': 128.0}, {'platform': 'macOS-26.5.2-arm64-arm-64bit-Mach-O', 'processor': 'arm', 'python_version': '3.14.6', 'machine': 'arm64', 'mlx_version': '0.32.0', 'mlx_vlm_version': '0.6.6', 'chip': 'Apple M3 Max', 'total_memory_gb': 128.0}]}`, limit `identical setup with at least three complete runs` - ✅ `candidate_tps_vs_reference` — actual `1.001955642681865`, limit `0.9` - ✅ `rswa_8k_bounded` — actual `{'pass_conditions': {'cache_bounded': True, 'tps_stable': True, '8k_test_passed': True}, 'tokens': 8192, 'repetition_rate': 0.20496894409937888}`, limit `{'min_tokens': 8192, 'max_repetition_rate': 0.25}` - ✅ `provenance_matches_release` — actual `{'source': 'baidu/Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'target': 'AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'source_config': {'path': 'config.json', 'sha256': '27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9'}, 'source_revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_revision': '55b8031a6c867de675279d9604e38cc94b9882a4', 'dataset_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'calibration_dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb'}`, limit `{'source': 'baidu/Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'target': 'AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'source_config_sha256': '27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9', 'source_revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_revision': '55b8031a6c867de675279d9604e38cc94b9882a4', 'dataset_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'calibration_dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb'}` - ✅ `calibration_recomputed` — actual `{'selected': {'label': 'affine8-head', 'precision': 'affine8'}, 'models': ['AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-affine8', 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-bfloat16', 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-mxfp8'], 'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'input_artifacts': {'calibration_baseline_accuracy': {'filename': 'calibration_baseline_accuracy.json', 'size': 5680, 'sha256': 'ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6'}, 'calibration_reference_performance': {'filename': 'calibration_reference_performance.json', 'size': 2271, 'sha256': '98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4'}, 'calibration_bfloat16_accuracy': {'filename': 'calibration_bfloat16_accuracy.json', 'size': 5678, 'sha256': '5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0'}, 'calibration_bfloat16_performance': {'filename': 'calibration_bfloat16_performance.json', 'size': 2244, 'sha256': 'dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0'}, 'calibration_mxfp8_accuracy': {'filename': 'calibration_mxfp8_accuracy.json', 'size': 5689, 'sha256': '77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26'}, 'calibration_mxfp8_performance': {'filename': 'calibration_mxfp8_performance.json', 'size': 2252, 'sha256': 'e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b'}, 'calibration_affine8_accuracy': {'filename': 'calibration_affine8_accuracy.json', 'size': 5657, 'sha256': 'c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19'}, 'calibration_affine8_performance': {'filename': 'calibration_affine8_performance.json', 'size': 2247, 'sha256': 'f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f'}}}`, limit `content-addressed inputs and fastest passing experiment` - ✅ `sensitivity_matches_calibration_dataset` — actual `{'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'baseline_metrics_match': True}`, limit `{'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'baseline_metrics_match': True}` - ✅ `candidate_precision_map_matches_evidence` — actual `91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75`, limit `91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75` - ✅ `precision_map_reproducible` — actual `c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a`, limit `c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a` Full per-sample evidence, sensitivity results, precision rules, PRD, ADR, and technical specification are included under `release/`, `quantization/`, and `docs/`. ## Usage Requires **Apple Silicon** and **mlx-vlm ≥ 0.6.4** (native Unlimited-OCR / R-SWA). Tested stack for this release: mlx-vlm `0.6.6`, mlx `0.32.0`. ```bash pip install 'mlx-vlm>=0.6.4' python -m mlx_vlm.generate \ --model AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \ --image document.png \ --prompt 'document parsing.' \ --max-tokens 4096 ``` Or with the AutomatosX toolkit (deskew, PDF, profiles): ```bash pip install 'unlimited-ocr[mlx]' # then use OCRPipeline / the unlimited-ocr CLI with model_path=AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 ``` This is quantization/conversion work, not foundation-model retraining or domain fine-tuning.