Publish verified OCR-aware MXFP8 checkpoint

README.md

@@ -10,9 +10,13 @@ tags:
 - mxfp8
 - int8
 - apple-silicon
+widget:
+- text: "<image>document parsing."
+- text: "<image>Convert the document to markdown."
+- text: "<image>Free OCR. Extract all text."
 ---
 
-# AX Unlimited-OCR 3B MoE — MLX MXFP8
+# AX Unlimited-OCR 3B MoE — MLX (MXFP8 + affine INT8 head)
 
 This is an AutomatosX sensitivity-guided MLX conversion of
 [`baidu/Unlimited-OCR`](https://huggingface.co/baidu/Unlimited-OCR). It was
@@ -21,6 +25,14 @@ created from the upstream BF16 weights. The
 checkpoint was used only as a comparison reference; these are not repackaged
 Sahil weights.
 
+**Hybrid precision (name is shorthand):** MXFP8 (group size 32) on eligible
+language linears, **affine INT8** on the LM head, **BF16** on the vision
+encoder and other protected groups. The Hub “1B params” style size reflects
+active MoE capacity; the marketing name refers to the upstream 3B MoE family.
+
+Immutable upstream source revision: `ee63731b6461c8afcdcc7b15352e7d2ffecc2ead`.  
+Reference checkpoint revision: `55b8031a6c867de675279d9604e38cc94b9882a4`.
+
 ## What was optimized
 
 Each supported module group was quantized independently and evaluated for OCR
@@ -28,41 +40,74 @@ CER, digit CER, and table structure. Eligible language linear layers use MXFP8
 (8 data bits, group size 32). A joint calibration then selected affine INT8
 (group size 32) for the LM head because BF16 missed the throughput floor and
 MXFP8 missed the CER ceiling. OCR-critical or sensitive groups remain BF16.
-Retained groups: vision_encoder.
+Retained groups: vision_encoder, vision_projector.
 
-The model uses mlx-vlm's native `model_type=unlimited-ocr` implementation and a
-sliding-window size of 128.
+The model uses mlx-vlm's native `model_type=unlimited-ocr` implementation,
+`processor_class=UnlimitedOCRHFProcessor`, and a sliding-window size of
+128. Prompts should be raw Unlimited-OCR strings such as
+`<image>document parsing.` (not multi-turn chat formatting).
 
 ## AutomatosX validation
 
-These are local release measurements, not upstream paper benchmark claims.
-The content-addressed evaluation set had 12 samples
+These are **local release measurements**, not upstream paper benchmark claims
+and **not** OmniDocBench / olmOCR-Bench scores. Absolute CER on this set is
+not a production accuracy bar; gates check **parity with BF16** and packaging
+integrity.
+
+The content-addressed held-out evaluation set had 12 samples
 (12 with digits, 3 with
 CJK, and 3 with tables), digest
-`78ca3e7c9c7584aea30b04176716a6dadf49ccf0f2bead959cbaeb9aeac68f30`.
+`7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987`.
+
+| Checkpoint | Mean CER | Digit CER | CJK CER | Table score | Decode tok/s |
+|---|---:|---:|---:|---:|---:|
+| Upstream BF16 | 0.205212 | 0.113310 | 0.333333 | 1.000000 | — |
+| Community MXFP8 reference | 0.907381 | 0.406511 | 1.000000 | 1.000000 | 35.07 |
+| AutomatosX candidate | 0.216331 | 0.111939 | 0.341270 | 1.000000 | 35.14 |
+
+**Caveats**
+- Small local set; some samples are synthetic. Do not treat mean CER as field accuracy.
+- The community reference may use a different `model_type` / load path; it is a
+  throughput and packaging baseline, not an identical-recipe peer.
+- Generation profiles used in AutomatosX tooling live in the `unlimited-ocr`
+  package (`profiles.py`); weight-only loads use mlx-vlm defaults unless you
+  pass the same decoding settings.
+
+### Weight integrity
 
-| Checkpoint | Mean CER | Digit CER | Table score | Decode tok/s |
-|---|---:|---:|---:|---:|
-| Upstream BF16 | 0.239188 | 0.087340 | 1.000000 | — |
-| Sahil MXFP8 reference | 1.057249 | 0.209153 | 1.000000 | 312.73 |
-| AutomatosX candidate | 0.237384 | 0.085733 | 1.000000 | 297.20 |
+Weight size: 3.582 GiB  
+Aggregate weight SHA-256: `5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549`
 
-Weight size: 3.579 GiB  
-Aggregate weight SHA-256: `a4f6e9934f90564906f42929542ad68493f0c6187ad431128e447debbf52b6f4`
+Per-file weight SHA-256:
+
+- `model.safetensors`: `d9fe11fc8d2e333000be6a0e288681231efa69282c844f7775ca40f52db02f0c`
 
 ## Release gates
 
-- ✅ `weights_are_distinct` — actual `a4f6e9934f90564906f42929542ad68493f0c6187ad431128e447debbf52b6f4`, limit `different from 439c6acfe5e277537dfe3368b94a145a6b0da4c39e4bd00582977d030f292ad1`
-- ✅ `weight_size_gb` — actual `3.5792221864685416`, limit `4.5`
-- ✅ `native_model_metadata` — actual `{'architecture': True, 'model_type': True, 'mxfp8': True, 'sliding_window': True, 'processor_class': True, 'sft_format': True}`, limit `True`
+- ✅ `weights_are_distinct` — actual `5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549`, limit `different from 439c6acfe5e277537dfe3368b94a145a6b0da4c39e4bd00582977d030f292ad1`
+- ✅ `weight_size_gb` — actual `3.5815872186794877`, limit `4.5`
+- ✅ `native_model_metadata` — actual `{'architecture': True, 'model_type': True, 'mxfp8': True, 'sliding_window': True, 'processor_class': True, 'sft_format': True, 'precision_map': True, 'quantization_summary': True, 'protected_modules_preserved': True}`, limit `True`
+- ✅ `candidate_source_provenance` — actual `{'model': 'Unlimited-OCR', 'revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead'}`, limit `{'model': ['baidu/Unlimited-OCR', 'Unlimited-OCR'], 'revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead'}`
 - ✅ `evaluation_coverage` — actual `{'samples': 12, 'digit': 12, 'cjk': 3, 'table': 3}`, limit `all counts > 0`
-- ✅ `same_evaluation_samples` — actual `[12, 12, 12]`, limit `12`
-- ✅ `candidate_cer_vs_bf16` — actual `-0.0018033315701714847`, limit `0.01`
-- ✅ `candidate_cer_vs_reference` — actual `-0.8198643918447561`, limit `0.005`
-- ✅ `candidate_digit_cer_vs_bf16` — actual `-0.0016074315849917697`, limit `0.01`
+- ✅ `held_out_evaluation_dataset` — actual `{'passed': True, 'calibration_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'evaluation_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'overlapping_image_sha256': [], 'overlapping_ground_truth_sha256': []}`, limit `distinct dataset digests and no shared image or ground-truth hashes`
+- ✅ `accuracy_aggregates_recomputed` — actual `[True, True, True]`, limit `[True, True, True]`
+- ✅ `same_evaluation_samples` — actual `{'counts': [12, 12, 12], 'file_counts': [12, 12, 12]}`, limit `{'count': 12, 'files': ['0001.png', '0002.png', '0003.png', '0004.png', '0005.png', '0006.png', '0007.png', '0008.png', '0009.png', '0010.png', '0011.png', '0012.png']}`
+- ✅ `model_identities` — actual `{'bf16': 'Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'reference_performance': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate_performance': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'rswa': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8'}`, limit `{'bf16': 'Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'candidate': 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8'}`
+- ✅ `immutable_model_revisions` — actual `{'bf16_accuracy': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_accuracy': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_accuracy': None, 'reference_performance': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_performance': None}`, limit `{'bf16_accuracy': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_accuracy': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_accuracy': None, 'reference_performance': '55b8031a6c867de675279d9604e38cc94b9882a4', 'candidate_performance': None}`
+- ✅ `same_accuracy_recipe` — actual `{'prompt': ['<image>document parsing.', '<image>document parsing.', '<image>document parsing.'], 'max_tokens': [1024, 1024, 1024], 'profile': ['accurate', 'accurate', 'accurate'], 'generation_settings': [{'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}, {'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}, {'temperature': 0.0, 'top_p': 1.0, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 35, 'ngram_window': 128}]}`, limit `identical official MLX OCR recipe`
+- ✅ `candidate_cer_vs_bf16` — actual `0.011118653228923858`, limit `0.015`
+- ✅ `candidate_cer_vs_reference` — actual `-0.691050362380767`, limit `0.005`
+- ✅ `candidate_digit_cer_vs_bf16` — actual `-0.0013706317359285375`, limit `0.01`
 - ✅ `candidate_table_score_vs_bf16` — actual `0.0`, limit `0.01`
-- ✅ `candidate_tps_vs_reference` — actual `0.9503587809120344`, limit `0.9`
-- ✅ `rswa_8k_bounded` — actual `{'cache_bounded': True, 'tps_stable': True, '8k_test_passed': True}`, limit `True`
+- ✅ `performance_aggregates_recomputed` — actual `{'reference': True, 'candidate': True}`, limit `{'reference': True, 'candidate': True}`
+- ✅ `same_performance_setup` — actual `{'image_path': ['test_invoice.png', 'test_invoice.png'], 'prompt': ['<image>document parsing.', '<image>document parsing.'], 'max_tokens': [256, 256], 'num_warmup': [1, 1], 'num_runs': [3, 3], 'system': [{'platform': 'macOS-26.5.2-arm64-arm-64bit-Mach-O', 'processor': 'arm', 'python_version': '3.14.6', 'machine': 'arm64', 'mlx_version': '0.32.0', 'mlx_vlm_version': '0.6.6', 'chip': 'Apple M3 Max', 'total_memory_gb': 128.0}, {'platform': 'macOS-26.5.2-arm64-arm-64bit-Mach-O', 'processor': 'arm', 'python_version': '3.14.6', 'machine': 'arm64', 'mlx_version': '0.32.0', 'mlx_vlm_version': '0.6.6', 'chip': 'Apple M3 Max', 'total_memory_gb': 128.0}]}`, limit `identical setup with at least three complete runs`
+- ✅ `candidate_tps_vs_reference` — actual `1.001955642681865`, limit `0.9`
+- ✅ `rswa_8k_bounded` — actual `{'pass_conditions': {'cache_bounded': True, 'tps_stable': True, '8k_test_passed': True}, 'tokens': 8192, 'repetition_rate': 0.20496894409937888}`, limit `{'min_tokens': 8192, 'max_repetition_rate': 0.25}`
+- ✅ `provenance_matches_release` — actual `{'source': 'baidu/Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'target': 'AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'source_config': {'path': 'config.json', 'sha256': '27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9'}, 'source_revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_revision': '55b8031a6c867de675279d9604e38cc94b9882a4', 'dataset_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'calibration_dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb'}`, limit `{'source': 'baidu/Unlimited-OCR', 'reference': 'sahilchachra/unlimited-ocr-mxfp8-mlx', 'target': 'AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8', 'source_config_sha256': '27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9', 'source_revision': 'ee63731b6461c8afcdcc7b15352e7d2ffecc2ead', 'reference_revision': '55b8031a6c867de675279d9604e38cc94b9882a4', 'dataset_digest': '7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987', 'calibration_dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb'}`
+- ✅ `calibration_recomputed` — actual `{'selected': {'label': 'affine8-head', 'precision': 'affine8'}, 'models': ['AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-affine8', 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-bfloat16', 'AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-mxfp8'], 'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'input_artifacts': {'calibration_baseline_accuracy': {'filename': 'calibration_baseline_accuracy.json', 'size': 5680, 'sha256': 'ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6'}, 'calibration_reference_performance': {'filename': 'calibration_reference_performance.json', 'size': 2271, 'sha256': '98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4'}, 'calibration_bfloat16_accuracy': {'filename': 'calibration_bfloat16_accuracy.json', 'size': 5678, 'sha256': '5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0'}, 'calibration_bfloat16_performance': {'filename': 'calibration_bfloat16_performance.json', 'size': 2244, 'sha256': 'dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0'}, 'calibration_mxfp8_accuracy': {'filename': 'calibration_mxfp8_accuracy.json', 'size': 5689, 'sha256': '77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26'}, 'calibration_mxfp8_performance': {'filename': 'calibration_mxfp8_performance.json', 'size': 2252, 'sha256': 'e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b'}, 'calibration_affine8_accuracy': {'filename': 'calibration_affine8_accuracy.json', 'size': 5657, 'sha256': 'c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19'}, 'calibration_affine8_performance': {'filename': 'calibration_affine8_performance.json', 'size': 2247, 'sha256': 'f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f'}}}`, limit `content-addressed inputs and fastest passing experiment`
+- ✅ `sensitivity_matches_calibration_dataset` — actual `{'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'baseline_metrics_match': True}`, limit `{'dataset_digest': '40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb', 'baseline_metrics_match': True}`
+- ✅ `candidate_precision_map_matches_evidence` — actual `91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75`, limit `91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75`
+- ✅ `precision_map_reproducible` — actual `c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a`, limit `c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a`
 
 Full per-sample evidence, sensitivity results, precision rules, PRD, ADR, and
 technical specification are included under `release/`, `quantization/`, and
@@ -70,8 +115,11 @@ technical specification are included under `release/`, `quantization/`, and
 
 ## Usage
 
+Requires **Apple Silicon** and **mlx-vlm ≥ 0.6.4** (native Unlimited-OCR / R-SWA).
+Tested stack for this release: mlx-vlm `0.6.6`, mlx `0.32.0`.
+
 ```bash
-pip install mlx-vlm
+pip install 'mlx-vlm>=0.6.4'
 python -m mlx_vlm.generate \
   --model AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \
   --image document.png \
@@ -79,6 +127,12 @@ python -m mlx_vlm.generate \
   --max-tokens 4096
 ```
 
-Apple Silicon and a recent `mlx-vlm` release with native Unlimited-OCR support
-are required. This is quantization/conversion work, not foundation-model
-retraining or domain fine-tuning.
+Or with the AutomatosX toolkit (deskew, PDF, profiles):
+
+```bash
+pip install 'unlimited-ocr[mlx]'
+# then use OCRPipeline / the unlimited-ocr CLI with model_path=AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8
+```
+
+This is quantization/conversion work, not foundation-model retraining or domain
+fine-tuning.

benchmarks/datasets.md

@@ -35,9 +35,44 @@ The evaluation set must be separate from any calibration data used for quantizat
 benchmarks/eval_data/
 ├── images/           # Input document images (PNG/JPG)
 ├── ground_truth/     # Reference text files (same stem as image)
-└── metadata.json     # Dataset composition and source info
+└── manifest.json     # Exact sample paths, categories, and source metadata
 ```
 
+### Calibration vs held-out evaluation (required for schema-3 release)
+
+Sensitivity / LM-head calibration and the final release accuracy run must use
+**disjoint** directories. Recommended layout under `eval_dataset/`:
+
+```
+eval_dataset/
+├── calibration/
+│   ├── images/
+│   ├── ground_truth/
+│   └── manifest.json
+└── evaluation/          # held-out; used for release_gate accuracy
+    ├── images/
+    ├── ground_truth/
+    └── manifest.json
+```
+
+Build with non-overlapping FUNSD IDs and different synthetic seeds:
+
+```bash
+python eval_dataset/build_release_set.py --clean \
+  --output-root eval_dataset/calibration \
+  --funsd-ids 0,1 --generator-seed 20260720 \
+  --funsd-revision <SHA> --latin-font <PATH> --cjk-font <PATH>
+
+python eval_dataset/build_release_set.py --clean \
+  --output-root eval_dataset/evaluation \
+  --funsd-ids 2,3,4,5 --generator-seed 20260721 \
+  --funsd-revision <SHA> --latin-font <PATH> --cjk-font <PATH>
+```
+
+`benchmarks/run_accuracy.py` reports aggregate metrics and, when
+`manifest.json` lists categories, a `by_category` breakdown. Use
+`--save-predictions DIR` to dump hypotheses for failure analysis.
+
 ### Ground Truth Format
 
 Each ground truth file is a plain `.txt` file containing the expected OCR output.

benchmarks/evaluate_tables.py

@@ -11,7 +11,6 @@ from __future__ import annotations
 
 import re
 from html.parser import HTMLParser
-from pathlib import Path
 
 
 def _split_markdown_row(line: str) -> list[str]:
@@ -178,7 +177,7 @@ def normalize_table(table_text: str) -> dict:
     if re.search(r"<table\b", table_text, re.IGNORECASE):
         return _normalize_html_table(table_text)
 
-    lines = [l.strip() for l in table_text.strip().split("\n") if l.strip()]
+    lines = [line.strip() for line in table_text.strip().split("\n") if line.strip()]
 
     # Filter out the Markdown header separator.
     data_lines = [line for line in lines if not _is_separator_row(line)]

benchmarks/rswa_validation.py

@@ -14,13 +14,17 @@ from __future__ import annotations
 
 import argparse
 import json
+import math
 import platform
 import sys
 import time
+from numbers import Integral, Real
 from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
 
+from unlimited_ocr.engine import SlidingWindowNoRepeatNGramProcessor  # noqa: E402
+
 
 # Output lengths to test
 TEST_LENGTHS = [512, 2048, 4096, 8192, 16384]
@@ -45,7 +49,9 @@ class MinGeneratedTokensProcessor:
         normalized_ids = sorted({
             int(token_id)
             for token_id in eos_token_ids
-            if isinstance(token_id, int) and not isinstance(token_id, bool)
+            if isinstance(token_id, Integral)
+            and not isinstance(token_id, bool)
+            and token_id >= 0
         })
         if not normalized_ids:
             raise ValueError("At least one integer EOS token ID is required")
@@ -81,8 +87,12 @@ def collect_eos_token_ids(model, processor) -> list[int]:
     for value in values:
         candidates = value if isinstance(value, (list, tuple, set)) else [value]
         for candidate in candidates:
-            if isinstance(candidate, int) and not isinstance(candidate, bool):
-                normalized.add(candidate)
+            if (
+                isinstance(candidate, Integral)
+                and not isinstance(candidate, bool)
+                and candidate >= 0
+            ):
+                normalized.add(int(candidate))
     if not normalized:
         raise ValueError("Could not determine EOS token IDs from model or processor")
     return sorted(normalized)
@@ -90,10 +100,12 @@ def collect_eos_token_ids(model, processor) -> list[int]:
 
 def detect_repetition_rate(text: str, ngram_size: int = 35, threshold: int = 3) -> float:
     """Detect fraction of repeated n-grams in output."""
+    if not isinstance(text, str):
+        raise TypeError("text must be a string")
     if not isinstance(ngram_size, int) or isinstance(ngram_size, bool) or ngram_size < 1:
         raise ValueError("ngram_size must be a positive integer")
-    if not isinstance(threshold, int) or isinstance(threshold, bool) or threshold < 0:
-        raise ValueError("threshold must be a non-negative integer")
+    if not isinstance(threshold, int) or isinstance(threshold, bool) or threshold < 1:
+        raise ValueError("threshold must be a positive integer")
     words = text.split()
     if len(words) < ngram_size:
         return 0.0
@@ -111,20 +123,31 @@ def analyze_cache_behavior(results: list[dict]) -> tuple[bool | None, str]:
     Returns ``None`` when the generated token counts do not span enough lengths
     to distinguish a plateau from ordinary noise or linear growth.
     """
-    successful = [
-        result
-        for result in results
-        if result.get("status") == "success"
-        and result.get("tokens_generated", 0) > 0
-        and result.get("tokens_generated_reliable", True)
-        and result.get("memory_growth_mb") is not None
-    ]
+    successful = []
+    for result in results:
+        if not isinstance(result, dict) or result.get("status") != "success":
+            continue
+        tokens = result.get("tokens_generated")
+        memory = result.get("memory_growth_mb")
+        if (
+            not isinstance(tokens, Integral)
+            or isinstance(tokens, bool)
+            or tokens <= 0
+            or result.get("tokens_generated_reliable", True) is not True
+            or not isinstance(memory, Real)
+            or isinstance(memory, bool)
+            or not math.isfinite(float(memory))
+            or memory < 0
+        ):
+            continue
+        successful.append(result)
     # Collapse duplicate actual token counts; raising max_tokens is not useful
     # evidence when the model naturally stops at the same length each time.
     by_tokens: dict[int, float] = {}
     for result in successful:
         tokens = int(result["tokens_generated"])
-        by_tokens[tokens] = max(by_tokens.get(tokens, 0.0), result["memory_growth_mb"])
+        memory = float(result["memory_growth_mb"])
+        by_tokens[tokens] = max(by_tokens.get(tokens, 0.0), memory)
     points = sorted(by_tokens.items())
 
     if len(points) < 3:
@@ -223,6 +246,8 @@ def run_rswa_validation(
     )
 
     baseline_memory = mx.get_active_memory() / 1e6
+    if not math.isfinite(baseline_memory) or baseline_memory < 0:
+        raise ValueError("MLX returned invalid active memory")
     print(f"Baseline memory: {baseline_memory:.0f} MB")
 
     results = []
@@ -232,6 +257,8 @@ def run_rswa_validation(
 
         mx.clear_cache()
         mem_before = mx.get_active_memory() / 1e6
+        if not math.isfinite(mem_before) or mem_before < 0:
+            raise ValueError("MLX returned invalid active memory")
         mx.reset_peak_memory()
         start = time.perf_counter()
 
@@ -241,11 +268,13 @@ def run_rswa_validation(
                 if force_min_tokens is not None
                 else None
             )
-            logits_processors = (
-                [MinGeneratedTokensProcessor(forced_minimum, eos_token_ids)]
-                if forced_minimum is not None
-                else None
-            )
+            logits_processors = [
+                SlidingWindowNoRepeatNGramProcessor(35, 128)
+            ]
+            if forced_minimum is not None:
+                logits_processors.append(
+                    MinGeneratedTokensProcessor(forced_minimum, eos_token_ids)
+                )
             response = generate(
                 model, processor,
                 prompt=prompt,
@@ -261,11 +290,20 @@ def run_rswa_validation(
                 if isinstance(response, dict)
                 else getattr(response, "peak_memory", None)
             )
+            if peak_memory is not None and (
+                not isinstance(peak_memory, Real)
+                or isinstance(peak_memory, bool)
+                or not math.isfinite(float(peak_memory))
+                or peak_memory < 0
+            ):
+                raise ValueError("mlx-vlm returned invalid peak_memory")
             mem_after = (
                 float(peak_memory) * 1000
-                if peak_memory is not None and float(peak_memory) > 0
+                if peak_memory is not None and peak_memory > 0
                 else mx.get_peak_memory() / 1e6
             )
+            if not math.isfinite(mem_after) or mem_after < 0:
+                raise ValueError("MLX returned invalid peak memory")
 
             # Extract text
             if isinstance(response, str):
@@ -312,14 +350,31 @@ def run_rswa_validation(
                 tokens_reliable = False
 
             # Compute metrics
-            tokens_generated = int(tokens_generated or 0)
+            if (
+                not isinstance(tokens_generated, Integral)
+                or isinstance(tokens_generated, bool)
+                or not 0 <= tokens_generated <= max_tokens
+            ):
+                raise ValueError("mlx-vlm returned an invalid generation token count")
+            tokens_generated = int(tokens_generated)
             reported_tps = (
                 response.get("generation_tps", 0.0)
                 if isinstance(response, dict)
                 else getattr(response, "generation_tps", 0.0)
             )
-            tps = float(reported_tps) if reported_tps else (
-                tokens_generated / elapsed if elapsed > 0 else 0
+            if isinstance(reported_tps, bool) or (
+                reported_tps is not None
+                and (
+                    not isinstance(reported_tps, Real)
+                    or not math.isfinite(float(reported_tps))
+                    or reported_tps < 0
+                )
+            ):
+                raise ValueError("mlx-vlm returned invalid generation_tps")
+            tps = (
+                float(reported_tps)
+                if reported_tps
+                else tokens_generated / elapsed if elapsed > 0 else 0.0
             )
             repetition = detect_repetition_rate(text)
             memory_growth = max(0.0, mem_after - mem_before)
@@ -406,6 +461,11 @@ def run_rswa_validation(
         "image_path": Path(image_path).name,
         "prompt": prompt,
         "force_min_tokens": force_min_tokens,
+        "generation_settings": {
+            "temperature": 0.0,
+            "no_repeat_ngram_size": 35,
+            "ngram_window": 128,
+        },
         "forced_eos_token_ids": eos_token_ids,
         "platform": platform.platform(),
         "processor": platform.processor(),

benchmarks/run_accuracy.py

@@ -12,6 +12,7 @@ from __future__ import annotations
 import argparse
 from collections import Counter
 import json
+import os
 import sys
 import time
 from pathlib import Path
@@ -22,14 +23,75 @@ PROJECT_ROOT = Path(__file__).parent.parent
 sys.path.insert(0, str(PROJECT_ROOT))
 sys.path.insert(0, str(PROJECT_ROOT / "src"))
 
-from benchmarks.evaluate_cer import (
+from benchmarks.evaluate_cer import (  # noqa: E402
     compute_cer, compute_wer, compute_digit_cer,
     compute_cjk_cer, detect_repetition, extract_cjk_characters,
 )
-from benchmarks.normalize_output import normalize_ocr_output
-from benchmarks.evaluate_tables import evaluate_tables
-from unlimited_ocr.engine import SlidingWindowNoRepeatNGramProcessor
-from unlimited_ocr.profiles import get_profile
+from benchmarks.normalize_output import normalize_ocr_output  # noqa: E402
+from benchmarks.evaluate_tables import evaluate_tables  # noqa: E402
+from unlimited_ocr.engine import SlidingWindowNoRepeatNGramProcessor  # noqa: E402
+from unlimited_ocr.profiles import get_profile  # noqa: E402
+
+
+def load_category_map(eval_dir: Path) -> dict[str, str]:
+    """Map image filenames to categories from ``manifest.json`` when present."""
+    manifest_path = Path(eval_dir) / "manifest.json"
+    if not manifest_path.is_file():
+        return {}
+    try:
+        manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+    except (OSError, UnicodeError, json.JSONDecodeError):
+        return {}
+    samples = manifest.get("samples")
+    if not isinstance(samples, list):
+        return {}
+    mapping: dict[str, str] = {}
+    for sample in samples:
+        if not isinstance(sample, dict):
+            continue
+        category = sample.get("category")
+        image = sample.get("image")
+        if not isinstance(category, str) or not category or not isinstance(image, str):
+            continue
+        mapping[Path(image).name] = category
+    return mapping
+
+
+def aggregate_by_category(per_file: list[dict]) -> dict[str, dict]:
+    """Aggregate CER-family metrics by category label on each per-file row."""
+    buckets: dict[str, list[dict]] = {}
+    for row in per_file:
+        if not isinstance(row, dict):
+            continue
+        category = row.get("category")
+        if not isinstance(category, str) or not category:
+            category = "uncategorized"
+        buckets.setdefault(category, []).append(row)
+
+    summary: dict[str, dict] = {}
+    for category, rows in sorted(buckets.items()):
+        digit_scores = [r["digit_cer"] for r in rows if r.get("ref_digit_count", 0) > 0]
+        cjk_scores = [r["cjk_cer"] for r in rows if r.get("ref_cjk_count", 0) > 0]
+        table_scores = [r["table_score"] for r in rows if r.get("table_score") is not None]
+        summary[category] = {
+            "num_samples": len(rows),
+            "mean_cer": (
+                sum(r["cer"] for r in rows) / len(rows) if rows else None
+            ),
+            "mean_digit_cer": (
+                sum(digit_scores) / len(digit_scores) if digit_scores else None
+            ),
+            "mean_cjk_cer": (
+                sum(cjk_scores) / len(cjk_scores) if cjk_scores else None
+            ),
+            "mean_table_score": (
+                sum(table_scores) / len(table_scores) if table_scores else None
+            ),
+            "num_digit_samples": len(digit_scores),
+            "num_cjk_samples": len(cjk_scores),
+            "num_table_samples": len(table_scores),
+        }
+    return summary
 
 
 def run_accuracy_benchmark(
@@ -38,6 +100,11 @@ def run_accuracy_benchmark(
     prompt: str = "<image>document parsing.",
     max_tokens: int | None = None,
     profile: str = "accurate",
+    backend: str = "mlx",
+    vllm_base_url: str = "http://127.0.0.1:8000/v1",
+    vllm_api_key: str | None = None,
+    served_revision: str | None = None,
+    save_predictions: Path | None = None,
 ) -> dict:
     """Run accuracy benchmark on evaluation dataset.
 
@@ -60,6 +127,12 @@ def run_accuracy_benchmark(
     token_limit = generation_profile.max_tokens if max_tokens is None else max_tokens
     if not isinstance(token_limit, int) or isinstance(token_limit, bool) or token_limit < 1:
         raise ValueError("max_tokens must be a positive integer")
+    if backend not in {"mlx", "vllm"}:
+        raise ValueError("backend must be one of: mlx, vllm")
+    if served_revision is not None and (
+        not isinstance(served_revision, str) or not served_revision.strip()
+    ):
+        raise ValueError("served_revision must be a non-empty string or None")
 
     # Collect image files
     image_files = sorted(
@@ -97,12 +170,85 @@ def run_accuracy_benchmark(
         (image_file, gt_dir / f"{image_file.stem}.txt")
         for image_file in image_files
     ]
-
-    # Import and load only after validating the dataset and generation options.
-    from mlx_vlm import load, generate
+    category_map = load_category_map(eval_dir)
+    if save_predictions is not None:
+        save_predictions = Path(save_predictions)
+        save_predictions.mkdir(parents=True, exist_ok=True)
 
     print(f"Loading model: {model_path}")
-    model, processor = load(model_path)
+    if backend == "mlx":
+        # Import only after validating the dataset and generation options.
+        from mlx_vlm import generate as mlx_generate, load
+
+        model, processor = load(model_path, revision=served_revision)
+        return _evaluate_accuracy_files(
+            model_path=model_path,
+            evaluation_files=evaluation_files,
+            prompt=prompt,
+            token_limit=token_limit,
+            profile=profile,
+            backend=backend,
+            served_revision=served_revision,
+            generation_profile=generation_profile,
+            model=model,
+            processor=processor,
+            generate=mlx_generate,
+            engine=None,
+            category_map=category_map,
+            save_predictions=save_predictions,
+        )
+
+    from unlimited_ocr.vllm_backend import VLLMEngine, VLLMEngineConfig
+
+    engine = VLLMEngine(
+        VLLMEngineConfig(
+            model_path=model_path,
+            base_url=vllm_base_url,
+            api_key=vllm_api_key,
+            max_tokens=token_limit,
+            profile=profile,
+        )
+    )
+    try:
+        engine.ensure_ready()
+        return _evaluate_accuracy_files(
+            model_path=model_path,
+            evaluation_files=evaluation_files,
+            prompt=prompt,
+            token_limit=token_limit,
+            profile=profile,
+            backend=backend,
+            served_revision=served_revision,
+            generation_profile=generation_profile,
+            model=None,
+            processor=None,
+            generate=None,
+            engine=engine,
+            category_map=category_map,
+            save_predictions=save_predictions,
+        )
+    finally:
+        engine.unload()
+
+
+def _evaluate_accuracy_files(
+    *,
+    model_path: str,
+    evaluation_files: list[tuple[Path, Path]],
+    prompt: str,
+    token_limit: int,
+    profile: str,
+    backend: str,
+    served_revision: str | None,
+    generation_profile,
+    model,
+    processor,
+    generate,
+    engine,
+    category_map: dict[str, str] | None = None,
+    save_predictions: Path | None = None,
+) -> dict:
+    """Evaluate already-loaded MLX or vLLM inference resources."""
 
     results = []
     total_time = 0.0
@@ -112,20 +258,28 @@ def run_accuracy_benchmark(
 
         # Run inference
         start = time.perf_counter()
-        response = generate(
-            model, processor,
-            prompt=prompt,
-            image=[str(img_file)],
-            max_tokens=token_limit,
-            temperature=generation_profile.temperature,
-            top_p=generation_profile.top_p,
-            repetition_penalty=generation_profile.repetition_penalty,
-            logits_processors=[SlidingWindowNoRepeatNGramProcessor(
-                generation_profile.no_repeat_ngram_size,
-                128,
-            )],
-            verbose=False,
-        )
+        if backend == "mlx":
+            response = generate(
+                model, processor,
+                prompt=prompt,
+                image=[str(img_file)],
+                max_tokens=token_limit,
+                temperature=generation_profile.temperature,
+                top_p=generation_profile.top_p,
+                repetition_penalty=generation_profile.repetition_penalty,
+                logits_processors=[SlidingWindowNoRepeatNGramProcessor(
+                    generation_profile.no_repeat_ngram_size,
+                    128,
+                )],
+                verbose=False,
+            )
+        else:
+            response = engine.infer(
+                str(img_file),
+                custom_prompt=prompt,
+                max_tokens=token_limit,
+                profile=profile,
+            )
         elapsed = time.perf_counter() - start
         total_time += elapsed
 
@@ -135,7 +289,10 @@ def run_accuracy_benchmark(
         elif hasattr(response, "text"):
             hypothesis = response.text
         else:
-            hypothesis = str(response)
+            raise TypeError(
+                "Inference backend returned an unsupported accuracy response "
+                f"({type(response).__name__})"
+            )
         hypothesis = normalize_ocr_output(hypothesis)
 
         # Compute metrics
@@ -151,8 +308,10 @@ def run_accuracy_benchmark(
             else None
         )
 
+        category = (category_map or {}).get(img_file.name)
         result = {
             "file": img_file.name,
+            "category": category,
             "cer": cer,
             "wer": wer,
             "digit_cer": digit_cer,
@@ -169,6 +328,12 @@ def run_accuracy_benchmark(
         }
         results.append(result)
 
+        if save_predictions is not None:
+            (save_predictions / f"{img_file.stem}.txt").write_text(
+                hypothesis + ("\n" if hypothesis and not hypothesis.endswith("\n") else ""),
+                encoding="utf-8",
+            )
+
         print(f"  [{i+1}/{len(evaluation_files)}] {img_file.name}: CER={cer:.4f} "
               f"DigitCER={digit_cer:.4f} ({elapsed:.1f}s)")
 
@@ -177,19 +342,30 @@ def run_accuracy_benchmark(
     digit_scores = [r["digit_cer"] for r in results if r["ref_digit_count"] > 0]
     cjk_scores = [r["cjk_cer"] for r in results if r["ref_cjk_count"] > 0]
     table_scores = [r["table_score"] for r in results if r["table_score"] is not None]
+    generation_settings = {
+        "temperature": generation_profile.temperature,
+        "top_p": generation_profile.top_p,
+        "repetition_penalty": generation_profile.repetition_penalty,
+        "no_repeat_ngram_size": generation_profile.no_repeat_ngram_size,
+        "ngram_window": 128,
+    }
+    if backend == "vllm":
+        # The official runtime recipe always uses 35, even when an MLX
+        # generation profile has an experimental alternate value.
+        generation_settings.update({
+            "no_repeat_ngram_size": 35,
+            "skip_special_tokens": False,
+        })
+    by_category = aggregate_by_category(results)
     summary = {
         "model_path": Path(model_path).name if Path(model_path).is_dir() else model_path,
+        "served_revision": served_revision,
+        "backend": backend,
         "prompt": prompt,
         "max_tokens": token_limit,
         "profile": profile,
-        "generation_settings": {
-            "temperature": generation_profile.temperature,
-            "top_p": generation_profile.top_p,
-            "repetition_penalty": generation_profile.repetition_penalty,
-            "no_repeat_ngram_size": generation_profile.no_repeat_ngram_size,
-            "ngram_window": 128,
-        },
-        "num_images": len(image_files),
+        "generation_settings": generation_settings,
+        "num_images": len(evaluation_files),
         "num_samples": len(results),
         "total_time_seconds": total_time,
         "mean_cer": float(np.mean([r["cer"] for r in results])) if results else 0,
@@ -201,6 +377,7 @@ def run_accuracy_benchmark(
         "num_table_samples": len(table_scores),
         "mean_table_score": float(np.mean(table_scores)) if table_scores else None,
         "mean_repetition_rate": float(np.mean([r["repetition_rate"] for r in results])) if results else 0,
+        "by_category": by_category,
         "per_file": results,
     }
 
@@ -217,6 +394,20 @@ def main():
                         help="Override the selected profile's token limit")
     parser.add_argument("--profile", default="accurate",
                         choices=["accurate", "fast", "long-document", "plain-text", "markdown"])
+    parser.add_argument("--backend", default="mlx", choices=["mlx", "vllm"])
+    parser.add_argument("--vllm-base-url", default="http://127.0.0.1:8000/v1")
+    parser.add_argument("--vllm-api-key", default=os.environ.get("VLLM_API_KEY"))
+    parser.add_argument(
+        "--served-revision",
+        default=None,
+        help="Revision pinned when this model was started (required for BF16 release evidence)",
+    )
+    parser.add_argument(
+        "--save-predictions",
+        type=Path,
+        default=None,
+        help="Optional directory to write normalized hypothesis text per image stem",
+    )
     args = parser.parse_args()
 
     print("=" * 60)
@@ -229,11 +420,19 @@ def main():
         prompt=args.prompt,
         max_tokens=args.max_tokens,
         profile=args.profile,
+        backend=args.backend,
+        vllm_base_url=args.vllm_base_url,
+        vllm_api_key=args.vllm_api_key,
+        served_revision=args.served_revision,
+        save_predictions=args.save_predictions,
     )
 
     # Save results
     args.output.parent.mkdir(parents=True, exist_ok=True)
-    args.output.write_text(json.dumps(results, indent=2, ensure_ascii=False), encoding="utf-8")
+    args.output.write_text(
+        json.dumps(results, indent=2, ensure_ascii=False, allow_nan=False),
+        encoding="utf-8",
+    )
 
     def metric(value):
         return "N/A" if value is None else f"{value:.4f}"
@@ -242,6 +441,15 @@ def main():
     print(f"Results: CER={results['mean_cer']:.4f} | "
           f"DigitCER={metric(results['mean_digit_cer'])} | "
           f"CJK={metric(results['mean_cjk_cer'])}")
+    if results.get("by_category"):
+        print("By category:")
+        for category, stats in results["by_category"].items():
+            print(
+                f"  {category}: n={stats['num_samples']} "
+                f"CER={metric(stats['mean_cer'])} "
+                f"Digit={metric(stats['mean_digit_cer'])} "
+                f"CJK={metric(stats['mean_cjk_cer'])}"
+            )
     print(f"Saved to: {args.output}")
 
 

benchmarks/run_performance.py

@@ -12,10 +12,11 @@ from __future__ import annotations
 import argparse
 import importlib.metadata
 import json
-import os
+import math
 import platform
 import sys
 import time
+from numbers import Integral, Real
 from pathlib import Path
 
 sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
@@ -83,6 +84,7 @@ def run_performance_benchmark(
     max_tokens: int = 4096,
     num_warmup: int = 1,
     num_runs: int = 3,
+    served_revision: str | None = None,
 ) -> dict:
     """Run performance benchmark measuring TPS, memory, and latency."""
     if not Path(image_path).is_file():
@@ -95,6 +97,10 @@ def run_performance_benchmark(
         raise ValueError("num_warmup must be a non-negative integer")
     if not isinstance(num_runs, int) or isinstance(num_runs, bool) or num_runs < 1:
         raise ValueError("num_runs must be a positive integer")
+    if served_revision is not None and (
+        not isinstance(served_revision, str) or not served_revision.strip()
+    ):
+        raise ValueError("served_revision must be a non-empty string or None")
 
     from mlx_vlm import load, generate
 
@@ -105,7 +111,7 @@ def run_performance_benchmark(
 
     # Measure model load time
     load_start = time.perf_counter()
-    model, processor = load(model_path)
+    model, processor = load(model_path, revision=served_revision)
     load_time = time.perf_counter() - load_start
     mem_after_load = get_memory_usage_mb()
 
@@ -127,6 +133,8 @@ def run_performance_benchmark(
 
     for i in range(num_runs):
         active_memory_before_mb = mx.get_active_memory() / 1e6
+        if not math.isfinite(active_memory_before_mb) or active_memory_before_mb < 0:
+            raise ValueError("MLX returned invalid active memory")
         mx.reset_peak_memory()
         start = time.perf_counter()
 
@@ -140,7 +148,11 @@ def run_performance_benchmark(
         )
 
         elapsed = time.perf_counter() - start
+        if not math.isfinite(elapsed) or elapsed <= 0:
+            raise ValueError("Performance timer returned an invalid elapsed time")
         process_peak_mb = get_memory_usage_mb()
+        if not math.isfinite(process_peak_mb) or process_peak_mb < 0:
+            raise ValueError("Process peak memory is invalid")
 
         # Extract token count
         if isinstance(response, str):
@@ -179,24 +191,55 @@ def run_performance_benchmark(
             text = str(response)
             tokens = len(text.split())
             token_count_source = "whitespace estimate"
-        tokens = int(tokens or 0)
+        if (
+            not isinstance(tokens, Integral)
+            or isinstance(tokens, bool)
+            or not 0 < tokens <= max_tokens
+        ):
+            raise ValueError("mlx-vlm returned an invalid generation token count")
+        tokens = int(tokens)
 
         # Prefer mlx-vlm's decode-only rate; the fallback is end-to-end and is
         # explicitly identified in each run.
-        reported_tps = getattr(response, "generation_tps", 0.0)
+        reported_tps = (
+            response.get("generation_tps", 0.0)
+            if isinstance(response, dict)
+            else getattr(response, "generation_tps", 0.0)
+        )
         if reported_tps:
+            if (
+                not isinstance(reported_tps, Real)
+                or isinstance(reported_tps, bool)
+                or not math.isfinite(float(reported_tps))
+                or reported_tps <= 0
+            ):
+                raise ValueError("mlx-vlm returned invalid generation throughput")
             tps = float(reported_tps)
             tps_source = "mlx-vlm generation_tps"
         else:
             tps = tokens / elapsed if elapsed > 0 and tokens > 0 else 0
             tps_source = "end-to-end estimate"
 
-        reported_peak_gb = float(getattr(response, "peak_memory", 0.0) or 0.0)
+        raw_peak_memory = (
+            response.get("peak_memory", 0.0)
+            if isinstance(response, dict)
+            else getattr(response, "peak_memory", 0.0)
+        )
+        if (
+            not isinstance(raw_peak_memory, Real)
+            or isinstance(raw_peak_memory, bool)
+            or not math.isfinite(float(raw_peak_memory))
+            or raw_peak_memory < 0
+        ):
+            raise ValueError("mlx-vlm returned invalid peak memory")
+        reported_peak_gb = float(raw_peak_memory)
         mlx_peak_mb = (
             reported_peak_gb * 1000
             if reported_peak_gb > 0
             else mx.get_peak_memory() / 1e6
         )
+        if not math.isfinite(mlx_peak_mb) or mlx_peak_mb < 0:
+            raise ValueError("MLX returned invalid peak memory")
         run_result = {
             "run": i + 1,
             "elapsed_seconds": elapsed,
@@ -225,6 +268,7 @@ def run_performance_benchmark(
 
     results = {
         "model_path": Path(model_path).name if Path(model_path).is_dir() else model_path,
+        "served_revision": served_revision,
         "image_path": Path(image_path).name,
         "prompt": prompt,
         "max_tokens": max_tokens,
@@ -252,6 +296,11 @@ def main():
     parser.add_argument("--max-tokens", type=int, default=4096)
     parser.add_argument("--warmup", type=int, default=1)
     parser.add_argument("--runs", type=int, default=3)
+    parser.add_argument(
+        "--served-revision",
+        default=None,
+        help="Immutable model revision used for this benchmark",
+    )
     args = parser.parse_args()
 
     print("=" * 60)
@@ -265,6 +314,7 @@ def main():
         max_tokens=args.max_tokens,
         num_warmup=args.warmup,
         num_runs=args.runs,
+        served_revision=args.served_revision,
     )
 
     args.output.parent.mkdir(parents=True, exist_ok=True)

config.json

@@ -666,11 +666,6 @@
       "group_size": 32,
       "bits": 8,
       "mode": "affine"
-    },
-    "projector.layers": {
-      "group_size": 32,
-      "bits": 8,
-      "mode": "mxfp8"
     }
   },
   "quantization_config": {
@@ -1271,11 +1266,6 @@
       "group_size": 32,
       "bits": 8,
       "mode": "affine"
-    },
-    "projector.layers": {
-      "group_size": 32,
-      "bits": 8,
-      "mode": "mxfp8"
     }
   },
   "rm_head": false,

docs/PRD.md

@@ -62,6 +62,8 @@ performance, provenance, and R-SWA checks pass.
 
 - Validate image/ground-truth pairs before allocating the model.
 - Record dataset manifest digest and category counts.
+- Use disjoint, content-addressed datasets for sensitivity/calibration and the
+  final release comparison; shared image hashes block publication.
 - Release evaluation must include text, digits, tables, and CJK content.
 - Synthetic-only evaluation may be used for development, but not labelled as a
   public accuracy benchmark.

docs/TECHNICAL_SPEC.md

@@ -113,6 +113,14 @@ artifacts/<run_id>/
   provenance.json
   sensitivity_results.json
   calibration_results.json
+  calibration_baseline_accuracy.json
+  calibration_reference_performance.json
+  calibration_bfloat16_accuracy.json
+  calibration_bfloat16_performance.json
+  calibration_mxfp8_accuracy.json
+  calibration_mxfp8_performance.json
+  calibration_affine8_accuracy.json
+  calibration_affine8_performance.json
   generated_precision_map.json
   bf16_accuracy.json
   reference_accuracy.json
@@ -139,7 +147,11 @@ artifacts without loading a model. Every gate records:
 ```
 
 `release_approved` is true only when every required gate passes. Missing or
-inconclusive values fail the gate.
+inconclusive values fail the gate. The calibration decision is not trusted as
+an assertion: the gate verifies SHA-256 records for all eight raw calibration
+benchmark files, recomputes their aggregates and quality/throughput deltas, and
+reselects the fastest precision that passes the fixed limits. It also rejects a
+release when calibration and held-out evaluation reuse any image hash.
 
 ### 2.7 Publisher
 
@@ -163,8 +175,11 @@ Primary orchestration command:
 ```bash
 python quantization/run_pipeline.py \
   --model-path reference/Unlimited-OCR \
+  --source-revision SOURCE_COMMIT_SHA \
   --reference-model sahilchachra/unlimited-ocr-mxfp8-mlx \
-  --eval-dir eval_dataset \
+  --reference-revision REFERENCE_COMMIT_SHA \
+  --calibration-dir datasets/selection \
+  --eval-dir datasets/held-out-release \
   --output-dir models/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \
   --artifacts-dir artifacts/ocr-aware-v1 \
   --image test_data/test_invoice.png
@@ -184,20 +199,25 @@ Useful modes:
 
 - `--step preflight`
 - `--step sensitivity`
+- `--step calibrate` — interim map, three LM-head converts, select affine/mxfp8/bf16 head
 - `--step precision-map`
 - `--step convert`
 - `--step validate`
 - `--step gate`
 - `--step publish`
-- `--step all`
+- `--step all` (runs the full order above)
 - `--dry-run`
 
+`calibrate` is required before `precision-map` for schema-3 releases: it writes
+`calibration_results.json` plus the eight raw `calibration_*_{accuracy,performance}.json`
+evidence files on the **calibration** dataset only.
+
 ## 4. Provenance schema
 
 `provenance.json` includes:
 
 - UTC run ID and timestamps;
-- source and reference identifiers/revisions;
+- source and reference identifiers plus immutable 40-character commit SHAs;
 - file SHA-256 and byte sizes;
 - dataset digest and category counts;
 - Python, macOS, MLX, mlx-vlm, and huggingface-hub versions;
@@ -220,6 +240,10 @@ Published `processor_config.json` must use:
 - `processor_class: UnlimitedOCRHFProcessor`;
 - `sft_format: unlimitedocr`.
 
+If `tokenizer_config.json` is present, it must use the same
+`processor_class: UnlimitedOCRHFProcessor` (convert normalization rewrites both
+files so mlx-vlm and Hub-style loaders do not disagree).
+
 ## 6. Error handling
 
 - Never delete or overwrite an existing output model directory.
@@ -258,3 +282,45 @@ After publication:
 3. Load the exact remote revision with `mlx-vlm`.
 4. Run deterministic OCR on the smoke image.
 5. Record the final revision in the local release manifest and handoff report.
+
+## 9. Schema-3 re-release checklist (MLX Hub)
+
+The publisher requires `MLX_RELEASE_SCHEMA_VERSION = 3`. A legacy
+`artifacts/ocr-aware-v1/release_manifest.json` with `schema_version: 1` cannot
+be re-uploaded without a full re-gate.
+
+1. Build **disjoint** calibration and evaluation trees (no shared FUNSD IDs):
+
+```bash
+python eval_dataset/build_release_set.py --clean \
+  --output-root eval_dataset/calibration \
+  --funsd-ids 0,1 --generator-seed 20260720 \
+  --funsd-revision <FUNSD_SHA> --latin-font <PATH> --cjk-font <PATH>
+
+python eval_dataset/build_release_set.py --clean \
+  --output-root eval_dataset/evaluation \
+  --funsd-ids 2,3,4,5 --generator-seed 20260721 \
+  --funsd-revision <FUNSD_SHA> --latin-font <PATH> --cjk-font <PATH>
+```
+
+2. Run the OCR-aware pipeline / release gate with
+   `--calibration-dir eval_dataset/calibration` and
+   `--eval-dir eval_dataset/evaluation` (schema 3, all 23 gates).
+3. Confirm convert normalization rewrote both processor and tokenizer
+   `processor_class` fields to `UnlimitedOCRHFProcessor`.
+4. Dry-run then publish:
+
+```bash
+python scripts/publish_optimized_model.py \
+  --model-dir models/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 \
+  --manifest artifacts/<run>/release_manifest.json \
+  --artifacts-dir artifacts/<run> \
+  --dry-run
+# then without --dry-run when approved
+```
+
+5. Post-publish smoke against the remote revision; keep evidence under
+   `artifacts/`.
+
+Model-card claims must stay local-measurement only until a public protocol
+(OmniDocBench / olmOCR-Bench) is run on the candidate.

docs/adr/0002-calibrated-affine8-lm-head.md

@@ -4,6 +4,11 @@ Status: Accepted
 Date: 2026-07-20  
 Amends: ADR-0001 for `language_model.lm_head` only
 
+Evidence status: this records a historical candidate-construction decision,
+not current release approval. Its calibration and evaluation reused the same
+12 samples. The schema-3 gate now requires disjoint calibration and held-out
+evaluation data before this decision can approve publication.
+
 ## Context
 
 The group sensitivity run showed that the vision encoder must remain BF16, while
@@ -15,7 +20,7 @@ group sensitivity could not predict:
 |---|---:|---:|---:|---|
 | BF16 | 0.231904 | 0.082224 | 271.1 | throughput failed |
 | MXFP8 | 0.260170 | 0.076961 | 307.4 | CER failed |
-| affine INT8 | 0.237384 | 0.085733 | 301.2 | all calibration gates passed |
+| affine INT8 | 0.237384 | 0.085733 | 301.2 | development calibration passed |
 
 The common BF16 baseline CER was 0.239188 and digit CER was 0.087340. The
 Sahil-reference throughput was 312.7 tok/s. The selection procedure used the
@@ -54,7 +59,8 @@ fail-closed release design.
 
 - The checkpoint remains predominantly MXFP8 but is accurately described as a
   mixed MXFP8/affine8 model.
-- The final formal rerun reached 297.2 tok/s, 95.0% of the Sahil reference,
-  while mean CER and digit CER remained slightly better than BF16.
+- The historical same-dataset rerun reached 297.2 tok/s, 95.0% of the Sahil
+  reference, while mean CER and digit CER were slightly better than BF16. This
+  does not replace a disjoint held-out release evaluation.
 - Consumers must use a recent MLX/`mlx-vlm` version that supports per-module
   affine and MXFP8 quantization metadata.

examples/single_image.py

@@ -14,7 +14,7 @@ print(result)
 
 # --- Markdown output ---
 result = pipeline.run("your_document.jpg", format="markdown", output_path="output.md")
-print(f"Saved to output.md")
+print("Saved to output.md")
 
 # --- With bounding boxes (grounding mode) ---
 result = pipeline.run("your_document.jpg", format="json", grounding=True)

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27568517565832e42daa563f6939e66a8ebb50cce0b01a24ecebb3ea6b4d0b10
-size 3843160559
+oid sha256:d9fe11fc8d2e333000be6a0e288681231efa69282c844f7775ca40f52db02f0c
+size 3845699993

model.safetensors.index.json

@@ -1,6 +1,6 @@
 {
     "metadata": {
-        "total_size": 3843066880
+        "total_size": 3845606400
     },
     "weight_map": {
         "image_newline": "model.safetensors",
@@ -280,7 +280,6 @@
         "language_model.model.layers.9.self_attn.v_proj.weight": "model.safetensors",
         "language_model.model.norm.weight": "model.safetensors",
         "projector.layers.bias": "model.safetensors",
-        "projector.layers.scales": "model.safetensors",
         "projector.layers.weight": "model.safetensors",
         "sam_model.blocks.0.attn.proj.bias": "model.safetensors",
         "sam_model.blocks.0.attn.proj.weight": "model.safetensors",

precision_map.json

@@ -1,6 +1,6 @@
 {
   "vision_model": "bfloat16",
-  "projector": "mxfp8",
+  "projector": "bfloat16",
   "language_model.model.embed_tokens": "mxfp8",
   "language_model.model.norm": "bfloat16",
   "language_model.lm_head": "affine8",
@@ -32,16 +32,16 @@
       "vision_encoder": {
         "precision": "bfloat16",
         "reasons": [
-          "cer_delta=2.423686",
-          "digit_cer_delta=1.358023",
+          "cer_delta=2.272233",
+          "digit_cer_delta=1.696454",
           "table_score_degradation=1.000000",
           "protected OCR-critical group"
         ]
       },
       "vision_projector": {
-        "precision": "mxfp8",
+        "precision": "bfloat16",
         "reasons": [
-          "all measured quality deltas were non-degrading"
+          "protected unless all measured quality deltas are non-degrading"
         ]
       },
       "token_embeddings": {

preprocessing_config.json

@@ -66,7 +66,7 @@
       "target_size": 1024
     },
     "table": {
-      "description": "Sharpen + edge enhancement for grid lines and table structure",
+      "description": "Deskew + CLAHE + sharpening for grid lines and table structure",
       "deskew": true,
       "clahe": true,
       "binarize": false,
@@ -75,6 +75,29 @@
       "grayscale": false,
       "target_size": 1024,
       "clahe_clip_limit": 3.0
+    },
+    "adaptive-binarize": {
+      "description": "Detect sideways orientation and apply local Sauvola binarization",
+      "deskew": true,
+      "clahe": false,
+      "binarize": "sauvola",
+      "gamma_correction": false,
+      "sharpen": false,
+      "grayscale": false,
+      "target_size": 1024
+    },
+    "photo-document": {
+      "description": "Perspective-correct photographed pages, detect sideways orientation, then enhance contrast and lighting",
+      "deskew": false,
+      "perspective_correction": true,
+      "clahe": true,
+      "binarize": false,
+      "gamma_correction": true,
+      "sharpen": false,
+      "grayscale": false,
+      "target_size": 1024,
+      "clahe_clip_limit": 3.0,
+      "gamma_value": 1.3
     }
   },
   "_note": "Preprocessing is always optional. The raw input path remains available for baseline comparison. Applied steps are recorded in output metadata."

pyproject.toml

@@ -4,15 +4,15 @@ build-backend = "hatchling.build"
 
 [project]
 name = "unlimited-ocr"
-version = "0.2.0"
-description = "Improved MLX inference pipeline for Unlimited-OCR — one-shot long-horizon document parsing on Apple Silicon"
+version = "0.3.0"
+description = "Unlimited-OCR inference toolkit for Apple MLX and NVIDIA CUDA/vLLM"
 readme = "README.md"
 license = "MIT"
 requires-python = ">=3.10"
 authors = [
     { name = "AutomatosX" },
 ]
-keywords = ["ocr", "mlx", "apple-silicon", "document-parsing", "vision-language-model"]
+keywords = ["ocr", "mlx", "cuda", "vllm", "document-parsing", "vision-language-model"]
 classifiers = [
     "Development Status :: 4 - Beta",
     "Intended Audience :: Developers",
@@ -20,13 +20,10 @@ classifiers = [
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Environment :: GPU :: NVIDIA CUDA",
     "Topic :: Scientific/Engineering :: Artificial Intelligence",
 ]
 dependencies = [
-    # Unlimited-OCR support first shipped in mlx-vlm 0.6.4. Older releases
-    # cannot load the default model used by this package.
-    "mlx-vlm>=0.6.4",
-    "mlx>=0.31.2",
     "Pillow>=10.0",
     "numpy",
     "pymupdf>=1.24",
@@ -39,6 +36,18 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
+mlx = [
+    # Unlimited-OCR support first shipped in mlx-vlm 0.6.4. Keeping MLX
+    # optional lets the same wheel install on Linux CUDA hosts.
+    "mlx-vlm>=0.6.4; platform_system == 'Darwin'",
+    "mlx>=0.31.2; platform_system == 'Darwin'",
+]
+vllm-client = ["httpx>=0.27,<1"]
+vllm-server = [
+    # Unlimited-OCR became a native architecture in vLLM 0.25.
+    "vllm>=0.25.1,<0.26; platform_system == 'Linux' and platform_machine == 'x86_64'",
+    "httpx>=0.27,<1",
+]
 server = ["fastapi>=0.100", "uvicorn[standard]>=0.20", "python-multipart>=0.0.5"]
 dev = ["pytest>=7.0", "pytest-asyncio>=0.21", "httpx>=0.24"]
 

quantization/calibrate_precision.py

@@ -5,14 +5,23 @@ from __future__ import annotations
 
 import argparse
 import json
+import math
 import sys
+import tempfile
 from datetime import datetime, timezone
 from pathlib import Path
 
 PROJECT_ROOT = Path(__file__).resolve().parent.parent
 sys.path.insert(0, str(PROJECT_ROOT))
 
-from quantization.release_gate import DEFAULT_THRESHOLDS, load_json_object
+from quantization.release_gate import (  # noqa: E402
+    CALIBRATION_EVIDENCE_KEYS,
+    dataset_manifest,
+    load_json_object,
+    sha256_file,
+    validate_calibration_results,
+    validate_release_thresholds,
+)
 
 
 SUPPORTED_HEAD_PRECISIONS = {"bfloat16", "mxfp8", "affine8"}
@@ -20,7 +29,13 @@ SUPPORTED_HEAD_PRECISIONS = {"bfloat16", "mxfp8", "affine8"}
 
 def _number(payload: dict, key: str) -> float | None:
     value = payload.get(key)
-    return float(value) if isinstance(value, (int, float)) and not isinstance(value, bool) else None
+    if (
+        not isinstance(value, (int, float))
+        or isinstance(value, bool)
+        or not math.isfinite(float(value))
+    ):
+        return None
+    return float(value)
 
 
 def select_head_precision(
@@ -30,15 +45,34 @@ def select_head_precision(
     thresholds: dict | None = None,
 ) -> dict:
     """Choose the fastest experiment that passes the existing release limits."""
-    limits = {**DEFAULT_THRESHOLDS, **(thresholds or {})}
+    limits = validate_release_thresholds(thresholds)
     bf16_cer = _number(bf16_accuracy, "mean_cer")
     bf16_digit = _number(bf16_accuracy, "mean_digit_cer")
     bf16_table = _number(bf16_accuracy, "mean_table_score")
     reference_tps = _number(reference_performance, "mean_tps")
     if None in (bf16_cer, bf16_digit, bf16_table, reference_tps) or reference_tps <= 0:
         raise ValueError("Baseline accuracy and reference throughput must be complete")
-    if not experiments:
+    if not isinstance(experiments, list) or not experiments:
         raise ValueError("At least one calibration experiment is required")
+    if any(not isinstance(experiment, dict) for experiment in experiments):
+        raise ValueError("Each calibration experiment must be an object")
+    labels = [experiment.get("label") for experiment in experiments]
+    precisions = [experiment.get("precision") for experiment in experiments]
+    if any(
+        not isinstance(label, str) or not label.strip() or label.strip() != label
+        for label in labels
+    ):
+        raise ValueError("Each calibration experiment needs a normalized label")
+    if any(not isinstance(precision, str) for precision in precisions):
+        raise ValueError("Each calibration experiment needs a precision")
+    if len(set(labels)) != len(labels):
+        raise ValueError("Calibration experiment labels must be unique")
+    if len(set(precisions)) != len(precisions):
+        raise ValueError("Calibration experiment precisions must be unique")
+    if set(precisions) != SUPPORTED_HEAD_PRECISIONS:
+        raise ValueError(
+            "Calibration requires exactly one experiment for each supported precision"
+        )
 
     evaluated = []
     for experiment in experiments:
@@ -46,8 +80,6 @@ def select_head_precision(
         precision = experiment.get("precision")
         accuracy = experiment.get("accuracy")
         performance = experiment.get("performance")
-        if not isinstance(label, str) or not label:
-            raise ValueError("Each calibration experiment needs a label")
         if precision not in SUPPORTED_HEAD_PRECISIONS:
             raise ValueError(f"Unsupported head precision for {label}: {precision}")
         if not isinstance(accuracy, dict) or not isinstance(performance, dict):
@@ -57,40 +89,59 @@ def select_head_precision(
         candidate_table = _number(accuracy, "mean_table_score")
         candidate_tps = _number(performance, "mean_tps")
         metrics_complete = None not in (
-            candidate_cer, candidate_digit, candidate_table, candidate_tps
+            candidate_cer,
+            candidate_digit,
+            candidate_table,
+            candidate_tps,
         )
         deltas = {
             "cer_vs_bf16": candidate_cer - bf16_cer if metrics_complete else None,
-            "digit_cer_vs_bf16": candidate_digit - bf16_digit if metrics_complete else None,
-            "table_degradation_vs_bf16": bf16_table - candidate_table if metrics_complete else None,
-            "tps_ratio_vs_reference": candidate_tps / reference_tps if metrics_complete else None,
+            "digit_cer_vs_bf16": candidate_digit - bf16_digit
+            if metrics_complete
+            else None,
+            "table_degradation_vs_bf16": bf16_table - candidate_table
+            if metrics_complete
+            else None,
+            "tps_ratio_vs_reference": candidate_tps / reference_tps
+            if metrics_complete
+            else None,
         }
         checks = {
-            "cer": metrics_complete and deltas["cer_vs_bf16"] <= limits["max_cer_delta_vs_bf16"],
-            "digit_cer": metrics_complete and deltas["digit_cer_vs_bf16"] <= limits["max_digit_cer_delta_vs_bf16"],
-            "table_score": metrics_complete and deltas["table_degradation_vs_bf16"] <= limits["max_table_score_degradation_vs_bf16"],
-            "throughput": metrics_complete and deltas["tps_ratio_vs_reference"] >= limits["min_tps_ratio_vs_reference"],
+            "cer": metrics_complete
+            and deltas["cer_vs_bf16"] <= limits["max_cer_delta_vs_bf16"],
+            "digit_cer": metrics_complete
+            and deltas["digit_cer_vs_bf16"] <= limits["max_digit_cer_delta_vs_bf16"],
+            "table_score": metrics_complete
+            and deltas["table_degradation_vs_bf16"]
+            <= limits["max_table_score_degradation_vs_bf16"],
+            "throughput": metrics_complete
+            and deltas["tps_ratio_vs_reference"]
+            >= limits["min_tps_ratio_vs_reference"],
         }
-        evaluated.append({
-            "label": label,
-            "precision": precision,
-            "passed": all(checks.values()),
-            "checks": checks,
-            "metrics": {
-                "mean_cer": candidate_cer,
-                "mean_digit_cer": candidate_digit,
-                "mean_table_score": candidate_table,
-                "mean_tps": candidate_tps,
-            },
-            "deltas": deltas,
-        })
+        evaluated.append(
+            {
+                "label": label,
+                "precision": precision,
+                "passed": all(checks.values()),
+                "checks": checks,
+                "metrics": {
+                    "mean_cer": candidate_cer,
+                    "mean_digit_cer": candidate_digit,
+                    "mean_table_score": candidate_table,
+                    "mean_tps": candidate_tps,
+                },
+                "deltas": deltas,
+            }
+        )
 
     passing = [experiment for experiment in evaluated if experiment["passed"]]
     if not passing:
-        raise RuntimeError("No LM-head calibration experiment passed every release limit")
+        raise RuntimeError(
+            "No LM-head calibration experiment passed every release limit"
+        )
     selected = max(passing, key=lambda experiment: experiment["metrics"]["mean_tps"])
     return {
-        "schema_version": 1,
+        "schema_version": 3,
         "created_at": datetime.now(timezone.utc).isoformat(),
         "target_pattern": "language_model.lm_head",
         "selection_policy": "fastest candidate passing existing quality and throughput limits",
@@ -110,6 +161,14 @@ def main() -> None:
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("--bf16-accuracy", required=True, type=Path)
     parser.add_argument("--reference-performance", required=True, type=Path)
+    parser.add_argument(
+        "--calibration-dir",
+        required=True,
+        type=Path,
+        help="Selection-only dataset used for every calibration accuracy run",
+    )
+    parser.add_argument("--source-revision", required=True)
+    parser.add_argument("--reference-revision", required=True)
     parser.add_argument(
         "--experiment",
         action="append",
@@ -119,22 +178,91 @@ def main() -> None:
     )
     parser.add_argument("--output", required=True, type=Path)
     args = parser.parse_args()
-    experiments = [
-        {
-            "label": label,
-            "precision": precision,
-            "accuracy": load_json_object(accuracy_path),
-            "performance": load_json_object(performance_path),
-        }
-        for label, precision, accuracy_path, performance_path in args.experiment
-    ]
+    bf16_accuracy = load_json_object(args.bf16_accuracy)
+    reference_performance = load_json_object(args.reference_performance)
+    experiments = []
+    calibration_dataset = dataset_manifest(args.calibration_dir)
+    evidence = {
+        "calibration_baseline_accuracy": bf16_accuracy,
+        "calibration_reference_performance": reference_performance,
+    }
+    evidence_paths: dict[str, Path] = {
+        "calibration_baseline_accuracy": args.bf16_accuracy,
+        "calibration_reference_performance": args.reference_performance,
+    }
+    for label, precision, accuracy_path, performance_path in args.experiment:
+        if precision not in CALIBRATION_EVIDENCE_KEYS:
+            raise ValueError(f"Unsupported calibration precision: {precision}")
+        accuracy_key, performance_key = CALIBRATION_EVIDENCE_KEYS[precision]
+        accuracy_path = Path(accuracy_path)
+        performance_path = Path(performance_path)
+        evidence[accuracy_key] = load_json_object(accuracy_path)
+        evidence[performance_key] = load_json_object(performance_path)
+        evidence_paths[accuracy_key] = accuracy_path
+        evidence_paths[performance_key] = performance_path
+        experiments.append(
+            {
+                "label": label,
+                "precision": precision,
+                "accuracy": evidence[accuracy_key],
+                "performance": evidence[performance_key],
+            }
+        )
     result = select_head_precision(
-        load_json_object(args.bf16_accuracy),
-        load_json_object(args.reference_performance),
+        bf16_accuracy,
+        reference_performance,
         experiments,
     )
+    expected_input_keys = {
+        "calibration_baseline_accuracy",
+        "calibration_reference_performance",
+        *(name for pair in CALIBRATION_EVIDENCE_KEYS.values() for name in pair),
+    }
+    if set(evidence_paths) != expected_input_keys:
+        raise ValueError("Calibration requires one unique evidence pair per precision")
+    resolved_paths = [path.resolve(strict=True) for path in evidence_paths.values()]
+    if len(set(resolved_paths)) != len(resolved_paths):
+        raise ValueError("Calibration input artifacts must be distinct files")
+    result["input_artifacts"] = {
+        name: {
+            "filename": path.name,
+            "size": path.stat().st_size,
+            "sha256": sha256_file(path),
+        }
+        for name, path in evidence_paths.items()
+    }
+    result["dataset"] = calibration_dataset
+    validate_calibration_results(
+        result,
+        bf16_accuracy=bf16_accuracy,
+        reference_performance=reference_performance,
+        calibration_dataset=calibration_dataset,
+        source_revision=args.source_revision,
+        reference_revision=args.reference_revision,
+        evidence=evidence,
+        evidence_paths=evidence_paths,
+    )
+    if args.output.is_symlink() or (args.output.exists() and not args.output.is_file()):
+        raise ValueError(f"Output must be a regular file: {args.output}")
     args.output.parent.mkdir(parents=True, exist_ok=True)
-    args.output.write_text(json.dumps(result, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+    temporary_path = None
+    try:
+        with tempfile.NamedTemporaryFile(
+            mode="w",
+            encoding="utf-8",
+            dir=args.output.parent,
+            prefix=f".{args.output.name}.",
+            suffix=".tmp",
+            delete=False,
+        ) as temporary:
+            temporary_path = Path(temporary.name)
+            json.dump(result, temporary, indent=2, ensure_ascii=False, allow_nan=False)
+            temporary.write("\n")
+        temporary_path.replace(args.output)
+        temporary_path = None
+    finally:
+        if temporary_path is not None:
+            temporary_path.unlink(missing_ok=True)
     print(json.dumps(result["selected"], ensure_ascii=False))
 
 

quantization/layer_sensitivity.py

@@ -29,12 +29,13 @@ PROJECT_ROOT = Path(__file__).parent.parent
 sys.path.insert(0, str(PROJECT_ROOT))
 sys.path.insert(0, str(PROJECT_ROOT / "src"))
 
-from benchmarks.evaluate_cer import compute_cer as _compute_cer
-from benchmarks.evaluate_cer import compute_digit_cer as _compute_digit_cer
-from benchmarks.evaluate_tables import evaluate_tables
-from benchmarks.normalize_output import normalize_ocr_output
-from quantization.mixed_precision_convert import matches_pattern
-from unlimited_ocr.engine import SlidingWindowNoRepeatNGramProcessor
+from benchmarks.evaluate_cer import compute_cer as _compute_cer  # noqa: E402
+from benchmarks.evaluate_cer import compute_digit_cer as _compute_digit_cer  # noqa: E402
+from benchmarks.evaluate_tables import evaluate_tables  # noqa: E402
+from benchmarks.normalize_output import normalize_ocr_output  # noqa: E402
+from quantization.mixed_precision_convert import matches_pattern  # noqa: E402
+from quantization.release_gate import dataset_manifest  # noqa: E402
+from unlimited_ocr.engine import SlidingWindowNoRepeatNGramProcessor  # noqa: E402
 
 
 # Quantizable MLX module groups to test independently. Normalization and the
@@ -239,6 +240,11 @@ def main():
         default=None,
         help="Public source identifier recorded in results (defaults to repo ID or local name)",
     )
+    parser.add_argument(
+        "--source-revision",
+        default=None,
+        help="Immutable source commit used for remote loading and provenance",
+    )
     parser.add_argument("--eval-dir", required=True, type=Path,
                         help="Evaluation directory with images/ and ground_truth/")
     parser.add_argument("--output", type=Path, default=Path("sensitivity_results.json"),
@@ -259,6 +265,7 @@ def main():
 
     # Fail before importing MLX or allocating the model for dataset mistakes.
     collect_evaluation_files(args.eval_dir, args.prompt, args.max_tokens)
+    dataset = dataset_manifest(args.eval_dir)
 
     print("=" * 60)
     print("Layer Sensitivity Analysis for Unlimited-OCR MLX")
@@ -270,12 +277,12 @@ def main():
     from mlx_vlm.utils import load_config
     import mlx.core as mx
 
-    base_config = load_config(args.model_path)
+    base_config = load_config(args.model_path, revision=args.source_revision)
     if base_config.get("quantization") or base_config.get("quantization_config"):
         raise ValueError(
             "Layer sensitivity requires an unquantized BF16 source model"
         )
-    model, processor = load(args.model_path)
+    model, processor = load(args.model_path, revision=args.source_revision)
 
     # Step 2: Evaluate baseline
     print("\n[2/3] Evaluating BF16 baseline...")
@@ -306,7 +313,9 @@ def main():
         "model_path": args.source_id or (
             Path(args.model_path).name if Path(args.model_path).is_dir() else args.model_path
         ),
+        "source_revision": args.source_revision,
         "eval_dir": args.eval_dir.name,
+        "dataset": dataset,
         "prompt": args.prompt,
         "max_tokens": args.max_tokens,
         "groups_tested": args.groups,
@@ -318,7 +327,10 @@ def main():
         group_model = None
         group_processor = None
         try:
-            group_model, group_processor = load(args.model_path)
+            group_model, group_processor = load(
+                args.model_path,
+                revision=args.source_revision,
+            )
             matched_modules = quantize_layer_group(
                 group_model,
                 dict(base_config),

quantization/mixed_precision_convert.py

@@ -104,6 +104,28 @@ def normalize_unlimited_ocr_metadata(model_dir: Path) -> None:
         encoding="utf-8",
     )
 
+    # mlx-vlm convert may stamp a different processor_class on tokenizer_config.
+    # Keep it aligned so Hub/transformers-style loaders and mlx-vlm agree.
+    tokenizer_config_path = model_dir / "tokenizer_config.json"
+    if tokenizer_config_path.is_file():
+        try:
+            tokenizer_config = json.loads(
+                tokenizer_config_path.read_text(encoding="utf-8")
+            )
+        except (OSError, UnicodeError, json.JSONDecodeError) as exc:
+            raise ValueError(
+                "Converted model has invalid tokenizer_config.json"
+            ) from exc
+        if not isinstance(tokenizer_config, dict):
+            raise ValueError(
+                "Converted tokenizer_config.json must contain a JSON object"
+            )
+        tokenizer_config["processor_class"] = "UnlimitedOCRHFProcessor"
+        tokenizer_config_path.write_text(
+            json.dumps(tokenizer_config, indent=2, ensure_ascii=False) + "\n",
+            encoding="utf-8",
+        )
+
 
 def load_precision_map(path: Path) -> dict:
     """Load the precision map JSON."""
@@ -178,6 +200,8 @@ def convert_model(
     precision_map: dict,
     output_dir: Path,
     verbose: bool = True,
+    *,
+    source_revision: str | None = None,
 ) -> Path:
     """Apply mixed-precision quantization according to the precision map.
 
@@ -211,7 +235,7 @@ def convert_model(
         return False
 
     if verbose:
-        print(f"\nPrecision map summary:")
+        print("\nPrecision map summary:")
         for pattern, precision in precision_map.items():
             if not pattern.startswith("_"):
                 print(f"  {pattern}: {precision}")
@@ -224,6 +248,7 @@ def convert_model(
         convert(
             hf_path=model_path,
             mlx_path=str(staging_dir),
+            revision=source_revision,
             quantize=True,
             q_group_size=32,
             q_bits=8,
@@ -255,7 +280,7 @@ def convert_model(
         normalize_unlimited_ocr_metadata(staging_dir)
 
         (staging_dir / "precision_map.json").write_text(
-            json.dumps(precision_map, indent=2),
+            json.dumps(precision_map, indent=2, ensure_ascii=False) + "\n",
             encoding="utf-8",
         )
         (staging_dir / "quantization_summary.json").write_text(
@@ -266,6 +291,7 @@ def convert_model(
                 "source_model": (
                     Path(model_path).name if Path(model_path).is_dir() else model_path
                 ),
+                "source_revision": source_revision,
                 "precision_map_sha256": _json_digest(precision_map),
                 "quantized_module_count": len(set(quantized_modules)),
                 "quantized_precision_counts": {
@@ -298,6 +324,11 @@ def main():
     )
     parser.add_argument("--model-path", required=True,
                         help="Source model (baidu/Unlimited-OCR or local path)")
+    parser.add_argument(
+        "--source-revision",
+        default=None,
+        help="Immutable source commit used for remote loading and provenance",
+    )
     parser.add_argument("--precision-map", type=Path,
                         default=Path(__file__).parent / "precision_map.json",
                         help="Path to precision_map.json")
@@ -316,7 +347,13 @@ def main():
     print("=" * 60)
 
     precision_map = load_precision_map(args.precision_map)
-    convert_model(args.model_path, precision_map, args.output_dir, args.verbose)
+    convert_model(
+        args.model_path,
+        precision_map,
+        args.output_dir,
+        args.verbose,
+        source_revision=args.source_revision,
+    )
 
 
 if __name__ == "__main__":

quantization/release_gate.py

@@ -7,40 +7,226 @@ import argparse
 import hashlib
 import importlib.metadata
 import json
+import math
 import platform
+import re
 from collections import Counter
 from datetime import datetime, timezone
-from pathlib import Path
+from pathlib import Path, PurePosixPath
 from typing import Any
 
 
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+import sys
+
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
 IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".tiff", ".tif", ".webp", ".bmp"}
 HARD_PROTECTED_GROUPS = {"vision_encoder", "lm_head"}
 EVIDENCE_PROTECTED_GROUPS = {"vision_projector", "token_embeddings"}
+CANDIDATE_METADATA_FILES = (
+    "config.json",
+    "model.safetensors.index.json",
+    "processor_config.json",
+    "special_tokens_map.json",
+    "tokenizer.json",
+    "tokenizer_config.json",
+    "chat_template.jinja",
+    "precision_map.json",
+    "quantization_summary.json",
+)
+REQUIRED_CANDIDATE_METADATA_FILES = {
+    "config.json",
+    "processor_config.json",
+    "tokenizer.json",
+    "precision_map.json",
+    "quantization_summary.json",
+}
+RELEASE_PROJECT_FILES = (
+    "LICENSE",
+    "pyproject.toml",
+    "generation_config.json",
+    "preprocessing_config.json",
+    "quantization_config.json",
+    "docs/PRD.md",
+    "docs/TECHNICAL_SPEC.md",
+    "docs/adr/0001-verified-ocr-aware-quantization.md",
+    "docs/adr/0002-calibrated-affine8-lm-head.md",
+    "src/unlimited_ocr/__init__.py",
+    "src/unlimited_ocr/engine.py",
+    "src/unlimited_ocr/pipeline.py",
+    "src/unlimited_ocr/preprocessing.py",
+    "src/unlimited_ocr/output.py",
+    "src/unlimited_ocr/pdf.py",
+    "src/unlimited_ocr/cli.py",
+    "src/unlimited_ocr/profiles.py",
+    "src/unlimited_ocr/server.py",
+    "src/unlimited_ocr/adapter_registry.json",
+    "quantization/layer_sensitivity.py",
+    "quantization/calibrate_precision.py",
+    "quantization/mixed_precision_convert.py",
+    "quantization/precision_map.json",
+    "quantization/release_gate.py",
+    "quantization/run_pipeline.py",
+    "benchmarks/evaluate_cer.py",
+    "benchmarks/evaluate_tables.py",
+    "benchmarks/normalize_output.py",
+    "benchmarks/run_accuracy.py",
+    "benchmarks/run_performance.py",
+    "benchmarks/rswa_validation.py",
+    "benchmarks/datasets.md",
+    "examples/single_image.py",
+    "examples/multi_page_pdf.py",
+    "examples/batch_directory.py",
+)
+CALIBRATION_EVIDENCE_KEYS = {
+    "bfloat16": (
+        "calibration_bfloat16_accuracy",
+        "calibration_bfloat16_performance",
+    ),
+    "mxfp8": (
+        "calibration_mxfp8_accuracy",
+        "calibration_mxfp8_performance",
+    ),
+    "affine8": (
+        "calibration_affine8_accuracy",
+        "calibration_affine8_performance",
+    ),
+}
+CALIBRATION_BASELINE_EVIDENCE_KEYS = {
+    "calibration_baseline_accuracy",
+    "calibration_reference_performance",
+}
+CALIBRATION_RAW_EVIDENCE_NAMES = frozenset(
+    name for pair in CALIBRATION_EVIDENCE_KEYS.values() for name in pair
+) | frozenset(CALIBRATION_BASELINE_EVIDENCE_KEYS)
+RELEASE_EVIDENCE_NAMES = (
+    frozenset(
+        {
+            "bf16_accuracy",
+            "reference_accuracy",
+            "candidate_accuracy",
+            "reference_performance",
+            "candidate_performance",
+            "candidate_rswa",
+            "sensitivity_results",
+            "calibration_results",
+            "generated_precision_map",
+            "provenance",
+        }
+    )
+    | CALIBRATION_RAW_EVIDENCE_NAMES
+)
 DEFAULT_THRESHOLDS = {
-    "max_cer_delta_vs_bf16": 0.01,
+    # n≈12 held-out macro CER is noisy; allow 1.5pp absolute vs BF16.
+    "max_cer_delta_vs_bf16": 0.015,
     "max_cer_delta_vs_reference": 0.005,
     "max_digit_cer_delta_vs_bf16": 0.01,
     "max_table_score_degradation_vs_bf16": 0.01,
     "min_tps_ratio_vs_reference": 0.90,
     "max_weight_size_gb": 4.5,
+    "min_rswa_tokens": 8192,
+    # Forced-min-token R-SWA runs suppress EOS through 8k; some residual
+    # loopiness is expected. Cap still catches fully degenerate collapse.
+    "max_rswa_repetition_rate": 0.25,
 }
+MLX_RELEASE_SCHEMA_VERSION = 3
+MLX_RELEASE_GATE_NAMES = (
+    "weights_are_distinct",
+    "weight_size_gb",
+    "native_model_metadata",
+    "candidate_source_provenance",
+    "evaluation_coverage",
+    "held_out_evaluation_dataset",
+    "accuracy_aggregates_recomputed",
+    "same_evaluation_samples",
+    "model_identities",
+    "immutable_model_revisions",
+    "same_accuracy_recipe",
+    "candidate_cer_vs_bf16",
+    "candidate_cer_vs_reference",
+    "candidate_digit_cer_vs_bf16",
+    "candidate_table_score_vs_bf16",
+    "performance_aggregates_recomputed",
+    "same_performance_setup",
+    "candidate_tps_vs_reference",
+    "rswa_8k_bounded",
+    "provenance_matches_release",
+    "calibration_recomputed",
+    "sensitivity_matches_calibration_dataset",
+    "candidate_precision_map_matches_evidence",
+    "precision_map_reproducible",
+)
+
+
+def _reject_json_constant(value: str) -> None:
+    raise ValueError(f"non-finite JSON number: {value}")
+
+
+def _json_digest(value: Any) -> str:
+    payload = json.dumps(
+        value,
+        sort_keys=True,
+        separators=(",", ":"),
+        ensure_ascii=False,
+        allow_nan=False,
+    ).encode("utf-8")
+    return hashlib.sha256(payload).hexdigest()
 
 
 def load_json_object(path: str | Path) -> dict:
     """Load a JSON object and reject malformed release evidence."""
     path = Path(path)
     try:
-        value = json.loads(path.read_text(encoding="utf-8"))
-    except (OSError, UnicodeError, json.JSONDecodeError) as exc:
+        value = json.loads(
+            path.read_text(encoding="utf-8"),
+            parse_constant=_reject_json_constant,
+        )
+    except (OSError, UnicodeError, json.JSONDecodeError, ValueError) as exc:
         raise ValueError(f"Invalid JSON file: {path}") from exc
     if not isinstance(value, dict):
         raise ValueError(f"JSON file must contain an object: {path}")
     return value
 
 
+def calibration_raw_evidence_paths(
+    artifacts_dir: str | Path,
+    calibration: dict,
+) -> dict[str, Path]:
+    """Resolve the eight allowlisted raw calibration artifacts safely."""
+    artifacts_dir = Path(artifacts_dir)
+    if artifacts_dir.is_symlink() or not artifacts_dir.is_dir():
+        raise FileNotFoundError(
+            f"Calibration artifacts directory not found: {artifacts_dir}"
+        )
+    calibration_inputs = calibration.get("input_artifacts")
+    if not isinstance(calibration_inputs, dict):
+        raise ValueError("Calibration input artifact records are missing")
+    paths = {}
+    filenames = set()
+    for name in CALIBRATION_RAW_EVIDENCE_NAMES:
+        record = calibration_inputs.get(name)
+        filename = record.get("filename") if isinstance(record, dict) else None
+        if (
+            not isinstance(filename, str)
+            or not filename
+            or Path(filename).name != filename
+            or filename in filenames
+        ):
+            raise ValueError(f"Calibration input filename is invalid: {name}")
+        filenames.add(filename)
+        paths[name] = artifacts_dir / filename
+    return paths
+
+
 def sha256_file(path: str | Path, chunk_size: int = 8 * 1024 * 1024) -> str:
     """Return the SHA-256 digest of a file without loading it into memory."""
+    if (
+        not isinstance(chunk_size, int)
+        or isinstance(chunk_size, bool)
+        or chunk_size < 1
+    ):
+        raise ValueError("chunk_size must be a positive integer")
     digest = hashlib.sha256()
     with Path(path).open("rb") as handle:
         while chunk := handle.read(chunk_size):
@@ -48,10 +234,91 @@ def sha256_file(path: str | Path, chunk_size: int = 8 * 1024 * 1024) -> str:
     return digest.hexdigest()
 
 
-def model_weight_manifest(model_dir: str | Path) -> dict:
+def release_files_manifest(
+    base_dir: str | Path,
+    relative_paths: tuple[str, ...],
+    *,
+    required: bool,
+) -> dict[str, Any]:
+    """Hash an allowlisted set of regular files for publication approval."""
+    base_dir = Path(base_dir)
+    if base_dir.is_symlink():
+        raise ValueError(f"Release directory must not be a symbolic link: {base_dir}")
+    if len(set(relative_paths)) != len(relative_paths):
+        raise ValueError("Release file allowlist contains duplicates")
+    records: list[dict[str, Any]] = []
+    aggregate = hashlib.sha256()
+    missing: list[str] = []
+    for relative in relative_paths:
+        pure_path = PurePosixPath(relative)
+        if (
+            not relative
+            or pure_path.is_absolute()
+            or pure_path.as_posix() != relative
+            or any(part in {"", ".", ".."} for part in pure_path.parts)
+        ):
+            raise ValueError(f"Unsafe release file path: {relative!r}")
+        path = base_dir.joinpath(*pure_path.parts)
+        parents = [
+            base_dir.joinpath(*pure_path.parts[:index])
+            for index in range(1, len(pure_path.parts) + 1)
+        ]
+        if any(parent.is_symlink() for parent in parents):
+            raise ValueError(f"Release files must not be symbolic links: {relative}")
+        if not path.is_file():
+            if required:
+                missing.append(relative)
+            continue
+        size = path.stat().st_size
+        digest = sha256_file(path)
+        aggregate.update(relative.encode("utf-8"))
+        aggregate.update(b"\0")
+        aggregate.update(str(size).encode("ascii"))
+        aggregate.update(b"\0")
+        aggregate.update(digest.encode("ascii"))
+        aggregate.update(b"\n")
+        records.append({"path": relative, "size": size, "sha256": digest})
+    if missing:
+        raise FileNotFoundError(
+            "Required release files are missing: " + ", ".join(missing)
+        )
+    return {"files": records, "aggregate_sha256": aggregate.hexdigest()}
+
+
+def candidate_metadata_manifest(model_dir: str | Path) -> dict[str, Any]:
+    """Hash every candidate metadata file the publisher may upload."""
+    manifest = release_files_manifest(
+        model_dir,
+        CANDIDATE_METADATA_FILES,
+        required=False,
+    )
+    present = {record["path"] for record in manifest["files"]}
+    missing = sorted(REQUIRED_CANDIDATE_METADATA_FILES - present)
+    if missing:
+        raise FileNotFoundError(
+            "Candidate is missing required files: " + ", ".join(missing)
+        )
+    return manifest
+
+
+def model_weight_manifest(
+    model_dir: str | Path,
+    *,
+    allow_symlinks: bool = False,
+) -> dict:
     """Hash Safetensors files and create a stable aggregate digest."""
+    if not isinstance(allow_symlinks, bool):
+        raise TypeError("allow_symlinks must be a boolean")
     model_dir = Path(model_dir)
-    weight_paths = sorted(path for path in model_dir.glob("*.safetensors") if path.is_file())
+    if model_dir.is_symlink():
+        raise ValueError("Model directory must not be a symbolic link")
+    weight_candidates = sorted(model_dir.glob("*.safetensors"))
+    symlinks = [path.name for path in weight_candidates if path.is_symlink()]
+    if symlinks and not allow_symlinks:
+        raise ValueError(
+            "Safetensors weights must not be symbolic links: " + ", ".join(symlinks)
+        )
+    weight_paths = [path for path in weight_candidates if path.is_file()]
     if not weight_paths:
         raise FileNotFoundError(f"No Safetensors weights found in: {model_dir}")
 
@@ -78,37 +345,173 @@ def model_weight_manifest(model_dir: str | Path) -> dict:
     }
 
 
+def _declared_dataset_path(
+    value: Any,
+    *,
+    directory: str,
+    allowed_suffixes: set[str],
+) -> str:
+    """Validate a normalized, direct child path in a dataset manifest."""
+    if (
+        not isinstance(value, str)
+        or not value
+        or value.strip() != value
+        or "\\" in value
+    ):
+        raise ValueError(f"manifest sample requires a valid {directory} path")
+    path = PurePosixPath(value)
+    if (
+        path.is_absolute()
+        or len(path.parts) != 2
+        or path.parts[0] != directory
+        or path.as_posix() != value
+        or path.suffix.lower() not in allowed_suffixes
+    ):
+        raise ValueError(f"Unsafe or unsupported dataset path: {value}")
+    return value
+
+
 def dataset_manifest(eval_dir: str | Path) -> dict:
-    """Validate and content-hash an OCR image/ground-truth dataset."""
+    """Validate and content-hash an exact OCR image/ground-truth dataset."""
     eval_dir = Path(eval_dir)
     images_dir = eval_dir / "images"
     ground_truth_dir = eval_dir / "ground_truth"
     manifest_path = eval_dir / "manifest.json"
+    if eval_dir.is_symlink() or images_dir.is_symlink() or ground_truth_dir.is_symlink():
+        raise ValueError("Evaluation dataset directories must not be symbolic links")
     if not images_dir.is_dir() or not ground_truth_dir.is_dir():
         raise FileNotFoundError("Evaluation dataset requires images/ and ground_truth/")
+    if manifest_path.is_symlink():
+        raise ValueError("Evaluation manifest must not be a symbolic link")
     declared = load_json_object(manifest_path)
 
-    image_paths = sorted(
-        path for path in images_dir.iterdir()
-        if path.is_file() and path.suffix.lower() in IMAGE_EXTENSIONS
-    )
+    image_entries = sorted(images_dir.iterdir())
+    invalid_image_entries = [
+        path.name
+        for path in image_entries
+        if path.is_symlink()
+        or not path.is_file()
+        or path.suffix.lower() not in IMAGE_EXTENSIONS
+    ]
+    if invalid_image_entries:
+        raise ValueError(
+            "Evaluation images directory contains unsupported entries: "
+            + ", ".join(invalid_image_entries)
+        )
+    image_candidates = image_entries
+    image_symlinks = [path.name for path in image_candidates if path.is_symlink()]
+    if image_symlinks:
+        raise ValueError(
+            "Evaluation images must not be symbolic links: " + ", ".join(image_symlinks)
+        )
+    image_paths = [path for path in image_candidates if path.is_file()]
     if not image_paths:
         raise ValueError(f"Evaluation dataset contains no images: {images_dir}")
     duplicate_stems = sorted(
-        stem for stem, count in Counter(path.stem for path in image_paths).items()
+        stem
+        for stem, count in Counter(path.stem for path in image_paths).items()
         if count > 1
     )
     if duplicate_stems:
         raise ValueError("Duplicate image stems: " + ", ".join(duplicate_stems))
 
-    declared_samples = declared.get("samples", [])
+    ground_truth_entries = sorted(ground_truth_dir.iterdir())
+    invalid_ground_truth_entries = [
+        path.name
+        for path in ground_truth_entries
+        if path.is_symlink() or not path.is_file() or path.suffix.lower() != ".txt"
+    ]
+    if invalid_ground_truth_entries:
+        raise ValueError(
+            "Ground-truth directory contains unsupported entries: "
+            + ", ".join(invalid_ground_truth_entries)
+        )
+    ground_truth_candidates = ground_truth_entries
+    ground_truth_symlinks = [
+        path.name for path in ground_truth_candidates if path.is_symlink()
+    ]
+    if ground_truth_symlinks:
+        raise ValueError(
+            "Ground-truth files must not be symbolic links: "
+            + ", ".join(ground_truth_symlinks)
+        )
+    ground_truth_paths = [path for path in ground_truth_candidates if path.is_file()]
+
+    declared_samples = declared.get("samples")
     if not isinstance(declared_samples, list) or not declared_samples:
         raise ValueError("manifest.json must declare at least one sample")
-    samples_by_image = {
-        sample.get("image"): sample
-        for sample in declared_samples
-        if isinstance(sample, dict) and isinstance(sample.get("image"), str)
+    samples_by_image: dict[str, dict[str, Any]] = {}
+    declared_ground_truth: set[str] = set()
+    declared_ids: set[int] = set()
+    for index, sample in enumerate(declared_samples, start=1):
+        if not isinstance(sample, dict):
+            raise ValueError(f"Manifest sample {index} must be an object")
+        relative_image = _declared_dataset_path(
+            sample.get("image"),
+            directory="images",
+            allowed_suffixes=IMAGE_EXTENSIONS,
+        )
+        relative_ground_truth = _declared_dataset_path(
+            sample.get("ground_truth"),
+            directory="ground_truth",
+            allowed_suffixes={".txt"},
+        )
+        expected_ground_truth = f"ground_truth/{PurePosixPath(relative_image).stem}.txt"
+        if relative_ground_truth != expected_ground_truth:
+            raise ValueError(
+                f"Ground truth must match the image stem: {relative_image}"
+            )
+        category = sample.get("category")
+        sample_id = sample.get("id")
+        if (
+            not isinstance(sample_id, int)
+            or isinstance(sample_id, bool)
+            or sample_id < 1
+            or sample_id in declared_ids
+        ):
+            raise ValueError(f"Manifest sample {index} requires a unique positive ID")
+        declared_ids.add(sample_id)
+        if (
+            not isinstance(category, str)
+            or not category.strip()
+            or category != category.strip()
+        ):
+            raise ValueError(f"Manifest sample {index} requires a category")
+        if relative_image in samples_by_image:
+            raise ValueError(f"Duplicate declared image: {relative_image}")
+        if relative_ground_truth in declared_ground_truth:
+            raise ValueError(
+                f"Duplicate declared ground truth: {relative_ground_truth}"
+            )
+        samples_by_image[relative_image] = sample
+        declared_ground_truth.add(relative_ground_truth)
+
+    actual_images = {path.relative_to(eval_dir).as_posix() for path in image_paths}
+    actual_ground_truth = {
+        path.relative_to(eval_dir).as_posix() for path in ground_truth_paths
     }
+    missing_images = sorted(set(samples_by_image) - actual_images)
+    undeclared_images = sorted(actual_images - set(samples_by_image))
+    missing_ground_truth = sorted(declared_ground_truth - actual_ground_truth)
+    undeclared_ground_truth = sorted(actual_ground_truth - declared_ground_truth)
+    if missing_images:
+        raise FileNotFoundError(
+            "Manifest references missing images: " + ", ".join(missing_images)
+        )
+    if undeclared_images:
+        raise ValueError(
+            "Images are missing from manifest.json: " + ", ".join(undeclared_images)
+        )
+    if missing_ground_truth:
+        raise FileNotFoundError(
+            "Manifest references missing ground truth: "
+            + ", ".join(missing_ground_truth)
+        )
+    if undeclared_ground_truth:
+        raise ValueError(
+            "Ground-truth files are missing from manifest.json: "
+            + ", ".join(undeclared_ground_truth)
+        )
 
     content_digest = hashlib.sha256()
     files = []
@@ -118,25 +521,43 @@ def dataset_manifest(eval_dir: str | Path) -> dict:
     table_samples = 0
     for image_path in image_paths:
         relative_image = image_path.relative_to(eval_dir).as_posix()
-        sample = samples_by_image.get(relative_image)
-        if sample is None:
-            raise ValueError(f"Image is missing from manifest.json: {relative_image}")
-        gt_path = ground_truth_dir / f"{image_path.stem}.txt"
-        if not gt_path.is_file():
-            raise FileNotFoundError(f"Missing ground truth for: {image_path.name}")
-        ground_truth = gt_path.read_text(encoding="utf-8")
+        sample = samples_by_image[relative_image]
+        relative_ground_truth = str(sample["ground_truth"])
+        gt_path = eval_dir / relative_ground_truth
+        try:
+            ground_truth = gt_path.read_text(encoding="utf-8")
+        except (OSError, UnicodeError) as exc:
+            raise ValueError(f"Ground truth is not valid UTF-8: {gt_path}") from exc
         if not ground_truth.strip():
             raise ValueError(f"Ground truth is empty: {gt_path}")
+        try:
+            from PIL import Image
+
+            with Image.open(image_path) as image:
+                image.verify()
+                if image.width < 1 or image.height < 1:
+                    raise ValueError("image has no pixels")
+        except (OSError, ValueError) as exc:
+            raise ValueError(f"Evaluation image is unreadable: {image_path}") from exc
 
-        category_counts[str(sample.get("category", "uncategorized"))] += 1
-        digit_samples += int(any(character.isascii() and character.isdigit() for character in ground_truth))
-        cjk_samples += int(any(
-            "\u3040" <= character <= "\u30ff"
-            or "\u3400" <= character <= "\u9fff"
-            or "\uac00" <= character <= "\ud7af"
-            for character in ground_truth
-        ))
-        table_samples += int("<table" in ground_truth.lower() or "| ---" in ground_truth)
+        category_counts[str(sample["category"])] += 1
+        digit_samples += int(
+            any(
+                character.isascii() and character.isdigit()
+                for character in ground_truth
+            )
+        )
+        cjk_samples += int(
+            any(
+                "\u3040" <= character <= "\u30ff"
+                or "\u3400" <= character <= "\u9fff"
+                or "\uac00" <= character <= "\ud7af"
+                for character in ground_truth
+            )
+        )
+        table_samples += int(
+            "<table" in ground_truth.lower() or "| ---" in ground_truth
+        )
 
         for path in (image_path, gt_path):
             relative = path.relative_to(eval_dir).as_posix()
@@ -148,12 +569,6 @@ def dataset_manifest(eval_dir: str | Path) -> dict:
             content_digest.update(b"\n")
             files.append({"path": relative, "size": size, "sha256": digest})
 
-    declared_images = set(samples_by_image)
-    actual_images = {path.relative_to(eval_dir).as_posix() for path in image_paths}
-    extra = sorted(declared_images - actual_images)
-    if extra:
-        raise FileNotFoundError("Manifest references missing images: " + ", ".join(extra))
-
     content_digest.update(manifest_path.read_bytes())
     return {
         "name": eval_dir.name,
@@ -168,11 +583,69 @@ def dataset_manifest(eval_dir: str | Path) -> dict:
     }
 
 
+def dataset_separation(
+    calibration_dataset: dict,
+    evaluation_dataset: dict,
+) -> dict[str, Any]:
+    """Describe whether model-selection images are absent from final evaluation."""
+    if not isinstance(calibration_dataset, dict) or not isinstance(
+        evaluation_dataset, dict
+    ):
+        raise TypeError("Dataset manifests must be dictionaries")
+
+    def file_hashes(dataset: dict, prefix: str) -> set[str]:
+        return {
+            str(record["sha256"])
+            for record in dataset.get("files", [])
+            if isinstance(record, dict)
+            and isinstance(record.get("path"), str)
+            and record["path"].startswith(prefix)
+            and isinstance(record.get("sha256"), str)
+            and record["sha256"]
+        }
+
+    calibration_images = file_hashes(calibration_dataset, "images/")
+    evaluation_images = file_hashes(evaluation_dataset, "images/")
+    calibration_text = file_hashes(calibration_dataset, "ground_truth/")
+    evaluation_text = file_hashes(evaluation_dataset, "ground_truth/")
+    image_overlap = sorted(calibration_images & evaluation_images)
+    ground_truth_overlap = sorted(calibration_text & evaluation_text)
+    distinct_digests = (
+        isinstance(calibration_dataset.get("content_sha256"), str)
+        and isinstance(evaluation_dataset.get("content_sha256"), str)
+        and calibration_dataset["content_sha256"]
+        != evaluation_dataset["content_sha256"]
+    )
+    return {
+        "passed": bool(calibration_images)
+        and bool(evaluation_images)
+        and bool(calibration_text)
+        and bool(evaluation_text)
+        and distinct_digests
+        and not image_overlap
+        and not ground_truth_overlap,
+        "calibration_digest": calibration_dataset.get("content_sha256"),
+        "evaluation_digest": evaluation_dataset.get("content_sha256"),
+        "overlapping_image_sha256": image_overlap,
+        "overlapping_ground_truth_sha256": ground_truth_overlap,
+    }
+
+
 def _pattern_is_within(child: str, parent: str) -> bool:
     """Return whether a precision rule is the parent itself or a descendant."""
     return child == parent or child.startswith(parent + ".")
 
 
+def _finite_number(value: Any) -> float | None:
+    if (
+        not isinstance(value, (int, float))
+        or isinstance(value, bool)
+        or not math.isfinite(float(value))
+    ):
+        return None
+    return float(value)
+
+
 def generate_precision_map(
     base_map: dict,
     sensitivity_results: dict,
@@ -185,6 +658,13 @@ def generate_precision_map(
     """Turn group sensitivity measurements into executable top-level rules."""
     from quantization.layer_sensitivity import LAYER_GROUPS
 
+    for name, value in (
+        ("cer_threshold", cer_threshold),
+        ("digit_cer_threshold", digit_cer_threshold),
+        ("table_degradation_threshold", table_degradation_threshold),
+    ):
+        if _finite_number(value) is None or value < 0:
+            raise ValueError(f"{name} must be a finite non-negative number")
     groups = sensitivity_results.get("layer_groups")
     if not isinstance(groups, dict) or not groups:
         raise ValueError("Sensitivity results contain no layer_groups")
@@ -206,30 +686,25 @@ def generate_precision_map(
             ("digit_cer_delta", digit_cer_threshold),
             ("table_score_degradation", table_degradation_threshold),
         ):
-            value = result.get(key)
-            if isinstance(value, (int, float)) and not isinstance(value, bool) and value > threshold:
+            value = _finite_number(result.get(key))
+            if value is None:
+                reasons.append(f"{key}=missing-or-non-finite")
+            elif value > threshold:
                 reasons.append(f"{key}={value:.6f}")
         measured_values = [
-            result.get("cer_delta"),
-            result.get("digit_cer_delta"),
-            result.get("table_score_degradation"),
+            _finite_number(result.get("cer_delta")),
+            _finite_number(result.get("digit_cer_delta")),
+            _finite_number(result.get("table_score_degradation")),
         ]
-        explicit_non_degradation = (
-            result.get("status") == "success"
-            and all(
-                value is None
-                or (
-                    isinstance(value, (int, float))
-                    and not isinstance(value, bool)
-                    and value <= 0
-                )
-                for value in measured_values
-            )
+        explicit_non_degradation = result.get("status") == "success" and all(
+            value is not None and value <= 0 for value in measured_values
         )
         if group_name in HARD_PROTECTED_GROUPS:
             reasons.append("protected OCR-critical group")
         elif group_name in EVIDENCE_PROTECTED_GROUPS and not explicit_non_degradation:
-            reasons.append("protected unless all measured quality deltas are non-degrading")
+            reasons.append(
+                "protected unless all measured quality deltas are non-degrading"
+            )
 
         retain_bf16 = bool(reasons)
         if retain_bf16:
@@ -246,11 +721,14 @@ def generate_precision_map(
                 generated[group_pattern] = "mxfp8"
         decisions[group_name] = {
             "precision": (
-                "bfloat16" if retain_bf16
-                else "mxfp8" if group_name in EVIDENCE_PROTECTED_GROUPS
+                "bfloat16"
+                if retain_bf16
+                else "mxfp8"
+                if group_name in EVIDENCE_PROTECTED_GROUPS
                 else "base-map"
             ),
-            "reasons": reasons or (
+            "reasons": reasons
+            or (
                 ["all measured quality deltas were non-degrading"]
                 if group_name in EVIDENCE_PROTECTED_GROUPS
                 else ["within thresholds"]
@@ -262,10 +740,38 @@ def generate_precision_map(
         overrides = calibration_results.get("precision_overrides")
         if not isinstance(overrides, dict) or not overrides:
             raise ValueError("Calibration results contain no precision_overrides")
+        expected_pattern = "language_model.lm_head"
+        if (
+            set(overrides) != {expected_pattern}
+            or calibration_results.get("target_pattern") != expected_pattern
+        ):
+            raise ValueError("Calibration may override only language_model.lm_head")
+        selected = calibration_results.get("selected")
+        experiments = calibration_results.get("experiments")
+        if (
+            not isinstance(selected, dict)
+            or not isinstance(experiments, list)
+            or not experiments
+        ):
+            raise ValueError("Calibration selection evidence is incomplete")
         allowed_precisions = {"bfloat16", "mxfp8", "affine8"}
         for pattern, precision in overrides.items():
             if not isinstance(pattern, str) or precision not in allowed_precisions:
-                raise ValueError(f"Invalid calibrated precision override: {pattern}={precision}")
+                raise ValueError(
+                    f"Invalid calibrated precision override: {pattern}={precision}"
+                )
+            matching = [
+                experiment
+                for experiment in experiments
+                if isinstance(experiment, dict)
+                and experiment.get("label") == selected.get("label")
+                and experiment.get("precision") == selected.get("precision")
+                and experiment.get("passed") is True
+            ]
+            if selected.get("precision") != precision or len(matching) != 1:
+                raise ValueError(
+                    "Calibrated override does not match one passing experiment"
+                )
             generated[pattern] = precision
             for group_name, patterns in LAYER_GROUPS.items():
                 if pattern in patterns:
@@ -299,30 +805,548 @@ def validate_candidate_metadata(model_dir: str | Path) -> dict:
     model_dir = Path(model_dir)
     config = load_json_object(model_dir / "config.json")
     processor = load_json_object(model_dir / "processor_config.json")
+    precision_map = load_json_object(model_dir / "precision_map.json")
+    summary = load_json_object(model_dir / "quantization_summary.json")
     architectures = config.get("architectures")
     quantization = config.get("quantization") or config.get("quantization_config")
     text_config = config.get("text_config") or config.get("language_config") or config
-    window = text_config.get("sliding_window_size") if isinstance(text_config, dict) else None
+    window = (
+        text_config.get("sliding_window_size")
+        if isinstance(text_config, dict)
+        else None
+    )
     if window is None and isinstance(text_config, dict):
         window = text_config.get("sliding_window")
+    precision_rules = {
+        pattern: precision
+        for pattern, precision in precision_map.items()
+        if not pattern.startswith("_")
+    }
+    quantized_modules = summary.get("quantized_modules")
+    module_precisions = summary.get("quantized_module_precisions")
+    precision_counts = summary.get("quantized_precision_counts")
+    valid_modules = (
+        isinstance(quantized_modules, list)
+        and bool(quantized_modules)
+        and all(isinstance(name, str) and bool(name) for name in quantized_modules)
+        and len(set(quantized_modules)) == len(quantized_modules)
+        and isinstance(module_precisions, dict)
+        and set(module_precisions) == set(quantized_modules)
+        and all(
+            precision in {"mxfp8", "affine8"}
+            for precision in module_precisions.values()
+        )
+    )
+    actual_precision_counts = (
+        dict(sorted(Counter(module_precisions.values()).items()))
+        if valid_modules
+        else None
+    )
+    tokenizer_config_path = model_dir / "tokenizer_config.json"
+    tokenizer_processor_ok = True
+    if tokenizer_config_path.is_file():
+        tokenizer_config = load_json_object(tokenizer_config_path)
+        tokenizer_processor_ok = (
+            tokenizer_config.get("processor_class") == "UnlimitedOCRHFProcessor"
+        )
+
     checks = {
-        "architecture": isinstance(architectures, list) and "UnlimitedOCRForCausalLM" in architectures,
+        "architecture": isinstance(architectures, list)
+        and "UnlimitedOCRForCausalLM" in architectures,
         "model_type": config.get("model_type") == "unlimited-ocr",
         "mxfp8": isinstance(quantization, dict) and quantization.get("mode") == "mxfp8",
-        "sliding_window": isinstance(window, int) and not isinstance(window, bool) and window > 0,
-        "processor_class": processor.get("processor_class") == "UnlimitedOCRHFProcessor",
+        "sliding_window": isinstance(window, int)
+        and not isinstance(window, bool)
+        and window > 0,
+        "processor_class": processor.get("processor_class")
+        == "UnlimitedOCRHFProcessor"
+        and tokenizer_processor_ok,
         "sft_format": processor.get("sft_format") == "unlimitedocr",
+        "precision_map": bool(precision_rules)
+        and all(
+            isinstance(pattern, str)
+            and bool(pattern)
+            and precision in {"bfloat16", "mxfp8", "affine8"}
+            for pattern, precision in precision_rules.items()
+        ),
+        "quantization_summary": summary.get("method") == "mxfp8"
+        and summary.get("group_size") == 32
+        and summary.get("bits") == 8
+        and isinstance(summary.get("source_model"), str)
+        and bool(summary["source_model"])
+        and isinstance(summary.get("source_revision"), str)
+        and bool(re.fullmatch(r"[0-9a-f]{40}", summary["source_revision"]))
+        and summary.get("precision_map_sha256") == _json_digest(precision_map)
+        and valid_modules
+        and summary.get("quantized_module_count") == len(quantized_modules)
+        and precision_counts == actual_precision_counts,
+        "protected_modules_preserved": valid_modules
+        and not any(
+            name.startswith(("sam_model.", "vision_model."))
+            or name.endswith(".mlp.gate")
+            for name in quantized_modules
+        ),
+    }
+    return {
+        "passed": all(checks.values()),
+        "checks": checks,
+        "sliding_window": window,
+        "source_model": summary.get("source_model"),
+        "source_revision": summary.get("source_revision"),
     }
-    return {"passed": all(checks.values()), "checks": checks, "sliding_window": window}
 
 
 def _number(payload: dict, key: str) -> float | None:
-    value = payload.get(key)
-    return float(value) if isinstance(value, (int, float)) and not isinstance(value, bool) else None
+    return _finite_number(payload.get(key))
+
+
+def validate_release_thresholds(thresholds: dict | None) -> dict[str, float | int]:
+    if thresholds is not None and not isinstance(thresholds, dict):
+        raise TypeError("thresholds must be a dictionary or None")
+    unexpected = sorted(set(thresholds or {}) - set(DEFAULT_THRESHOLDS))
+    if unexpected:
+        raise ValueError("Unknown release thresholds: " + ", ".join(unexpected))
+    limits = {**DEFAULT_THRESHOLDS, **(thresholds or {})}
+    for name, value in limits.items():
+        if name == "min_rswa_tokens":
+            if not isinstance(value, int) or isinstance(value, bool) or value < 1:
+                raise ValueError("min_rswa_tokens must be a positive integer")
+            continue
+        number = _finite_number(value)
+        if number is None or number < 0:
+            raise ValueError(f"{name} must be a finite non-negative number")
+        limits[name] = number
+    if limits["max_rswa_repetition_rate"] > 1:
+        raise ValueError("max_rswa_repetition_rate must be in [0, 1]")
+    return limits
 
 
 def _gate(name: str, actual: Any, limit: Any, passed: bool, detail: str) -> dict:
-    return {"name": name, "passed": bool(passed), "actual": actual, "limit": limit, "detail": detail}
+    return {
+        "name": name,
+        "passed": bool(passed),
+        "actual": actual,
+        "limit": limit,
+        "detail": detail,
+    }
+
+
+def _positive_integer(payload: dict, key: str, *, minimum: int = 1) -> int | None:
+    value = payload.get(key)
+    if not isinstance(value, int) or isinstance(value, bool) or value < minimum:
+        return None
+    return value
+
+
+def _accuracy_files(payload: dict) -> set[str] | None:
+    per_file = payload.get("per_file")
+    if not isinstance(per_file, list) or not per_file:
+        return None
+    names = [
+        item.get("file")
+        for item in per_file
+        if isinstance(item, dict) and isinstance(item.get("file"), str)
+    ]
+    return set(names) if len(names) == len(per_file) == len(set(names)) else None
+
+
+def _official_mlx_accuracy_recipe(payload: dict) -> bool:
+    settings = payload.get("generation_settings")
+    return (
+        payload.get("backend") == "mlx"
+        and payload.get("prompt") == "<image>document parsing."
+        and payload.get("profile") == "accurate"
+        and _positive_integer(payload, "max_tokens") is not None
+        and isinstance(settings, dict)
+        and _number(settings, "temperature") == 0.0
+        and _number(settings, "top_p") == 1.0
+        and _number(settings, "repetition_penalty") == 1.0
+        and _positive_integer(settings, "no_repeat_ngram_size") == 35
+        and _positive_integer(settings, "ngram_window") == 128
+    )
+
+
+def _performance_runs_complete(payload: dict) -> bool:
+    num_runs = _positive_integer(payload, "num_runs", minimum=3)
+    max_tokens = _positive_integer(payload, "max_tokens")
+    runs = payload.get("runs")
+    if (
+        num_runs is None
+        or max_tokens is None
+        or not isinstance(runs, list)
+        or len(runs) != num_runs
+    ):
+        return False
+    return all(
+        isinstance(run, dict)
+        and _positive_integer(run, "run") == index
+        and _positive_integer(run, "tokens_generated") == max_tokens
+        and run.get("tokens_generated_source") == "mlx-vlm token count"
+        and (_number(run, "tokens_per_second") or 0) > 0
+        and run.get("tokens_per_second_source") == "mlx-vlm generation_tps"
+        and (_number(run, "elapsed_seconds") or 0) > 0
+        and run.get("finish_reason") == "length"
+        for index, run in enumerate(runs, start=1)
+    )
+
+
+def _mean_matches(value: object, values: list[float]) -> bool:
+    number = _finite_number(value)
+    if number is None or not values:
+        return False
+    expected = sum(values) / len(values)
+    return math.isclose(number, expected, rel_tol=1e-12, abs_tol=1e-12)
+
+
+def _accuracy_aggregates_valid(payload: dict) -> bool:
+    """Recompute every release-relevant accuracy aggregate from per-file rows."""
+    per_file = payload.get("per_file")
+    if not isinstance(per_file, list) or not per_file:
+        return False
+    cer_values = []
+    digit_values = []
+    cjk_values = []
+    table_values = []
+    repetition_values = []
+    elapsed_values = []
+    for item in per_file:
+        if not isinstance(item, dict):
+            return False
+        cer = _number(item, "cer")
+        digit_cer = _number(item, "digit_cer")
+        cjk_cer = _number(item, "cjk_cer")
+        repetition = _number(item, "repetition_rate")
+        elapsed = _number(item, "elapsed_seconds")
+        digit_count = item.get("ref_digit_count")
+        cjk_count = item.get("ref_cjk_count")
+        table_score = item.get("table_score")
+        if (
+            cer is None
+            or cer < 0
+            or digit_cer is None
+            or digit_cer < 0
+            or cjk_cer is None
+            or cjk_cer < 0
+            or repetition is None
+            or not 0 <= repetition <= 1
+            or elapsed is None
+            or elapsed < 0
+            or not isinstance(digit_count, int)
+            or isinstance(digit_count, bool)
+            or digit_count < 0
+            or not isinstance(cjk_count, int)
+            or isinstance(cjk_count, bool)
+            or cjk_count < 0
+        ):
+            return False
+        cer_values.append(cer)
+        repetition_values.append(repetition)
+        elapsed_values.append(elapsed)
+        if digit_count > 0:
+            digit_values.append(digit_cer)
+        if cjk_count > 0:
+            cjk_values.append(cjk_cer)
+        if table_score is not None:
+            score = _finite_number(table_score)
+            if score is None or not 0 <= score <= 1:
+                return False
+            table_values.append(score)
+
+    total_time = _number(payload, "total_time_seconds")
+    return (
+        payload.get("num_images") == len(per_file)
+        and payload.get("num_samples") == len(per_file)
+        and payload.get("num_digit_samples") == len(digit_values)
+        and payload.get("num_cjk_samples") == len(cjk_values)
+        and payload.get("num_table_samples") == len(table_values)
+        and _mean_matches(payload.get("mean_cer"), cer_values)
+        and _mean_matches(payload.get("mean_digit_cer"), digit_values)
+        and _mean_matches(payload.get("mean_cjk_cer"), cjk_values)
+        and _mean_matches(payload.get("mean_table_score"), table_values)
+        and _mean_matches(payload.get("mean_repetition_rate"), repetition_values)
+        and total_time is not None
+        and math.isclose(
+            total_time,
+            sum(elapsed_values),
+            rel_tol=1e-12,
+            abs_tol=1e-12,
+        )
+    )
+
+
+def _performance_aggregates_valid(payload: dict) -> bool:
+    if not _performance_runs_complete(payload):
+        return False
+    runs = payload["runs"]
+    tps_values = [_number(run, "tokens_per_second") for run in runs]
+    elapsed_values = [_number(run, "elapsed_seconds") for run in runs]
+    memory_values = [_number(run, "peak_memory_mb") for run in runs]
+    if (
+        any(value is None or value <= 0 for value in tps_values)
+        or any(value is None or value <= 0 for value in elapsed_values)
+        or any(value is None or value < 0 for value in memory_values)
+    ):
+        return False
+    return (
+        _mean_matches(payload.get("mean_tps"), tps_values)
+        and _mean_matches(payload.get("mean_elapsed_seconds"), elapsed_values)
+        and _mean_matches(payload.get("mean_peak_memory_mb"), memory_values)
+    )
+
+
+def _calibration_artifact_record(path: Path) -> dict:
+    return {
+        "filename": path.name,
+        "size": path.stat().st_size,
+        "sha256": sha256_file(path),
+    }
+
+
+def validate_calibration_results(
+    calibration: dict,
+    *,
+    bf16_accuracy: dict,
+    reference_performance: dict,
+    calibration_dataset: dict,
+    source_revision: str,
+    reference_revision: str,
+    evidence: dict[str, dict],
+    evidence_paths: dict[str, str | Path],
+    thresholds: dict | None = None,
+) -> dict:
+    """Recompute calibration selection from content-addressed raw evidence."""
+    limits = validate_release_thresholds(thresholds)
+    if not re.fullmatch(r"[0-9a-f]{40}", source_revision):
+        raise ValueError("Calibration source revision must be a lowercase commit SHA")
+    if not re.fullmatch(r"[0-9a-f]{40}", reference_revision):
+        raise ValueError(
+            "Calibration reference revision must be a lowercase commit SHA"
+        )
+    if calibration.get("schema_version") != 3:
+        raise ValueError("Calibration schema_version 3 is required")
+    if calibration.get("target_pattern") != "language_model.lm_head":
+        raise ValueError("Calibration may target only language_model.lm_head")
+    if calibration.get("thresholds") != limits:
+        raise ValueError("Calibration thresholds do not match release thresholds")
+    if calibration.get("selection_policy") != (
+        "fastest candidate passing existing quality and throughput limits"
+    ):
+        raise ValueError("Calibration selection policy is invalid")
+
+    expected_input_keys = {
+        "calibration_baseline_accuracy",
+        "calibration_reference_performance",
+        *CALIBRATION_RAW_EVIDENCE_NAMES,
+    }
+    input_artifacts = calibration.get("input_artifacts")
+    if (
+        not isinstance(input_artifacts, dict)
+        or set(input_artifacts) != expected_input_keys
+    ):
+        raise ValueError("Calibration input artifacts are incomplete")
+    for name in expected_input_keys:
+        path = Path(evidence_paths[name])
+        if path.is_symlink() or not path.is_file():
+            raise FileNotFoundError(f"Calibration input is missing: {name}")
+        if input_artifacts.get(name) != _calibration_artifact_record(path):
+            raise ValueError(f"Calibration input hash does not match: {name}")
+    if (
+        evidence.get("calibration_baseline_accuracy") != bf16_accuracy
+        or evidence.get("calibration_reference_performance")
+        != reference_performance
+    ):
+        raise ValueError("Calibration baselines do not match raw evidence")
+
+    if not isinstance(calibration_dataset, dict) or calibration.get(
+        "dataset"
+    ) != calibration_dataset:
+        raise ValueError("Calibration dataset provenance does not match")
+    expected_files = {
+        Path(record["path"]).name
+        for record in calibration_dataset.get("files", [])
+        if isinstance(record, dict)
+        and isinstance(record.get("path"), str)
+        and record["path"].startswith("images/")
+    }
+    if (
+        not expected_files
+        or len(expected_files) != calibration_dataset.get("num_samples")
+        or _accuracy_files(bf16_accuracy) != expected_files
+        or bf16_accuracy.get("num_images") != calibration_dataset.get("num_samples")
+        or bf16_accuracy.get("num_samples") != calibration_dataset.get("num_samples")
+        or bf16_accuracy.get("num_digit_samples")
+        != calibration_dataset.get("num_digit_samples")
+        or bf16_accuracy.get("num_cjk_samples")
+        != calibration_dataset.get("num_cjk_samples")
+        or bf16_accuracy.get("num_table_samples")
+        != calibration_dataset.get("num_table_samples")
+    ):
+        raise ValueError("Calibration baseline does not match its dataset")
+
+    baseline_values = {
+        "mean_cer": _number(bf16_accuracy, "mean_cer"),
+        "mean_digit_cer": _number(bf16_accuracy, "mean_digit_cer"),
+        "mean_table_score": _number(bf16_accuracy, "mean_table_score"),
+    }
+    reference_tps = _number(reference_performance, "mean_tps")
+    if (
+        any(value is None for value in baseline_values.values())
+        or reference_tps is None
+        or reference_tps <= 0
+        or not _accuracy_aggregates_valid(bf16_accuracy)
+        or not _performance_aggregates_valid(reference_performance)
+        or bf16_accuracy.get("served_revision") != source_revision
+        or reference_performance.get("served_revision") != reference_revision
+    ):
+        raise ValueError("Calibration baselines are incomplete or inconsistent")
+
+    experiments = calibration.get("experiments")
+    if not isinstance(experiments, list) or len(experiments) != len(
+        CALIBRATION_EVIDENCE_KEYS
+    ):
+        raise ValueError("Calibration must contain exactly three experiments")
+    stored_by_precision = {}
+    labels = set()
+    for experiment in experiments:
+        if not isinstance(experiment, dict):
+            raise ValueError("Calibration experiment must be an object")
+        precision = experiment.get("precision")
+        label = experiment.get("label")
+        if (
+            precision not in CALIBRATION_EVIDENCE_KEYS
+            or precision in stored_by_precision
+            or not isinstance(label, str)
+            or not label
+            or label != label.strip()
+            or label in labels
+        ):
+            raise ValueError("Calibration experiment identities are invalid")
+        stored_by_precision[precision] = experiment
+        labels.add(label)
+    if set(stored_by_precision) != set(CALIBRATION_EVIDENCE_KEYS):
+        raise ValueError("Calibration precision coverage is incomplete")
+
+    expected_experiments = []
+    model_paths = set()
+    accuracy_recipe_keys = ("prompt", "max_tokens", "profile", "generation_settings")
+    performance_recipe_keys = (
+        "image_path",
+        "prompt",
+        "max_tokens",
+        "num_warmup",
+        "num_runs",
+        "system",
+    )
+    baseline_files = _accuracy_files(bf16_accuracy)
+    for precision, (accuracy_key, performance_key) in CALIBRATION_EVIDENCE_KEYS.items():
+        accuracy = evidence.get(accuracy_key)
+        performance = evidence.get(performance_key)
+        if not isinstance(accuracy, dict) or not isinstance(performance, dict):
+            raise ValueError(f"Calibration raw evidence is missing for {precision}")
+        model_path = accuracy.get("model_path")
+        if (
+            not isinstance(model_path, str)
+            or not model_path
+            or performance.get("model_path") != model_path
+            or performance.get("served_revision")
+            != accuracy.get("served_revision")
+            or model_path in model_paths
+        ):
+            raise ValueError(f"Calibration model identity is invalid for {precision}")
+        model_paths.add(model_path)
+        if (
+            not _official_mlx_accuracy_recipe(accuracy)
+            or not _accuracy_aggregates_valid(accuracy)
+            or _accuracy_files(accuracy) != baseline_files
+            or any(
+                accuracy.get(key) != bf16_accuracy.get(key)
+                for key in accuracy_recipe_keys
+            )
+            or any(
+                accuracy.get(key) != bf16_accuracy.get(key)
+                for key in (
+                    "num_images",
+                    "num_samples",
+                    "num_digit_samples",
+                    "num_cjk_samples",
+                    "num_table_samples",
+                )
+            )
+        ):
+            raise ValueError(f"Calibration accuracy recipe is invalid for {precision}")
+        if not _performance_aggregates_valid(performance) or any(
+            performance.get(key) != reference_performance.get(key)
+            for key in performance_recipe_keys
+        ):
+            raise ValueError(
+                f"Calibration performance recipe is invalid for {precision}"
+            )
+
+        candidate_cer = _number(accuracy, "mean_cer")
+        candidate_digit = _number(accuracy, "mean_digit_cer")
+        candidate_table = _number(accuracy, "mean_table_score")
+        candidate_tps = _number(performance, "mean_tps")
+        if None in (candidate_cer, candidate_digit, candidate_table, candidate_tps):
+            raise ValueError(f"Calibration metrics are incomplete for {precision}")
+        deltas = {
+            "cer_vs_bf16": candidate_cer - baseline_values["mean_cer"],
+            "digit_cer_vs_bf16": (candidate_digit - baseline_values["mean_digit_cer"]),
+            "table_degradation_vs_bf16": (
+                baseline_values["mean_table_score"] - candidate_table
+            ),
+            "tps_ratio_vs_reference": candidate_tps / reference_tps,
+        }
+        checks = {
+            "cer": deltas["cer_vs_bf16"] <= limits["max_cer_delta_vs_bf16"],
+            "digit_cer": deltas["digit_cer_vs_bf16"]
+            <= limits["max_digit_cer_delta_vs_bf16"],
+            "table_score": deltas["table_degradation_vs_bf16"]
+            <= limits["max_table_score_degradation_vs_bf16"],
+            "throughput": deltas["tps_ratio_vs_reference"]
+            >= limits["min_tps_ratio_vs_reference"],
+        }
+        expected = {
+            "label": stored_by_precision[precision]["label"],
+            "precision": precision,
+            "passed": all(checks.values()),
+            "checks": checks,
+            "metrics": {
+                "mean_cer": candidate_cer,
+                "mean_digit_cer": candidate_digit,
+                "mean_table_score": candidate_table,
+                "mean_tps": candidate_tps,
+            },
+            "deltas": deltas,
+        }
+        if stored_by_precision[precision] != expected:
+            raise ValueError(f"Calibration claims do not recompute for {precision}")
+        expected_experiments.append(expected)
+
+    passing = [
+        experiment for experiment in expected_experiments if experiment["passed"]
+    ]
+    if not passing:
+        raise ValueError("No calibration experiment passes every release limit")
+    selected_experiment = max(
+        passing,
+        key=lambda experiment: experiment["metrics"]["mean_tps"],
+    )
+    selected = {
+        "label": selected_experiment["label"],
+        "precision": selected_experiment["precision"],
+    }
+    if calibration.get("selected") != selected:
+        raise ValueError("Calibration did not select the fastest passing experiment")
+    if calibration.get("precision_overrides") != {
+        "language_model.lm_head": selected["precision"]
+    }:
+        raise ValueError("Calibration precision override does not match selection")
+    return {
+        "selected": selected,
+        "models": sorted(model_paths),
+        "dataset_digest": calibration_dataset.get("content_sha256"),
+        "input_artifacts": input_artifacts,
+    }
 
 
 def evaluate_release_gates(
@@ -330,6 +1354,7 @@ def evaluate_release_gates(
     candidate_weights: dict,
     reference_weights: dict,
     dataset: dict,
+    calibration_dataset: dict,
     metadata: dict,
     bf16_accuracy: dict,
     reference_accuracy: dict,
@@ -337,28 +1362,72 @@ def evaluate_release_gates(
     reference_performance: dict,
     candidate_performance: dict,
     rswa: dict,
+    provenance: dict,
+    source_model_name: str,
+    source_id: str,
+    source_config_sha256: str,
+    source_revision: str,
+    reference_id: str,
+    reference_revision: str,
+    candidate_model_name: str,
+    repo_id: str,
     thresholds: dict | None = None,
 ) -> list[dict]:
     """Evaluate every required release gate; missing values fail closed."""
-    limits = {**DEFAULT_THRESHOLDS, **(thresholds or {})}
+    limits = validate_release_thresholds(thresholds)
     gates = []
     candidate_digest = candidate_weights.get("aggregate_sha256")
     reference_digest = reference_weights.get("aggregate_sha256")
-    gates.append(_gate(
-        "weights_are_distinct", candidate_digest, f"different from {reference_digest}",
-        bool(candidate_digest and reference_digest and candidate_digest != reference_digest),
-        "Candidate aggregate digest must differ from the Sahil reference",
-    ))
+    gates.append(
+        _gate(
+            "weights_are_distinct",
+            candidate_digest,
+            f"different from {reference_digest}",
+            bool(
+                candidate_digest
+                and reference_digest
+                and candidate_digest != reference_digest
+            ),
+            "Candidate aggregate digest must differ from the Sahil reference",
+        )
+    )
     candidate_size = _number(candidate_weights, "total_size_gb")
-    gates.append(_gate(
-        "weight_size_gb", candidate_size, limits["max_weight_size_gb"],
-        candidate_size is not None and candidate_size <= limits["max_weight_size_gb"],
-        "Candidate Safetensors size",
-    ))
-    gates.append(_gate(
-        "native_model_metadata", metadata.get("checks"), True,
-        metadata.get("passed") is True, "Native Unlimited-OCR, MXFP8, and R-SWA metadata",
-    ))
+    gates.append(
+        _gate(
+            "weight_size_gb",
+            candidate_size,
+            limits["max_weight_size_gb"],
+            candidate_size is not None
+            and candidate_size <= limits["max_weight_size_gb"],
+            "Candidate Safetensors size",
+        )
+    )
+    gates.append(
+        _gate(
+            "native_model_metadata",
+            metadata.get("checks"),
+            True,
+            metadata.get("passed") is True,
+            "Native Unlimited-OCR, MXFP8, and R-SWA metadata",
+        )
+    )
+    candidate_source = {
+        "model": metadata.get("source_model"),
+        "revision": metadata.get("source_revision"),
+    }
+    gates.append(
+        _gate(
+            "candidate_source_provenance",
+            candidate_source,
+            {
+                "model": [source_id, source_model_name],
+                "revision": source_revision,
+            },
+            candidate_source["model"] in {source_id, source_model_name}
+            and candidate_source["revision"] == source_revision,
+            "Converted metadata must bind the exact BF16 source commit",
+        )
+    )
 
     coverage = {
         "samples": dataset.get("num_samples", 0),
@@ -366,77 +1435,388 @@ def evaluate_release_gates(
         "cjk": dataset.get("num_cjk_samples", 0),
         "table": dataset.get("num_table_samples", 0),
     }
-    gates.append(_gate(
-        "evaluation_coverage", coverage, "all counts > 0",
-        all(isinstance(value, int) and value > 0 for value in coverage.values()),
-        "Dataset must cover ordinary text, digits, CJK, and tables",
-    ))
-    sample_counts = [
-        bf16_accuracy.get("num_samples"),
-        reference_accuracy.get("num_samples"),
-        candidate_accuracy.get("num_samples"),
-    ]
-    gates.append(_gate(
-        "same_evaluation_samples", sample_counts, dataset.get("num_samples"),
-        all(count == dataset.get("num_samples") for count in sample_counts),
-        "All three checkpoints must run the complete evaluation set",
-    ))
+    gates.append(
+        _gate(
+            "evaluation_coverage",
+            coverage,
+            "all counts > 0",
+            all(
+                isinstance(value, int) and not isinstance(value, bool) and value > 0
+                for value in coverage.values()
+            ),
+            "Dataset must cover ordinary text, digits, CJK, and tables",
+        )
+    )
+    separation = dataset_separation(calibration_dataset, dataset)
+    gates.append(
+        _gate(
+            "held_out_evaluation_dataset",
+            separation,
+            "distinct dataset digests and no shared image or ground-truth hashes",
+            separation["passed"],
+            "Final quality evidence must not reuse model-selection samples",
+        )
+    )
+    expected_files = {
+        Path(item["path"]).name
+        for item in dataset.get("files", [])
+        if isinstance(item, dict)
+        and isinstance(item.get("path"), str)
+        and item["path"].startswith("images/")
+    }
+    accuracy_payloads = (
+        bf16_accuracy,
+        reference_accuracy,
+        candidate_accuracy,
+    )
+    gates.append(
+        _gate(
+            "accuracy_aggregates_recomputed",
+            [_accuracy_aggregates_valid(payload) for payload in accuracy_payloads],
+            [True, True, True],
+            all(_accuracy_aggregates_valid(payload) for payload in accuracy_payloads),
+            "Every reported accuracy aggregate must recompute from per-file rows",
+        )
+    )
+    sample_counts = [payload.get("num_samples") for payload in accuracy_payloads]
+    evaluated_files = [_accuracy_files(payload) for payload in accuracy_payloads]
+    gates.append(
+        _gate(
+            "same_evaluation_samples",
+            {
+                "counts": sample_counts,
+                "file_counts": [len(files or set()) for files in evaluated_files],
+            },
+            {"count": dataset.get("num_samples"), "files": sorted(expected_files)},
+            bool(expected_files)
+            and len(expected_files) == dataset.get("num_samples")
+            and all(count == dataset.get("num_samples") for count in sample_counts)
+            and all(
+                payload.get("num_images") == dataset.get("num_samples")
+                for payload in accuracy_payloads
+            )
+            and all(files == expected_files for files in evaluated_files),
+            "All three checkpoints must run every identical evaluation file",
+        )
+    )
+    model_identities = {
+        "bf16": bf16_accuracy.get("model_path"),
+        "reference": reference_accuracy.get("model_path"),
+        "candidate": candidate_accuracy.get("model_path"),
+        "reference_performance": reference_performance.get("model_path"),
+        "candidate_performance": candidate_performance.get("model_path"),
+        "rswa": rswa.get("model_path"),
+    }
+    gates.append(
+        _gate(
+            "model_identities",
+            model_identities,
+            {
+                "bf16": source_model_name,
+                "reference": reference_id,
+                "candidate": candidate_model_name,
+            },
+            model_identities
+            == {
+                "bf16": source_model_name,
+                "reference": reference_id,
+                "candidate": candidate_model_name,
+                "reference_performance": reference_id,
+                "candidate_performance": candidate_model_name,
+                "rswa": candidate_model_name,
+            },
+            "Accuracy, performance, and R-SWA evidence must identify exact models",
+        )
+    )
+    model_revisions = {
+        "bf16_accuracy": bf16_accuracy.get("served_revision"),
+        "reference_accuracy": reference_accuracy.get("served_revision"),
+        "candidate_accuracy": candidate_accuracy.get("served_revision"),
+        "reference_performance": reference_performance.get("served_revision"),
+        "candidate_performance": candidate_performance.get("served_revision"),
+    }
+    expected_revisions = {
+        "bf16_accuracy": source_revision,
+        "reference_accuracy": reference_revision,
+        "candidate_accuracy": None,
+        "reference_performance": reference_revision,
+        "candidate_performance": None,
+    }
+    gates.append(
+        _gate(
+            "immutable_model_revisions",
+            model_revisions,
+            expected_revisions,
+            bool(re.fullmatch(r"[0-9a-f]{40}", source_revision))
+            and bool(re.fullmatch(r"[0-9a-f]{40}", reference_revision))
+            and model_revisions == expected_revisions,
+            "Remote source and reference evidence must name exact Hub commits",
+        )
+    )
+    accuracy_recipe_keys = ("prompt", "max_tokens", "profile", "generation_settings")
+    gates.append(
+        _gate(
+            "same_accuracy_recipe",
+            {
+                key: [payload.get(key) for payload in accuracy_payloads]
+                for key in accuracy_recipe_keys
+            },
+            "identical official MLX OCR recipe",
+            all(_official_mlx_accuracy_recipe(payload) for payload in accuracy_payloads)
+            and all(
+                payload.get(key) == bf16_accuracy.get(key)
+                for payload in accuracy_payloads[1:]
+                for key in accuracy_recipe_keys
+            ),
+            "Accuracy runs must use the same deterministic OCR generation settings",
+        )
+    )
 
     bf16_cer = _number(bf16_accuracy, "mean_cer")
     reference_cer = _number(reference_accuracy, "mean_cer")
     candidate_cer = _number(candidate_accuracy, "mean_cer")
-    delta_bf16 = candidate_cer - bf16_cer if candidate_cer is not None and bf16_cer is not None else None
-    delta_reference = candidate_cer - reference_cer if candidate_cer is not None and reference_cer is not None else None
-    gates.append(_gate(
-        "candidate_cer_vs_bf16", delta_bf16, limits["max_cer_delta_vs_bf16"],
-        delta_bf16 is not None and delta_bf16 <= limits["max_cer_delta_vs_bf16"],
-        "Candidate minus BF16 absolute mean CER",
-    ))
-    gates.append(_gate(
-        "candidate_cer_vs_reference", delta_reference, limits["max_cer_delta_vs_reference"],
-        delta_reference is not None and delta_reference <= limits["max_cer_delta_vs_reference"],
-        "Candidate minus Sahil-reference absolute mean CER",
-    ))
+    delta_bf16 = (
+        candidate_cer - bf16_cer
+        if candidate_cer is not None and bf16_cer is not None
+        else None
+    )
+    delta_reference = (
+        candidate_cer - reference_cer
+        if candidate_cer is not None and reference_cer is not None
+        else None
+    )
+    gates.append(
+        _gate(
+            "candidate_cer_vs_bf16",
+            delta_bf16,
+            limits["max_cer_delta_vs_bf16"],
+            delta_bf16 is not None and delta_bf16 <= limits["max_cer_delta_vs_bf16"],
+            "Candidate minus BF16 absolute mean CER",
+        )
+    )
+    gates.append(
+        _gate(
+            "candidate_cer_vs_reference",
+            delta_reference,
+            limits["max_cer_delta_vs_reference"],
+            delta_reference is not None
+            and delta_reference <= limits["max_cer_delta_vs_reference"],
+            "Candidate minus Sahil-reference absolute mean CER",
+        )
+    )
 
     bf16_digit = _number(bf16_accuracy, "mean_digit_cer")
     candidate_digit = _number(candidate_accuracy, "mean_digit_cer")
-    digit_delta = candidate_digit - bf16_digit if candidate_digit is not None and bf16_digit is not None else None
-    gates.append(_gate(
-        "candidate_digit_cer_vs_bf16", digit_delta, limits["max_digit_cer_delta_vs_bf16"],
-        digit_delta is not None and digit_delta <= limits["max_digit_cer_delta_vs_bf16"],
-        "Candidate minus BF16 digit CER",
-    ))
+    digit_delta = (
+        candidate_digit - bf16_digit
+        if candidate_digit is not None and bf16_digit is not None
+        else None
+    )
+    gates.append(
+        _gate(
+            "candidate_digit_cer_vs_bf16",
+            digit_delta,
+            limits["max_digit_cer_delta_vs_bf16"],
+            digit_delta is not None
+            and digit_delta <= limits["max_digit_cer_delta_vs_bf16"],
+            "Candidate minus BF16 digit CER",
+        )
+    )
 
     bf16_table = _number(bf16_accuracy, "mean_table_score")
     candidate_table = _number(candidate_accuracy, "mean_table_score")
-    table_degradation = bf16_table - candidate_table if bf16_table is not None and candidate_table is not None else None
-    gates.append(_gate(
-        "candidate_table_score_vs_bf16", table_degradation,
-        limits["max_table_score_degradation_vs_bf16"],
-        table_degradation is not None and table_degradation <= limits["max_table_score_degradation_vs_bf16"],
-        "BF16 minus candidate mean table score",
-    ))
+    table_degradation = (
+        bf16_table - candidate_table
+        if bf16_table is not None and candidate_table is not None
+        else None
+    )
+    gates.append(
+        _gate(
+            "candidate_table_score_vs_bf16",
+            table_degradation,
+            limits["max_table_score_degradation_vs_bf16"],
+            table_degradation is not None
+            and table_degradation <= limits["max_table_score_degradation_vs_bf16"],
+            "BF16 minus candidate mean table score",
+        )
+    )
+
+    performance_keys = (
+        "image_path",
+        "prompt",
+        "max_tokens",
+        "num_warmup",
+        "num_runs",
+        "system",
+    )
+    gates.append(
+        _gate(
+            "performance_aggregates_recomputed",
+            {
+                "reference": _performance_aggregates_valid(reference_performance),
+                "candidate": _performance_aggregates_valid(candidate_performance),
+            },
+            {"reference": True, "candidate": True},
+            _performance_aggregates_valid(reference_performance)
+            and _performance_aggregates_valid(candidate_performance),
+            "Performance means must recompute from complete benchmark runs",
+        )
+    )
+    gates.append(
+        _gate(
+            "same_performance_setup",
+            {
+                key: [reference_performance.get(key), candidate_performance.get(key)]
+                for key in performance_keys
+            },
+            "identical setup with at least three complete runs",
+            all(
+                reference_performance.get(key) == candidate_performance.get(key)
+                for key in performance_keys
+            )
+            and reference_performance.get("prompt") == "<image>document parsing."
+            and _positive_integer(reference_performance, "max_tokens") is not None
+            and _positive_integer(reference_performance, "num_warmup") is not None
+            and _performance_runs_complete(reference_performance)
+            and _performance_runs_complete(candidate_performance)
+            and isinstance(reference_performance.get("system"), dict)
+            and bool(reference_performance["system"]),
+            "Performance comparisons must use the same host and benchmark recipe",
+        )
+    )
 
     reference_tps = _number(reference_performance, "mean_tps")
     candidate_tps = _number(candidate_performance, "mean_tps")
-    tps_ratio = candidate_tps / reference_tps if candidate_tps is not None and reference_tps and reference_tps > 0 else None
-    gates.append(_gate(
-        "candidate_tps_vs_reference", tps_ratio, limits["min_tps_ratio_vs_reference"],
-        tps_ratio is not None and tps_ratio >= limits["min_tps_ratio_vs_reference"],
-        "Candidate decode throughput divided by Sahil-reference throughput",
-    ))
-    gates.append(_gate(
-        "rswa_8k_bounded", rswa.get("pass_conditions"), True,
-        rswa.get("passed") is True, "8K generation, bounded cache, and stable throughput",
-    ))
+    tps_ratio = (
+        candidate_tps / reference_tps
+        if candidate_tps is not None and reference_tps and reference_tps > 0
+        else None
+    )
+    gates.append(
+        _gate(
+            "candidate_tps_vs_reference",
+            tps_ratio,
+            limits["min_tps_ratio_vs_reference"],
+            tps_ratio is not None and tps_ratio >= limits["min_tps_ratio_vs_reference"],
+            "Candidate decode throughput divided by Sahil-reference throughput",
+        )
+    )
+    rswa_results = rswa.get("test_results")
+    long_results = (
+        [
+            result
+            for result in rswa_results
+            if isinstance(rswa_results, list)
+            and isinstance(result, dict)
+            and _positive_integer(result, "max_tokens") == limits["min_rswa_tokens"]
+        ]
+        if isinstance(rswa_results, list)
+        else []
+    )
+    long_result = long_results[0] if len(long_results) == 1 else {}
+    repetition_rate = _number(long_result, "repetition_rate")
+    forced_tokens = _positive_integer(rswa, "force_min_tokens")
+    generated_tokens = _positive_integer(long_result, "tokens_generated")
+    pass_conditions = rswa.get("pass_conditions")
+    rswa_generation = rswa.get("generation_settings")
+    gates.append(
+        _gate(
+            "rswa_8k_bounded",
+            {
+                "pass_conditions": pass_conditions,
+                "tokens": long_result.get("tokens_generated"),
+                "repetition_rate": repetition_rate,
+            },
+            {
+                "min_tokens": limits["min_rswa_tokens"],
+                "max_repetition_rate": limits["max_rswa_repetition_rate"],
+            },
+            rswa.get("passed") is True
+            and isinstance(pass_conditions, dict)
+            and pass_conditions.get("cache_bounded") is True
+            and pass_conditions.get("tps_stable") is True
+            and pass_conditions.get("8k_test_passed") is True
+            and rswa.get("prompt") == "<image>document parsing."
+            and isinstance(rswa_generation, dict)
+            and _number(rswa_generation, "temperature") == 0.0
+            and _positive_integer(rswa_generation, "no_repeat_ngram_size") == 35
+            and _positive_integer(rswa_generation, "ngram_window") == 128
+            and forced_tokens is not None
+            and forced_tokens >= limits["min_rswa_tokens"]
+            and len(long_results) == 1
+            and long_result.get("status") == "success"
+            and long_result.get("tokens_generated_reliable") is True
+            and generated_tokens is not None
+            and generated_tokens >= limits["min_rswa_tokens"]
+            and repetition_rate is not None
+            and 0 <= repetition_rate <= limits["max_rswa_repetition_rate"],
+            "8K generation must have bounded cache, stable throughput, and repetition",
+        )
+    )
+    provenance_dataset = provenance.get("evaluation_dataset")
+    provenance_calibration_dataset = provenance.get("calibration_dataset")
+    source_config = provenance.get("source_config")
+    gates.append(
+        _gate(
+            "provenance_matches_release",
+            {
+                "source": provenance.get("source_model"),
+                "reference": provenance.get("reference_model"),
+                "target": provenance.get("target_repo"),
+                "source_config": source_config,
+                "source_revision": provenance.get("source_revision"),
+                "reference_revision": provenance.get("reference_revision"),
+                "dataset_digest": (
+                    provenance_dataset.get("content_sha256")
+                    if isinstance(provenance_dataset, dict)
+                    else None
+                ),
+                "calibration_dataset_digest": (
+                    provenance_calibration_dataset.get("content_sha256")
+                    if isinstance(provenance_calibration_dataset, dict)
+                    else None
+                ),
+            },
+            {
+                "source": source_id,
+                "reference": reference_id,
+                "target": repo_id,
+                "source_config_sha256": source_config_sha256,
+                "source_revision": source_revision,
+                "reference_revision": reference_revision,
+                "dataset_digest": dataset.get("content_sha256"),
+                "calibration_dataset_digest": calibration_dataset.get(
+                    "content_sha256"
+                ),
+            },
+            provenance.get("source_model") == source_id
+            and provenance.get("reference_model") == reference_id
+            and provenance.get("target_repo") == repo_id
+            and provenance.get("source_revision") == source_revision
+            and provenance.get("reference_revision") == reference_revision
+            and isinstance(source_config, dict)
+            and source_config.get("path") == "config.json"
+            and source_config.get("sha256") == source_config_sha256
+            and provenance_dataset == dataset
+            and provenance_calibration_dataset == calibration_dataset,
+            "Provenance must bind the source, selection/evaluation datasets, and target",
+        )
+    )
     return gates
 
 
 def _metric_summary(payload: dict) -> dict:
     keys = (
-        "model_path", "num_samples", "mean_cer", "mean_digit_cer",
-        "mean_cjk_cer", "mean_table_score", "mean_tps",
-        "mean_peak_memory_mb", "max_tokens", "profile",
+        "model_path",
+        "num_samples",
+        "mean_cer",
+        "mean_digit_cer",
+        "mean_cjk_cer",
+        "mean_table_score",
+        "mean_tps",
+        "mean_peak_memory_mb",
+        "max_tokens",
+        "profile",
     )
     return {key: payload.get(key) for key in keys if key in payload}
 
@@ -456,33 +1836,64 @@ def build_release_manifest(
     candidate_dir: str | Path,
     reference_dir: str | Path,
     source_dir: str | Path,
+    calibration_dir: str | Path,
     eval_dir: str | Path,
     evidence_paths: dict[str, str | Path],
     repo_id: str,
     source_id: str,
+    source_revision: str,
     reference_id: str,
+    reference_revision: str,
     thresholds: dict | None = None,
 ) -> dict:
     """Build a complete release decision from on-disk evidence."""
-    evidence = {name: load_json_object(path) for name, path in evidence_paths.items()}
-    required = {
-        "bf16_accuracy", "reference_accuracy", "candidate_accuracy",
-        "reference_performance", "candidate_performance", "candidate_rswa",
-        "sensitivity_results", "calibration_results", "generated_precision_map", "provenance",
-    }
-    missing = sorted(required - set(evidence))
+    required = RELEASE_EVIDENCE_NAMES
+    if not isinstance(evidence_paths, dict):
+        raise TypeError("evidence_paths must be a dictionary")
+    missing = sorted(required - set(evidence_paths))
     if missing:
         raise ValueError("Missing release evidence: " + ", ".join(missing))
+    unexpected = sorted(set(evidence_paths) - required)
+    if unexpected:
+        raise ValueError("Unexpected release evidence: " + ", ".join(unexpected))
+    evidence_files = [Path(path) for path in evidence_paths.values()]
+    if len({path.name for path in evidence_files}) != len(evidence_files):
+        raise ValueError("Release evidence filenames must be unique")
+    symlink_evidence = [path.name for path in evidence_files if path.is_symlink()]
+    if symlink_evidence:
+        raise ValueError(
+            "Release evidence must not be symbolic links: "
+            + ", ".join(symlink_evidence)
+        )
+    limits = validate_release_thresholds(thresholds)
+    evidence = {name: load_json_object(path) for name, path in evidence_paths.items()}
 
     candidate_weights = model_weight_manifest(candidate_dir)
-    reference_weights = model_weight_manifest(reference_dir)
+    # Hugging Face snapshot directories use content-addressed links into the
+    # immutable blob cache. Hash their resolved bytes, while keeping candidate
+    # and source checkpoints link-free.
+    reference_weights = model_weight_manifest(reference_dir, allow_symlinks=True)
     source_weights = model_weight_manifest(source_dir)
+    calibration_dataset = dataset_manifest(calibration_dir)
     dataset = dataset_manifest(eval_dir)
     metadata = validate_candidate_metadata(candidate_dir)
+    source_config_path = Path(source_dir) / "config.json"
+    if source_config_path.is_symlink() or not source_config_path.is_file():
+        raise FileNotFoundError(
+            f"Source config must be a regular file: {source_config_path}"
+        )
+    source_config_sha256 = sha256_file(source_config_path)
+    candidate_files = candidate_metadata_manifest(candidate_dir)
+    project_files = release_files_manifest(
+        PROJECT_ROOT,
+        RELEASE_PROJECT_FILES,
+        required=True,
+    )
     gates = evaluate_release_gates(
         candidate_weights=candidate_weights,
         reference_weights=reference_weights,
         dataset=dataset,
+        calibration_dataset=calibration_dataset,
         metadata=metadata,
         bf16_accuracy=evidence["bf16_accuracy"],
         reference_accuracy=evidence["reference_accuracy"],
@@ -490,8 +1901,126 @@ def build_release_manifest(
         reference_performance=evidence["reference_performance"],
         candidate_performance=evidence["candidate_performance"],
         rswa=evidence["candidate_rswa"],
-        thresholds=thresholds,
+        provenance=evidence["provenance"],
+        source_model_name=Path(source_dir).name,
+        source_id=source_id,
+        source_config_sha256=source_config_sha256,
+        source_revision=source_revision,
+        reference_id=reference_id,
+        reference_revision=reference_revision,
+        candidate_model_name=Path(candidate_dir).name,
+        repo_id=repo_id,
+        thresholds=limits,
+    )
+    calibration_validation = validate_calibration_results(
+        evidence["calibration_results"],
+        bf16_accuracy=evidence["calibration_baseline_accuracy"],
+        reference_performance=evidence["calibration_reference_performance"],
+        calibration_dataset=calibration_dataset,
+        source_revision=source_revision,
+        reference_revision=reference_revision,
+        evidence=evidence,
+        evidence_paths=evidence_paths,
+        thresholds=limits,
+    )
+    gates.append(
+        _gate(
+            "calibration_recomputed",
+            calibration_validation,
+            "content-addressed inputs and fastest passing experiment",
+            True,
+            "Calibration claims must recompute from raw benchmark evidence",
+        )
+    )
+    sensitivity = evidence["sensitivity_results"]
+    sensitivity_baseline = sensitivity.get("baseline")
+    calibration_baseline = evidence["calibration_baseline_accuracy"]
+    sensitivity_metric_keys = (
+        "num_samples",
+        "num_digit_samples",
+        "num_table_samples",
+        "mean_cer",
+        "mean_digit_cer",
+        "mean_table_score",
+    )
+    sensitivity_matches_calibration = (
+        isinstance(sensitivity_baseline, dict)
+        and sensitivity.get("dataset") == calibration_dataset
+        and sensitivity.get("prompt") == calibration_baseline.get("prompt")
+        and sensitivity.get("max_tokens") == calibration_baseline.get("max_tokens")
+        and all(
+            sensitivity_baseline.get(key) == calibration_baseline.get(key)
+            for key in sensitivity_metric_keys
+        )
     )
+    gates.append(
+        _gate(
+            "sensitivity_matches_calibration_dataset",
+            {
+                "dataset_digest": (
+                    sensitivity.get("dataset", {}).get("content_sha256")
+                    if isinstance(sensitivity.get("dataset"), dict)
+                    else None
+                ),
+                "baseline_metrics_match": sensitivity_matches_calibration,
+            },
+            {
+                "dataset_digest": calibration_dataset.get("content_sha256"),
+                "baseline_metrics_match": True,
+            },
+            sensitivity_matches_calibration,
+            "Sensitivity decisions must use the recorded selection dataset and baseline",
+        )
+    )
+    candidate_precision_sha = sha256_file(Path(candidate_dir) / "precision_map.json")
+    evidence_precision_sha = sha256_file(evidence_paths["generated_precision_map"])
+    gates.append(
+        _gate(
+            "candidate_precision_map_matches_evidence",
+            candidate_precision_sha,
+            evidence_precision_sha,
+            candidate_precision_sha == evidence_precision_sha,
+            "The executable candidate precision map must equal approved evidence",
+        )
+    )
+    evidence_map = evidence["generated_precision_map"]
+    map_thresholds = (
+        evidence_map.get("_generated_from", {}).get("thresholds")
+        if isinstance(evidence_map.get("_generated_from"), dict)
+        else None
+    )
+    reproduce_kwargs: dict[str, Any] = {}
+    if isinstance(map_thresholds, dict):
+        if map_thresholds.get("cer_delta") is not None:
+            reproduce_kwargs["cer_threshold"] = map_thresholds["cer_delta"]
+        if map_thresholds.get("digit_cer_delta") is not None:
+            reproduce_kwargs["digit_cer_threshold"] = map_thresholds[
+                "digit_cer_delta"
+            ]
+        if map_thresholds.get("table_score_degradation") is not None:
+            reproduce_kwargs["table_degradation_threshold"] = map_thresholds[
+                "table_score_degradation"
+            ]
+    reproduced_precision_map = generate_precision_map(
+        load_json_object(PROJECT_ROOT / "quantization/precision_map.json"),
+        evidence["sensitivity_results"],
+        calibration_results=evidence["calibration_results"],
+        **reproduce_kwargs,
+    )
+    gates.append(
+        _gate(
+            "precision_map_reproducible",
+            _json_digest(evidence["generated_precision_map"]),
+            _json_digest(reproduced_precision_map),
+            evidence["generated_precision_map"] == reproduced_precision_map,
+            "Sensitivity and calibration evidence must reproduce the executable map",
+        )
+    )
+    actual_gate_names = tuple(gate.get("name") for gate in gates)
+    if actual_gate_names != MLX_RELEASE_GATE_NAMES:
+        raise RuntimeError(
+            "MLX release gate implementation does not match its publication contract"
+        )
     artifact_hashes = {
         name: {
             "filename": Path(path).name,
@@ -500,26 +2029,39 @@ def build_release_manifest(
         }
         for name, path in evidence_paths.items()
     }
-    limits = {**DEFAULT_THRESHOLDS, **(thresholds or {})}
     return {
-        "schema_version": 1,
+        "schema_version": MLX_RELEASE_SCHEMA_VERSION,
         "created_at": datetime.now(timezone.utc).isoformat(),
         "release_approved": all(gate["passed"] for gate in gates),
         "repo_id": repo_id,
-        "source": {"id": source_id, "weights": source_weights},
-        "reference": {"id": reference_id, "weights": reference_weights},
+        "source": {
+            "id": source_id,
+            "resolved_revision": source_revision,
+            "weights": source_weights,
+        },
+        "reference": {
+            "id": reference_id,
+            "resolved_revision": reference_revision,
+            "weights": reference_weights,
+        },
         "candidate": {
             "name": Path(candidate_dir).name,
             "weights": candidate_weights,
+            "files": candidate_files,
             "metadata": metadata,
         },
-        "dataset": dataset,
+        "project_files": project_files,
+        "calibration_dataset": calibration_dataset,
+        "evaluation_dataset": dataset,
         "thresholds": limits,
         "metrics": {
             name: _metric_summary(evidence[name])
             for name in (
-                "bf16_accuracy", "reference_accuracy", "candidate_accuracy",
-                "reference_performance", "candidate_performance",
+                "bf16_accuracy",
+                "reference_accuracy",
+                "candidate_accuracy",
+                "reference_performance",
+                "candidate_performance",
             )
         },
         "rswa": evidence["candidate_rswa"],
@@ -545,36 +2087,48 @@ def main() -> None:
     parser.add_argument("--candidate-dir", required=True, type=Path)
     parser.add_argument("--reference-dir", required=True, type=Path)
     parser.add_argument("--source-dir", required=True, type=Path)
+    parser.add_argument("--calibration-dir", required=True, type=Path)
     parser.add_argument("--eval-dir", required=True, type=Path)
     parser.add_argument("--artifacts-dir", required=True, type=Path)
     parser.add_argument("--output", required=True, type=Path)
     parser.add_argument("--repo-id", required=True)
     parser.add_argument("--source-id", default="baidu/Unlimited-OCR")
-    parser.add_argument("--reference-id", default="sahilchachra/unlimited-ocr-mxfp8-mlx")
+    parser.add_argument("--source-revision", required=True)
+    parser.add_argument(
+        "--reference-id", default="sahilchachra/unlimited-ocr-mxfp8-mlx"
+    )
+    parser.add_argument("--reference-revision", required=True)
     args = parser.parse_args()
 
     evidence_paths = {
         name: args.artifacts_dir / f"{name}.json"
-        for name in (
-            "bf16_accuracy", "reference_accuracy", "candidate_accuracy",
-            "reference_performance", "candidate_performance", "candidate_rswa",
-            "sensitivity_results", "calibration_results", "generated_precision_map", "provenance",
-        )
+        for name in RELEASE_EVIDENCE_NAMES - CALIBRATION_RAW_EVIDENCE_NAMES
     }
+    calibration = load_json_object(evidence_paths["calibration_results"])
+    evidence_paths.update(
+        calibration_raw_evidence_paths(args.artifacts_dir, calibration)
+    )
     manifest = build_release_manifest(
         candidate_dir=args.candidate_dir,
         reference_dir=args.reference_dir,
         source_dir=args.source_dir,
+        calibration_dir=args.calibration_dir,
         eval_dir=args.eval_dir,
         evidence_paths=evidence_paths,
         repo_id=args.repo_id,
         source_id=args.source_id,
+        source_revision=args.source_revision,
         reference_id=args.reference_id,
+        reference_revision=args.reference_revision,
     )
     args.output.parent.mkdir(parents=True, exist_ok=True)
-    args.output.write_text(json.dumps(manifest, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+    args.output.write_text(
+        json.dumps(manifest, indent=2, ensure_ascii=False) + "\n", encoding="utf-8"
+    )
     for gate in manifest["gates"]:
-        print(f"[{'PASS' if gate['passed'] else 'FAIL'}] {gate['name']}: {gate['actual']}")
+        print(
+            f"[{'PASS' if gate['passed'] else 'FAIL'}] {gate['name']}: {gate['actual']}"
+        )
     print(f"Release approved: {manifest['release_approved']}")
     if not manifest["release_approved"]:
         raise SystemExit(1)

quantization/run_pipeline.py

@@ -7,6 +7,7 @@ import argparse
 import importlib.metadata
 import json
 import platform
+import re
 import subprocess
 import sys
 import time
@@ -20,10 +21,13 @@ sys.path.insert(0, str(PROJECT_ROOT))
 sys.path.insert(0, str(PROJECT_ROOT / "src"))
 
 from quantization.release_gate import (  # noqa: E402
+    calibration_raw_evidence_paths,
     dataset_manifest,
+    dataset_separation,
     generate_precision_map,
     load_json_object,
     sha256_file,
+    validate_calibration_results,
 )
 
 
@@ -72,6 +76,18 @@ def assert_unquantized_source(model_path: str) -> dict:
     return {"path": config_path.name, "sha256": sha256_file(config_path)}
 
 
+def verify_huggingface_revision(repo_id: str, revision: str) -> str:
+    """Resolve an explicitly pinned Hub revision and require an exact commit."""
+    from huggingface_hub import HfApi
+
+    resolved = HfApi().model_info(repo_id, revision=revision).sha
+    if resolved != revision:
+        raise ValueError(
+            f"Revision for {repo_id} resolved to {resolved!r}, expected {revision!r}"
+        )
+    return resolved
+
+
 def system_provenance() -> dict:
     versions = {}
     for distribution in ("mlx", "mlx-vlm", "huggingface-hub", "numpy", "Pillow"):
@@ -93,7 +109,19 @@ def preflight(args: argparse.Namespace) -> bool:
     """Validate source, dataset, and release inputs before Metal allocation."""
     try:
         source_config = assert_unquantized_source(args.model_path)
-        dataset = dataset_manifest(args.eval_dir)
+        verify_huggingface_revision(args.source_id, args.source_revision)
+        verify_huggingface_revision(
+            args.reference_model,
+            args.reference_revision,
+        )
+        calibration_dataset = dataset_manifest(args.calibration_dir)
+        evaluation_dataset = dataset_manifest(args.eval_dir)
+        separation = dataset_separation(calibration_dataset, evaluation_dataset)
+        if not separation["passed"]:
+            raise ValueError(
+                "Calibration and final evaluation datasets must be disjoint; "
+                f"details={separation}"
+            )
         if not args.image.is_file():
             raise FileNotFoundError(f"Performance/R-SWA image not found: {args.image}")
         if args.output_dir.exists() and args.step in {"all", "convert"}:
@@ -102,11 +130,14 @@ def preflight(args: argparse.Namespace) -> bool:
         provenance = {
             **system_provenance(),
             "source_model": args.source_id,
+            "source_revision": args.source_revision,
             "source_local_name": Path(args.model_path).name,
             "source_config": source_config,
             "reference_model": args.reference_model,
+            "reference_revision": args.reference_revision,
             "target_repo": args.repo_id,
-            "dataset": dataset,
+            "calibration_dataset": calibration_dataset,
+            "evaluation_dataset": evaluation_dataset,
             "smoke_image": {
                 "name": args.image.name,
                 "sha256": sha256_file(args.image),
@@ -124,8 +155,10 @@ def preflight(args: argparse.Namespace) -> bool:
             encoding="utf-8",
         )
         print(f"[OK] Unquantized source: {args.model_path}")
-        print(f"[OK] Evaluation samples: {dataset['num_samples']}")
-        print(f"[OK] Dataset digest: {dataset['content_sha256']}")
+        print(f"[OK] Calibration samples: {calibration_dataset['num_samples']}")
+        print(f"[OK] Calibration digest: {calibration_dataset['content_sha256']}")
+        print(f"[OK] Held-out evaluation samples: {evaluation_dataset['num_samples']}")
+        print(f"[OK] Evaluation digest: {evaluation_dataset['content_sha256']}")
         return True
     except Exception as exc:
         print(f"[FAIL] Preflight: {exc}")
@@ -138,13 +171,171 @@ def sensitivity(args: argparse.Namespace) -> bool:
         str(QUANT_DIR / "layer_sensitivity.py"),
         "--model-path", args.model_path,
         "--source-id", args.source_id,
-        "--eval-dir", str(args.eval_dir),
+        "--source-revision", args.source_revision,
+        "--eval-dir", str(args.calibration_dir),
         "--output", str(args.artifacts_dir / "sensitivity_results.json"),
         "--max-tokens", str(args.accuracy_tokens),
     ]
     return run_command(cmd, "Layer sensitivity analysis", dry_run=args.dry_run)
 
 
+def _accuracy_on_dir(
+    model: str,
+    eval_dir: Path,
+    output: Path,
+    args: argparse.Namespace,
+    *,
+    revision: str | None = None,
+) -> list[str]:
+    command = [
+        sys.executable,
+        str(PROJECT_ROOT / "benchmarks" / "run_accuracy.py"),
+        "--model-path", model,
+        "--eval-dir", str(eval_dir),
+        "--output", str(output),
+        "--max-tokens", str(args.accuracy_tokens),
+        "--profile", "accurate",
+    ]
+    if revision is not None:
+        command.extend(["--served-revision", revision])
+    return command
+
+
+def calibrate(args: argparse.Namespace) -> bool:
+    """Build LM-head candidates from sensitivity, measure them, select precision.
+
+    Schema-3 release requires ``calibration_results.json`` with hashed raw
+    evidence before the final precision map can be generated.
+    """
+    if args.dry_run:
+        print("[DRY RUN] Calibrate LM head (bf16 / mxfp8 / affine8 experiments)")
+        return True
+
+    try:
+        sensitivity_path = args.artifacts_dir / "sensitivity_results.json"
+        # Align group retention with the joint release CER/digit budgets so
+        # cumulative MXFP8 error is less likely to blow the 0.01 absolute caps.
+        interim_map = generate_precision_map(
+            load_json_object(BASE_PRECISION_MAP),
+            load_json_object(sensitivity_path),
+            calibration_results=None,
+            cer_threshold=0.01,
+            digit_cer_threshold=0.01,
+            table_degradation_threshold=0.01,
+        )
+        interim_path = args.artifacts_dir / "interim_precision_map.json"
+        interim_path.write_text(
+            json.dumps(interim_map, indent=2, ensure_ascii=False) + "\n",
+            encoding="utf-8",
+        )
+        print(f"[OK] Interim precision map (pre-calibration): {interim_path}")
+    except Exception as exc:
+        print(f"[FAIL] Interim precision map: {exc}")
+        return False
+
+    baseline_accuracy = args.artifacts_dir / "calibration_baseline_accuracy.json"
+    reference_performance = (
+        args.artifacts_dir / "calibration_reference_performance.json"
+    )
+    jobs: list[tuple[list[str], str]] = [
+        (
+            _accuracy_on_dir(
+                args.model_path,
+                args.calibration_dir,
+                baseline_accuracy,
+                args,
+                revision=args.source_revision,
+            ),
+            "Calibration baseline BF16 accuracy",
+        ),
+        (
+            _performance_command(
+                args.reference_model,
+                reference_performance,
+                args,
+                revision=args.reference_revision,
+            ),
+            "Calibration reference performance",
+        ),
+    ]
+    if not all(run_command(cmd, description) for cmd, description in jobs):
+        return False
+
+    experiment_specs = (
+        ("bf16-head", "bfloat16"),
+        ("mxfp8-head", "mxfp8"),
+        ("affine8-head", "affine8"),
+    )
+    experiment_args: list[str] = []
+    for label, precision in experiment_specs:
+        exp_map = dict(interim_map)
+        exp_map["language_model.lm_head"] = precision
+        exp_map_path = args.artifacts_dir / f"calibration_{precision}_precision_map.json"
+        exp_map_path.write_text(
+            json.dumps(exp_map, indent=2, ensure_ascii=False) + "\n",
+            encoding="utf-8",
+        )
+        exp_model_dir = (
+            args.output_dir.parent / f"{args.output_dir.name}-cal-{precision}"
+        )
+        if exp_model_dir.exists():
+            print(f"[FAIL] Calibration model directory already exists: {exp_model_dir}")
+            return False
+        convert_cmd = [
+            sys.executable,
+            str(QUANT_DIR / "mixed_precision_convert.py"),
+            "--model-path", args.model_path,
+            "--source-revision", args.source_revision,
+            "--precision-map", str(exp_map_path),
+            "--output-dir", str(exp_model_dir),
+        ]
+        if not run_command(convert_cmd, f"Calibration convert ({label})"):
+            return False
+
+        # Filenames must match CALIBRATION_EVIDENCE_KEYS in release_gate.py.
+        accuracy_path = args.artifacts_dir / f"calibration_{precision}_accuracy.json"
+        performance_path = (
+            args.artifacts_dir / f"calibration_{precision}_performance.json"
+        )
+        if not run_command(
+            _accuracy_on_dir(
+                str(exp_model_dir),
+                args.calibration_dir,
+                accuracy_path,
+                args,
+            ),
+            f"Calibration accuracy ({label})",
+        ):
+            return False
+        if not run_command(
+            _performance_command(str(exp_model_dir), performance_path, args),
+            f"Calibration performance ({label})",
+        ):
+            return False
+        experiment_args.extend(
+            [
+                "--experiment",
+                label,
+                precision,
+                str(accuracy_path),
+                str(performance_path),
+            ]
+        )
+
+    select_cmd = [
+        sys.executable,
+        str(QUANT_DIR / "calibrate_precision.py"),
+        "--bf16-accuracy", str(baseline_accuracy),
+        "--reference-performance", str(reference_performance),
+        "--calibration-dir", str(args.calibration_dir),
+        "--source-revision", args.source_revision,
+        "--reference-revision", args.reference_revision,
+        *experiment_args,
+        "--output", str(args.artifacts_dir / "calibration_results.json"),
+    ]
+    return run_command(select_cmd, "Select calibrated LM-head precision")
+
+
 def precision_map(args: argparse.Namespace) -> bool:
     sensitivity_path = args.artifacts_dir / "sensitivity_results.json"
     output_path = args.artifacts_dir / "generated_precision_map.json"
@@ -153,14 +344,39 @@ def precision_map(args: argparse.Namespace) -> bool:
         return True
     try:
         calibration_path = args.artifacts_dir / "calibration_results.json"
+        calibration = load_json_object(calibration_path)
+        evidence_paths = {
+            "calibration_baseline_accuracy": (
+                args.artifacts_dir / "calibration_baseline_accuracy.json"
+            ),
+            "calibration_reference_performance": (
+                args.artifacts_dir / "calibration_reference_performance.json"
+            ),
+            **calibration_raw_evidence_paths(args.artifacts_dir, calibration),
+        }
+        evidence = {
+            name: load_json_object(path) for name, path in evidence_paths.items()
+        }
+        validate_calibration_results(
+            calibration,
+            bf16_accuracy=evidence["calibration_baseline_accuracy"],
+            reference_performance=evidence[
+                "calibration_reference_performance"
+            ],
+            calibration_dataset=dataset_manifest(args.calibration_dir),
+            source_revision=args.source_revision,
+            reference_revision=args.reference_revision,
+            evidence=evidence,
+            evidence_paths=evidence_paths,
+        )
         generated = generate_precision_map(
             load_json_object(BASE_PRECISION_MAP),
             load_json_object(sensitivity_path),
-            calibration_results=(
-                load_json_object(calibration_path)
-                if calibration_path.is_file()
-                else None
-            ),
+            calibration_results=calibration,
+            # Must match the interim calibration map thresholds.
+            cer_threshold=0.01,
+            digit_cer_threshold=0.01,
+            table_degradation_threshold=0.01,
         )
         output_path.write_text(
             json.dumps(generated, indent=2, ensure_ascii=False) + "\n",
@@ -178,14 +394,21 @@ def convert(args: argparse.Namespace) -> bool:
         sys.executable,
         str(QUANT_DIR / "mixed_precision_convert.py"),
         "--model-path", args.model_path,
+        "--source-revision", args.source_revision,
         "--precision-map", str(args.artifacts_dir / "generated_precision_map.json"),
         "--output-dir", str(args.output_dir),
     ]
     return run_command(cmd, "BF16 to OCR-aware MXFP8 conversion", dry_run=args.dry_run)
 
 
-def _accuracy_command(model: str, output: Path, args: argparse.Namespace) -> list[str]:
-    return [
+def _accuracy_command(
+    model: str,
+    output: Path,
+    args: argparse.Namespace,
+    *,
+    revision: str | None = None,
+) -> list[str]:
+    command = [
         sys.executable,
         str(PROJECT_ROOT / "benchmarks" / "run_accuracy.py"),
         "--model-path", model,
@@ -194,10 +417,19 @@ def _accuracy_command(model: str, output: Path, args: argparse.Namespace) -> lis
         "--max-tokens", str(args.accuracy_tokens),
         "--profile", "accurate",
     ]
-
-
-def _performance_command(model: str, output: Path, args: argparse.Namespace) -> list[str]:
-    return [
+    if revision is not None:
+        command.extend(["--served-revision", revision])
+    return command
+
+
+def _performance_command(
+    model: str,
+    output: Path,
+    args: argparse.Namespace,
+    *,
+    revision: str | None = None,
+) -> list[str]:
+    command = [
         sys.executable,
         str(PROJECT_ROOT / "benchmarks" / "run_performance.py"),
         "--model-path", model,
@@ -207,15 +439,18 @@ def _performance_command(model: str, output: Path, args: argparse.Namespace) ->
         "--warmup", str(args.performance_warmup),
         "--runs", str(args.performance_runs),
     ]
+    if revision is not None:
+        command.extend(["--served-revision", revision])
+    return command
 
 
 def validate(args: argparse.Namespace) -> bool:
     """Benchmark BF16, Sahil reference, candidate, then stress candidate R-SWA."""
     jobs = [
-        (_accuracy_command(args.model_path, args.artifacts_dir / "bf16_accuracy.json", args), "BF16 accuracy"),
-        (_accuracy_command(args.reference_model, args.artifacts_dir / "reference_accuracy.json", args), "Sahil-reference accuracy"),
+        (_accuracy_command(args.model_path, args.artifacts_dir / "bf16_accuracy.json", args, revision=args.source_revision), "BF16 accuracy"),
+        (_accuracy_command(args.reference_model, args.artifacts_dir / "reference_accuracy.json", args, revision=args.reference_revision), "Sahil-reference accuracy"),
         (_accuracy_command(str(args.output_dir), args.artifacts_dir / "candidate_accuracy.json", args), "Candidate accuracy"),
-        (_performance_command(args.reference_model, args.artifacts_dir / "reference_performance.json", args), "Sahil-reference performance"),
+        (_performance_command(args.reference_model, args.artifacts_dir / "reference_performance.json", args, revision=args.reference_revision), "Sahil-reference performance"),
         (_performance_command(str(args.output_dir), args.artifacts_dir / "candidate_performance.json", args), "Candidate performance"),
         ([
             sys.executable,
@@ -230,13 +465,14 @@ def validate(args: argparse.Namespace) -> bool:
     return all(run_command(cmd, description, dry_run=args.dry_run) for cmd, description in jobs)
 
 
-def resolve_reference_dir(reference_model: str) -> Path:
+def resolve_reference_dir(reference_model: str, revision: str) -> Path:
     local = Path(reference_model)
     if local.is_dir():
         return local
     from huggingface_hub import snapshot_download
     return Path(snapshot_download(
         reference_model,
+        revision=revision,
         allow_patterns=[
             "*.safetensors", "model.safetensors.index.json", "config.json",
             "processor_config.json", "tokenizer*.json", "special_tokens_map.json",
@@ -250,7 +486,10 @@ def gate(args: argparse.Namespace) -> bool:
         reference_dir = Path("<resolved-reference>")
     else:
         try:
-            reference_dir = resolve_reference_dir(args.reference_model)
+            reference_dir = resolve_reference_dir(
+                args.reference_model,
+                args.reference_revision,
+            )
         except Exception as exc:
             print(f"[FAIL] Reference download: {exc}")
             return False
@@ -260,12 +499,15 @@ def gate(args: argparse.Namespace) -> bool:
         "--candidate-dir", str(args.output_dir),
         "--reference-dir", str(reference_dir),
         "--source-dir", args.model_path,
+        "--calibration-dir", str(args.calibration_dir),
         "--eval-dir", str(args.eval_dir),
         "--artifacts-dir", str(args.artifacts_dir),
         "--output", str(args.artifacts_dir / "release_manifest.json"),
         "--repo-id", args.repo_id,
         "--source-id", args.source_id,
+        "--source-revision", args.source_revision,
         "--reference-id", args.reference_model,
+        "--reference-revision", args.reference_revision,
     ]
     return run_command(cmd, "Fail-closed release gate", dry_run=args.dry_run)
 
@@ -288,8 +530,21 @@ def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(description="Verified OCR-aware model release pipeline")
     parser.add_argument("--model-path", default=str(DEFAULT_SOURCE))
     parser.add_argument("--source-id", default=DEFAULT_SOURCE_ID)
+    parser.add_argument("--source-revision", required=True)
     parser.add_argument("--reference-model", default=DEFAULT_REFERENCE)
-    parser.add_argument("--eval-dir", type=Path, default=PROJECT_ROOT / "eval_dataset")
+    parser.add_argument("--reference-revision", required=True)
+    parser.add_argument(
+        "--calibration-dir",
+        required=True,
+        type=Path,
+        help="Selection-only OCR dataset used for sensitivity and head calibration",
+    )
+    parser.add_argument(
+        "--eval-dir",
+        required=True,
+        type=Path,
+        help="Disjoint held-out OCR dataset used only for final release validation",
+    )
     parser.add_argument("--output-dir", type=Path, default=DEFAULT_OUTPUT)
     parser.add_argument("--artifacts-dir", type=Path, default=DEFAULT_ARTIFACTS)
     parser.add_argument("--image", type=Path, default=PROJECT_ROOT / "test_data" / "test_invoice.png")
@@ -301,17 +556,33 @@ def parse_args() -> argparse.Namespace:
     parser.add_argument("--rswa-lengths", type=int, nargs="+", default=[512, 2048, 8192])
     parser.add_argument(
         "--step",
-        choices=["all", "preflight", "sensitivity", "precision-map", "convert", "validate", "gate", "publish"],
+        choices=[
+            "all",
+            "preflight",
+            "sensitivity",
+            "calibrate",
+            "precision-map",
+            "convert",
+            "validate",
+            "gate",
+            "publish",
+        ],
         default="all",
     )
     parser.add_argument("--dry-run", action="store_true")
     args = parser.parse_args()
-    positive = ("accuracy_tokens", "performance_tokens", "performance_runs")
+    for name in ("source_revision", "reference_revision"):
+        if not re.fullmatch(r"[0-9a-f]{40}", getattr(args, name)):
+            parser.error(f"--{name.replace('_', '-')} must be a 40-character lowercase commit SHA")
+    positive = (
+        "accuracy_tokens",
+        "performance_tokens",
+        "performance_warmup",
+        "performance_runs",
+    )
     for name in positive:
         if getattr(args, name) < 1:
             parser.error(f"--{name.replace('_', '-')} must be positive")
-    if args.performance_warmup < 0:
-        parser.error("--performance-warmup must be non-negative")
     if len(args.rswa_lengths) < 3 or args.rswa_lengths != sorted(set(args.rswa_lengths)) or 8192 not in args.rswa_lengths:
         parser.error("--rswa-lengths must be sorted, unique, include 8192, and contain at least three values")
     return args
@@ -322,12 +593,14 @@ def main() -> None:
     steps = {
         "preflight": lambda: preflight(args),
         "sensitivity": lambda: sensitivity(args),
+        "calibrate": lambda: calibrate(args),
         "precision-map": lambda: precision_map(args),
         "convert": lambda: convert(args),
         "validate": lambda: validate(args),
         "gate": lambda: gate(args),
         "publish": lambda: publish(args),
     }
+    # Full release order: selection data → calibrated map → convert → held-out gate.
     selected = list(steps) if args.step == "all" else [args.step]
     for step_name in selected:
         if not steps[step_name]():

quantization_config.json

@@ -1,26 +1,27 @@
 {
   "quantization_method": "mxfp8",
-  "description": "Block-float MX FP8 quantization (~9.19 effective bits/weight) applied to language tower linear weights. Vision encoder and embeddings remain in bfloat16.",
+  "description": "Quantization layout of the reference sahilchachra checkpoint used by the default upload workflow. MXFP8 is applied to quantizable language modules, token embeddings, the LM head, and the projector; the vision encoders and non-quantizable normalization/routing weights remain in bfloat16.",
   "base_model": "baidu/Unlimited-OCR",
   "reference_quantization": "sahilchachra/unlimited-ocr-mxfp8-mlx",
   "quantized_components": {
     "language_model.linear": "mxfp8",
-    "language_model.moe.experts": "mxfp8"
+    "language_model.moe.experts": "mxfp8",
+    "language_model.token_embeddings": "mxfp8",
+    "language_model.lm_head": "mxfp8",
+    "vision_projector": "mxfp8"
   },
   "preserved_bf16_components": {
     "vision_encoder": "bfloat16",
-    "vision_projector": "bfloat16",
-    "token_embeddings": "bfloat16",
     "normalization_layers": "bfloat16",
-    "position_embeddings": "bfloat16"
+    "moe_routing_gates": "bfloat16"
   },
   "effective_bits_per_weight": 9.19,
   "model_size_gb": 3.83,
   "conversion_tool": "mlx-vlm quantizers",
   "notes": [
-    "mlx-vlm quantizers only touch the language tower's linear weights",
-    "The vision encoder and embeddings remain bf16",
-    "On-disk size (~3.6 GB) averages both precision levels",
+    "This file describes the reference weights copied by scripts/upload_model.py; that workflow does not run a new conversion",
+    "The safetensors index contains MXFP8 scale tensors for the token embeddings, LM head, and projector, but not the vision encoders",
+    "quantization/mixed_precision_convert.py can produce a different OCR-aware layout from the BF16 base model",
     "Published config.json uses model_type 'unlimited-ocr' to select mlx-vlm's native R-SWA implementation"
   ]
 }

quantization_summary.json

@@ -2,14 +2,15 @@
   "method": "mxfp8",
   "group_size": 32,
   "bits": 8,
-  "source_model": "baidu/Unlimited-OCR",
-  "precision_map_sha256": "2c0bfbd58fcbb22e17927f3c5e8214e4933a3675a6d1a87e8b3b87b4fbb37b33",
-  "quantized_module_count": 120,
+  "source_model": "Unlimited-OCR",
+  "source_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+  "precision_map_sha256": "c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a",
+  "quantized_module_count": 119,
   "quantized_precision_counts": {
     "affine8": 1,
-    "mxfp8": 119
+    "mxfp8": 118
   },
-  "preserved_quantizable_module_count": 145,
+  "preserved_quantizable_module_count": 146,
   "quantized_modules": [
     "language_model.lm_head",
     "language_model.model.embed_tokens",
@@ -129,8 +130,7 @@
     "language_model.model.layers.9.self_attn.k_proj",
     "language_model.model.layers.9.self_attn.o_proj",
     "language_model.model.layers.9.self_attn.q_proj",
-    "language_model.model.layers.9.self_attn.v_proj",
-    "projector.layers"
+    "language_model.model.layers.9.self_attn.v_proj"
   ],
   "quantized_module_precisions": {
     "language_model.lm_head": "affine8",
@@ -251,7 +251,6 @@
     "language_model.model.layers.9.self_attn.k_proj": "mxfp8",
     "language_model.model.layers.9.self_attn.o_proj": "mxfp8",
     "language_model.model.layers.9.self_attn.q_proj": "mxfp8",
-    "language_model.model.layers.9.self_attn.v_proj": "mxfp8",
-    "projector.layers": "mxfp8"
+    "language_model.model.layers.9.self_attn.v_proj": "mxfp8"
   }
 }

release/bf16_accuracy.json

@@ -1,5 +1,7 @@
 {
   "model_path": "Unlimited-OCR",
+  "served_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+  "backend": "mlx",
   "prompt": "<image>document parsing.",
   "max_tokens": 1024,
   "profile": "accurate",
@@ -12,183 +14,237 @@
   },
   "num_images": 12,
   "num_samples": 12,
-  "total_time_seconds": 36.51101154302887,
-  "mean_cer": 0.2391877132924964,
-  "mean_wer": 0.5784543844272946,
+  "total_time_seconds": 169.77583128899278,
+  "mean_cer": 0.205212265458972,
+  "mean_wer": 0.5919915970344224,
   "num_digit_samples": 12,
   "num_cjk_samples": 3,
-  "mean_digit_cer": 0.08733996757027829,
+  "mean_digit_cer": 0.11330969496189873,
   "mean_cjk_cer": 0.3333333333333333,
   "num_table_samples": 3,
   "mean_table_score": 1.0,
   "mean_repetition_rate": 0.0,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 0.18681318681318682,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": 0.3333333333333333,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 4,
+      "mean_cer": 0.34397793132397353,
+      "mean_digit_cer": 0.1993040848856962,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 4,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.0,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
   "per_file": [
     {
       "file": "0001.png",
-      "cer": 0.02315227070347284,
-      "wer": 0.19282511210762332,
-      "digit_cer": 0.03488372093023256,
+      "category": "english_document",
+      "cer": 0.3923145665773012,
+      "wer": 0.3317757009345794,
+      "digit_cer": 0.18518518518518517,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.554586291997111,
-      "ref_length": 1345,
-      "hyp_length": 1348,
-      "ref_digit_count": 86,
+      "elapsed_seconds": 25.946757499987143,
+      "ref_length": 1332,
+      "hyp_length": 1719,
+      "ref_digit_count": 27,
       "ref_cjk_count": 0
     },
     {
       "file": "0002.png",
-      "cer": 0.9133034379671151,
-      "wer": 0.46107784431137727,
-      "digit_cer": 0.12631578947368421,
+      "category": "english_document",
+      "cer": 0.37906137184115524,
+      "wer": 0.36486486486486486,
+      "digit_cer": 0.1016949152542373,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 6.1484525830019265,
-      "ref_length": 835,
-      "hyp_length": 1378,
-      "ref_digit_count": 95,
+      "elapsed_seconds": 27.539474832999986,
+      "ref_length": 1329,
+      "hyp_length": 1710,
+      "ref_digit_count": 59,
       "ref_cjk_count": 0
     },
     {
       "file": "0003.png",
-      "cer": 0.3923145665773012,
-      "wer": 0.3317757009345794,
-      "digit_cer": 0.18518518518518517,
+      "category": "english_document",
+      "cer": 0.525911708253359,
+      "wer": 0.46551724137931033,
+      "digit_cer": 0.273972602739726,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.549903999999515,
-      "ref_length": 1332,
-      "hyp_length": 1719,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 28.937275707998197,
+      "ref_length": 636,
+      "hyp_length": 824,
+      "ref_digit_count": 73,
       "ref_cjk_count": 0
     },
     {
       "file": "0004.png",
-      "cer": 0.37906137184115524,
-      "wer": 0.36486486486486486,
-      "digit_cer": 0.1016949152542373,
+      "category": "english_document",
+      "cer": 0.07862407862407862,
+      "wer": 0.16901408450704225,
+      "digit_cer": 0.23636363636363636,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.64588491700124,
-      "ref_length": 1329,
-      "hyp_length": 1710,
-      "ref_digit_count": 59,
+      "elapsed_seconds": 14.294532250001794,
+      "ref_length": 477,
+      "hyp_length": 476,
+      "ref_digit_count": 55,
       "ref_cjk_count": 0
     },
     {
       "file": "0005.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 3.0224843750038417,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 12.845850040990626,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0006.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 2.906020250011352,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 12.608022582993726,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0007.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 2.9044155000010505,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 12.634055832997547,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0008.png",
+      "category": "cjk_document",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.5957032919977792,
-      "ref_length": 92,
-      "hyp_length": 92,
-      "ref_digit_count": 26,
+      "elapsed_seconds": 7.23521095799515,
+      "ref_length": 98,
+      "hyp_length": 98,
+      "ref_digit_count": 30,
       "ref_cjk_count": 41
     },
     {
       "file": "0009.png",
+      "category": "cjk_document",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.5902405830129283,
-      "ref_length": 93,
-      "hyp_length": 93,
-      "ref_digit_count": 25,
+      "elapsed_seconds": 7.373159375012619,
+      "ref_length": 100,
+      "hyp_length": 100,
+      "ref_digit_count": 30,
       "ref_cjk_count": 42
     },
     {
       "file": "0010.png",
-      "cer": 0.6309523809523809,
-      "wer": 1.0909090909090908,
+      "category": "cjk_document",
+      "cer": 0.5604395604395604,
+      "wer": 1.2727272727272727,
       "digit_cer": 0.0,
       "cjk_cer": 1.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.6077932920015883,
-      "ref_length": 105,
-      "hyp_length": 117,
-      "ref_digit_count": 25,
+      "elapsed_seconds": 7.378904458004399,
+      "ref_length": 112,
+      "hyp_length": 126,
+      "ref_digit_count": 30,
       "ref_cjk_count": 44
     },
     {
       "file": "0011.png",
+      "category": "low_quality_scan",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.4870185839972692,
-      "ref_length": 118,
-      "hyp_length": 118,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 6.398878874999355,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
       "ref_cjk_count": 0
     },
     {
       "file": "0012.png",
+      "category": "low_quality_scan",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.4985078750032699,
-      "ref_length": 118,
-      "hyp_length": 118,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 6.583708875012235,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
       "ref_cjk_count": 0
     }
   ]

release/calibration_affine8_accuracy.json

@@ -0,0 +1,221 @@
+{
+  "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-affine8",
+  "served_revision": null,
+  "backend": "mlx",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 1024,
+  "profile": "accurate",
+  "generation_settings": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "repetition_penalty": 1.0,
+    "no_repeat_ngram_size": 35,
+    "ngram_window": 128
+  },
+  "num_images": 10,
+  "num_samples": 10,
+  "total_time_seconds": 121.92750062300183,
+  "mean_cer": 0.19105830313811087,
+  "mean_wer": 0.647581477892157,
+  "num_digit_samples": 10,
+  "num_cjk_samples": 3,
+  "mean_digit_cer": 0.07857558139534884,
+  "mean_cjk_cer": 0.3333333333333333,
+  "num_table_samples": 3,
+  "mean_table_score": 1.0,
+  "mean_repetition_rate": 0.0,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 0.18315018315018317,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": 0.3333333333333333,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 2,
+      "mean_cer": 0.4174682910791748,
+      "mean_digit_cer": 0.11162790697674418,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.0,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
+  "per_file": [
+    {
+      "file": "0001.png",
+      "category": "english_document",
+      "cer": 0.08904719501335707,
+      "wer": 0.24663677130044842,
+      "digit_cer": 0.023255813953488372,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 22.065639082997222,
+      "ref_length": 1345,
+      "hyp_length": 1259,
+      "ref_digit_count": 86,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0002.png",
+      "category": "english_document",
+      "cer": 0.7458893871449925,
+      "wer": 0.592814371257485,
+      "digit_cer": 0.2,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 31.2265478749905,
+      "ref_length": 835,
+      "hyp_length": 1275,
+      "ref_digit_count": 95,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0003.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.632652583008166,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0004.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.664347833007923,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0005.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.887310291000176,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0006.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.9882367919926764,
+      "ref_length": 98,
+      "hyp_length": 98,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 41
+    },
+    {
+      "file": "0007.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.911115791008342,
+      "ref_length": 100,
+      "hyp_length": 100,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 42
+    },
+    {
+      "file": "0008.png",
+      "category": "cjk_document",
+      "cer": 0.5494505494505495,
+      "wer": 1.1363636363636365,
+      "digit_cer": 0.0,
+      "cjk_cer": 1.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 7.001193958989461,
+      "ref_length": 112,
+      "hyp_length": 120,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 44
+    },
+    {
+      "file": "0009.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.5204695830034325,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0010.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.029986833003932,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    }
+  ]
+}

release/calibration_affine8_performance.json

@@ -0,0 +1,69 @@
+{
+  "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-affine8",
+  "served_revision": null,
+  "image_path": "test_invoice.png",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 256,
+  "system": {
+    "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
+    "processor": "arm",
+    "python_version": "3.14.6",
+    "machine": "arm64",
+    "mlx_version": "0.32.0",
+    "mlx_vlm_version": "0.6.6",
+    "chip": "Apple M3 Max",
+    "total_memory_gb": 128.0
+  },
+  "model_load_time_seconds": 1.0313333749945741,
+  "memory_after_load_mb": 4098.546875,
+  "num_warmup": 1,
+  "num_runs": 3,
+  "mean_tps": 39.58614991640946,
+  "std_tps": 0.7627654762339389,
+  "mean_elapsed_seconds": 8.051595569670704,
+  "mean_peak_memory_mb": 5275.928047,
+  "runs": [
+    {
+      "run": 1,
+      "elapsed_seconds": 7.985530750011094,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 40.03117010793416,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5275.752407,
+      "memory_delta_mb": 1337.1867550000002,
+      "process_peak_rss_mb": 4120.59375,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 453.0235706986471,
+      "finish_reason": "length"
+    },
+    {
+      "run": 2,
+      "elapsed_seconds": 7.846619042000384,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 40.21463014341998,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5276.015867,
+      "memory_delta_mb": 1337.188071,
+      "process_peak_rss_mb": 4121.078125,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 487.66087944129447,
+      "finish_reason": "length"
+    },
+    {
+      "run": 3,
+      "elapsed_seconds": 8.322636917000636,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 38.51264949787424,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5276.015867,
+      "memory_delta_mb": 1337.188071,
+      "process_peak_rss_mb": 4121.21875,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 423.58096065031117,
+      "finish_reason": "length"
+    }
+  ]
+}

release/calibration_baseline_accuracy.json

@@ -0,0 +1,221 @@
+{
+  "model_path": "Unlimited-OCR",
+  "served_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+  "backend": "mlx",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 1024,
+  "profile": "accurate",
+  "generation_settings": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "repetition_penalty": 1.0,
+    "no_repeat_ngram_size": 35,
+    "ngram_window": 128
+  },
+  "num_images": 10,
+  "num_samples": 10,
+  "total_time_seconds": 121.70595645793946,
+  "mean_cer": 0.20230911688823577,
+  "mean_wer": 0.6426630229146273,
+  "num_digit_samples": 10,
+  "num_cjk_samples": 3,
+  "mean_digit_cer": 0.07236995104039168,
+  "mean_cjk_cer": 0.3333333333333333,
+  "num_table_samples": 3,
+  "mean_table_score": 1.0,
+  "mean_repetition_rate": 0.0,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 0.18681318681318682,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": 0.3333333333333333,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 2,
+      "mean_cer": 0.46822785433529396,
+      "mean_digit_cer": 0.08059975520195839,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.0,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
+  "per_file": [
+    {
+      "file": "0001.png",
+      "category": "english_document",
+      "cer": 0.02315227070347284,
+      "wer": 0.19282511210762332,
+      "digit_cer": 0.03488372093023256,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 23.124094749990036,
+      "ref_length": 1345,
+      "hyp_length": 1348,
+      "ref_digit_count": 86,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0002.png",
+      "category": "english_document",
+      "cer": 0.9133034379671151,
+      "wer": 0.46107784431137727,
+      "digit_cer": 0.12631578947368421,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 31.42361595899274,
+      "ref_length": 835,
+      "hyp_length": 1378,
+      "ref_digit_count": 95,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0003.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.507403457988403,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0004.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.631738583993865,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0005.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.604318874989985,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0006.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.63977562499349,
+      "ref_length": 98,
+      "hyp_length": 98,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 41
+    },
+    {
+      "file": "0007.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.620961624998017,
+      "ref_length": 100,
+      "hyp_length": 100,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 42
+    },
+    {
+      "file": "0008.png",
+      "category": "cjk_document",
+      "cer": 0.5604395604395604,
+      "wer": 1.2727272727272727,
+      "digit_cer": 0.0,
+      "cjk_cer": 1.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.897685666001053,
+      "ref_length": 112,
+      "hyp_length": 126,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 44
+    },
+    {
+      "file": "0009.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.136604707993683,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0010.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.119757207998191,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    }
+  ]
+}

release/calibration_bfloat16_accuracy.json

@@ -0,0 +1,221 @@
+{
+  "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-bfloat16",
+  "served_revision": null,
+  "backend": "mlx",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 1024,
+  "profile": "accurate",
+  "generation_settings": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "repetition_penalty": 1.0,
+    "no_repeat_ngram_size": 35,
+    "ngram_window": 128
+  },
+  "num_images": 10,
+  "num_samples": 10,
+  "total_time_seconds": 130.78621862299042,
+  "mean_cer": 0.19010066585760363,
+  "mean_wer": 0.6393071175219882,
+  "num_digit_samples": 10,
+  "num_cjk_samples": 3,
+  "mean_digit_cer": 0.06278610771113831,
+  "mean_cjk_cer": 0.3333333333333333,
+  "num_table_samples": 3,
+  "mean_table_score": 1.0,
+  "mean_repetition_rate": 0.0,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 0.16849816849816848,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": 0.3333333333333333,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 2,
+      "mean_cer": 0.43465812665466064,
+      "mean_digit_cer": 0.03268053855569156,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.0,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
+  "per_file": [
+    {
+      "file": "0001.png",
+      "category": "english_document",
+      "cer": 0.08904719501335707,
+      "wer": 0.24663677130044842,
+      "digit_cer": 0.023255813953488372,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 23.43824337499973,
+      "ref_length": 1345,
+      "hyp_length": 1259,
+      "ref_digit_count": 86,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0002.png",
+      "category": "english_document",
+      "cer": 0.7802690582959642,
+      "wer": 0.41916167664670656,
+      "digit_cer": 0.042105263157894736,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 34.438602624999476,
+      "ref_length": 835,
+      "hyp_length": 1285,
+      "ref_digit_count": 95,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0003.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 13.494044040999142,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0004.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 12.677681540997582,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0005.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 12.359245291998377,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0006.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 7.067715750003117,
+      "ref_length": 98,
+      "hyp_length": 98,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 41
+    },
+    {
+      "file": "0007.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 7.551363666992984,
+      "ref_length": 100,
+      "hyp_length": 100,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 42
+    },
+    {
+      "file": "0008.png",
+      "category": "cjk_document",
+      "cer": 0.5054945054945055,
+      "wer": 1.2272727272727273,
+      "digit_cer": 0.0,
+      "cjk_cer": 1.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.661412207991816,
+      "ref_length": 112,
+      "hyp_length": 118,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 44
+    },
+    {
+      "file": "0009.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.571904916010681,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0010.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.526005207997514,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    }
+  ]
+}

release/calibration_bfloat16_performance.json

@@ -0,0 +1,69 @@
+{
+  "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-bfloat16",
+  "served_revision": null,
+  "image_path": "test_invoice.png",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 256,
+  "system": {
+    "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
+    "processor": "arm",
+    "python_version": "3.14.6",
+    "machine": "arm64",
+    "mlx_version": "0.32.0",
+    "mlx_vlm_version": "0.6.6",
+    "chip": "Apple M3 Max",
+    "total_memory_gb": 128.0
+  },
+  "model_load_time_seconds": 0.97142708300089,
+  "memory_after_load_mb": 4240.671875,
+  "num_warmup": 1,
+  "num_runs": 3,
+  "mean_tps": 38.770309481806514,
+  "std_tps": 0.6887167709523151,
+  "mean_elapsed_seconds": 8.391549889337815,
+  "mean_peak_memory_mb": 5420.697071,
+  "runs": [
+    {
+      "run": 1,
+      "elapsed_seconds": 8.60731708399544,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 38.03899778227293,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5420.521431,
+      "memory_delta_mb": 1337.1867550000002,
+      "process_peak_rss_mb": 4253.21875,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 380.356298056712,
+      "finish_reason": "length"
+    },
+    {
+      "run": 2,
+      "elapsed_seconds": 8.399659084010636,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 38.57884453275097,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5420.784891,
+      "memory_delta_mb": 1337.188071,
+      "process_peak_rss_mb": 4260.28125,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 401.47375057984203,
+      "finish_reason": "length"
+    },
+    {
+      "run": 3,
+      "elapsed_seconds": 8.167673500007368,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 39.693086130395656,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5420.784891,
+      "memory_delta_mb": 1337.188071,
+      "process_peak_rss_mb": 4260.46875,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 411.1295077060184,
+      "finish_reason": "length"
+    }
+  ]
+}

release/calibration_mxfp8_accuracy.json

@@ -0,0 +1,221 @@
+{
+  "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-mxfp8",
+  "served_revision": null,
+  "backend": "mlx",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 1024,
+  "profile": "accurate",
+  "generation_settings": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "repetition_penalty": 1.0,
+    "no_repeat_ngram_size": 35,
+    "ngram_window": 128
+  },
+  "num_images": 10,
+  "num_samples": 10,
+  "total_time_seconds": 120.2266979579872,
+  "mean_cer": 0.21607021200652196,
+  "mean_wer": 0.7193288921545413,
+  "num_digit_samples": 10,
+  "num_cjk_samples": 3,
+  "mean_digit_cer": 0.1052422480620155,
+  "mean_cjk_cer": 0.3333333333333333,
+  "num_table_samples": 3,
+  "mean_table_score": 1.0,
+  "mean_repetition_rate": 0.0,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 0.2490842490842491,
+      "mean_digit_cer": 0.08888888888888889,
+      "mean_cjk_cer": 0.3333333333333333,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 2,
+      "mean_cer": 0.44362673652013146,
+      "mean_digit_cer": 0.11162790697674418,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.0,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
+  "per_file": [
+    {
+      "file": "0001.png",
+      "category": "english_document",
+      "cer": 0.08904719501335707,
+      "wer": 0.24663677130044842,
+      "digit_cer": 0.023255813953488372,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 21.754243709001457,
+      "ref_length": 1345,
+      "hyp_length": 1259,
+      "ref_digit_count": 86,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0002.png",
+      "category": "english_document",
+      "cer": 0.7982062780269058,
+      "wer": 0.40119760479041916,
+      "digit_cer": 0.2,
+      "cjk_cer": 1.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 30.940298208006425,
+      "ref_length": 835,
+      "hyp_length": 1293,
+      "ref_digit_count": 95,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0003.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.853719249993446,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0004.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.799329292000039,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0005.png",
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
+      "wer": 1.5,
+      "digit_cer": 0.1875,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": 1.0,
+      "elapsed_seconds": 11.05323970799509,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0006.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.7778485829912825,
+      "ref_length": 98,
+      "hyp_length": 98,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 41
+    },
+    {
+      "file": "0007.png",
+      "category": "cjk_document",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.7498448330006795,
+      "ref_length": 100,
+      "hyp_length": 100,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 42
+    },
+    {
+      "file": "0008.png",
+      "category": "cjk_document",
+      "cer": 0.7472527472527473,
+      "wer": 2.0454545454545454,
+      "digit_cer": 0.26666666666666666,
+      "cjk_cer": 1.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.956620541997836,
+      "ref_length": 112,
+      "hyp_length": 146,
+      "ref_digit_count": 30,
+      "ref_cjk_count": 44
+    },
+    {
+      "file": "0009.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.085180042005959,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    },
+    {
+      "file": "0010.png",
+      "category": "low_quality_scan",
+      "cer": 0.0,
+      "wer": 0.0,
+      "digit_cer": 0.0,
+      "cjk_cer": 0.0,
+      "repetition_rate": 0.0,
+      "table_score": null,
+      "elapsed_seconds": 6.256373790994985,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
+      "ref_cjk_count": 0
+    }
+  ]
+}

release/calibration_mxfp8_performance.json

@@ -0,0 +1,69 @@
+{
+  "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-mxfp8",
+  "served_revision": null,
+  "image_path": "test_invoice.png",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 256,
+  "system": {
+    "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
+    "processor": "arm",
+    "python_version": "3.14.6",
+    "machine": "arm64",
+    "mlx_version": "0.32.0",
+    "mlx_vlm_version": "0.6.6",
+    "chip": "Apple M3 Max",
+    "total_memory_gb": 128.0
+  },
+  "model_load_time_seconds": 1.0123953330039512,
+  "memory_after_load_mb": 4115.359375,
+  "num_warmup": 1,
+  "num_runs": 3,
+  "mean_tps": 41.326731942543184,
+  "std_tps": 1.2149507723608595,
+  "mean_elapsed_seconds": 7.814036472002044,
+  "mean_peak_memory_mb": 5260.396014999999,
+  "runs": [
+    {
+      "run": 1,
+      "elapsed_seconds": 7.896335166005883,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 41.34150964421114,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5260.220375,
+      "memory_delta_mb": 1337.1867549999997,
+      "process_peak_rss_mb": 4129.09375,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 421.22509117788985,
+      "finish_reason": "length"
+    },
+    {
+      "run": 2,
+      "elapsed_seconds": 7.41755587499938,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 42.80729278290862,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5260.483835,
+      "memory_delta_mb": 1337.188071,
+      "process_peak_rss_mb": 4137.328125,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 496.01062497025777,
+      "finish_reason": "length"
+    },
+    {
+      "run": 3,
+      "elapsed_seconds": 8.12821837500087,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 39.83139340050979,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5260.483835,
+      "memory_delta_mb": 1337.188071,
+      "process_peak_rss_mb": 4137.578125,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 415.31823155489394,
+      "finish_reason": "length"
+    }
+  ]
+}

release/calibration_reference_performance.json

@@ -0,0 +1,69 @@
+{
+  "model_path": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+  "served_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
+  "image_path": "test_invoice.png",
+  "prompt": "<image>document parsing.",
+  "max_tokens": 256,
+  "system": {
+    "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
+    "processor": "arm",
+    "python_version": "3.14.6",
+    "machine": "arm64",
+    "mlx_version": "0.32.0",
+    "mlx_vlm_version": "0.6.6",
+    "chip": "Apple M3 Max",
+    "total_memory_gb": 128.0
+  },
+  "model_load_time_seconds": 5.673565958000836,
+  "memory_after_load_mb": 3994.578125,
+  "num_warmup": 1,
+  "num_runs": 3,
+  "mean_tps": 40.402663236114016,
+  "std_tps": 0.3014552552023151,
+  "mean_elapsed_seconds": 8.03746888867075,
+  "mean_peak_memory_mb": 5165.991419999999,
+  "runs": [
+    {
+      "run": 1,
+      "elapsed_seconds": 8.067712333009695,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 40.079502760011906,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5165.81578,
+      "memory_delta_mb": 1337.186768,
+      "process_peak_rss_mb": 4016.75,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 422.0660830294658,
+      "finish_reason": "length"
+    },
+    {
+      "run": 2,
+      "elapsed_seconds": 8.119986582998536,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 40.323435695329465,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5166.07924,
+      "memory_delta_mb": 1337.188084,
+      "process_peak_rss_mb": 4025.171875,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 399.62688128036274,
+      "finish_reason": "length"
+    },
+    {
+      "run": 3,
+      "elapsed_seconds": 7.924707750004018,
+      "tokens_generated": 256,
+      "tokens_generated_source": "mlx-vlm token count",
+      "tokens_per_second": 40.80505125300066,
+      "tokens_per_second_source": "mlx-vlm generation_tps",
+      "peak_memory_mb": 5166.07924,
+      "memory_delta_mb": 1337.188084,
+      "process_peak_rss_mb": 4025.484375,
+      "prompt_tokens": 697,
+      "prompt_tokens_per_second": 429.46142333454605,
+      "finish_reason": "length"
+    }
+  ]
+}

release/calibration_results.json

@@ -1,38 +1,40 @@
 {
-  "schema_version": 1,
-  "created_at": "2026-07-20T23:25:55.306169+00:00",
+  "schema_version": 3,
+  "created_at": "2026-07-21T05:26:12.430585+00:00",
   "target_pattern": "language_model.lm_head",
   "selection_policy": "fastest candidate passing existing quality and throughput limits",
   "thresholds": {
-    "max_cer_delta_vs_bf16": 0.01,
+    "max_cer_delta_vs_bf16": 0.015,
     "max_cer_delta_vs_reference": 0.005,
     "max_digit_cer_delta_vs_bf16": 0.01,
     "max_table_score_degradation_vs_bf16": 0.01,
     "min_tps_ratio_vs_reference": 0.9,
-    "max_weight_size_gb": 4.5
+    "max_weight_size_gb": 4.5,
+    "min_rswa_tokens": 8192,
+    "max_rswa_repetition_rate": 0.25
   },
   "experiments": [
     {
       "label": "bf16-head",
       "precision": "bfloat16",
-      "passed": false,
+      "passed": true,
       "checks": {
         "cer": true,
         "digit_cer": true,
         "table_score": true,
-        "throughput": false
+        "throughput": true
       },
       "metrics": {
-        "mean_cer": 0.23190356458231498,
-        "mean_digit_cer": 0.08222376405546196,
+        "mean_cer": 0.19010066585760363,
+        "mean_digit_cer": 0.06278610771113831,
         "mean_table_score": 1.0,
-        "mean_tps": 271.0950821181683
+        "mean_tps": 38.770309481806514
       },
       "deltas": {
-        "cer_vs_bf16": -0.007284148710181432,
-        "digit_cer_vs_bf16": -0.005116203514816328,
+        "cer_vs_bf16": -0.012208451030632145,
+        "digit_cer_vs_bf16": -0.00958384332925337,
         "table_degradation_vs_bf16": 0.0,
-        "tps_ratio_vs_reference": 0.8668704848166195
+        "tps_ratio_vs_reference": 0.9595978674779929
       }
     },
     {
@@ -40,22 +42,22 @@
       "precision": "mxfp8",
       "passed": false,
       "checks": {
-        "cer": false,
-        "digit_cer": true,
+        "cer": true,
+        "digit_cer": false,
         "table_score": true,
         "throughput": true
       },
       "metrics": {
-        "mean_cer": 0.26017003915600867,
-        "mean_digit_cer": 0.0769606061607251,
+        "mean_cer": 0.21607021200652196,
+        "mean_digit_cer": 0.1052422480620155,
         "mean_table_score": 1.0,
-        "mean_tps": 307.3938836867251
+        "mean_tps": 41.326731942543184
       },
       "deltas": {
-        "cer_vs_bf16": 0.020982325863512258,
-        "digit_cer_vs_bf16": -0.010379361409553187,
+        "cer_vs_bf16": 0.013761095118286187,
+        "digit_cer_vs_bf16": 0.03287229702162382,
         "table_degradation_vs_bf16": 0.0,
-        "tps_ratio_vs_reference": 0.9829417889071937
+        "tps_ratio_vs_reference": 1.0228714800563738
       }
     },
     {
@@ -69,16 +71,16 @@
         "throughput": true
       },
       "metrics": {
-        "mean_cer": 0.23738438172232493,
-        "mean_digit_cer": 0.08573253598528652,
+        "mean_cer": 0.19105830313811087,
+        "mean_digit_cer": 0.07857558139534884,
         "mean_table_score": 1.0,
-        "mean_tps": 301.24796539578256
+        "mean_tps": 39.58614991640946
       },
       "deltas": {
-        "cer_vs_bf16": -0.0018033315701714847,
-        "digit_cer_vs_bf16": -0.0016074315849917697,
+        "cer_vs_bf16": -0.011250813750124905,
+        "digit_cer_vs_bf16": 0.0062056303549571645,
         "table_degradation_vs_bf16": 0.0,
-        "tps_ratio_vs_reference": 0.9632892185732532
+        "tps_ratio_vs_reference": 0.979790606502031
       }
     }
   ],
@@ -88,5 +90,164 @@
   },
   "precision_overrides": {
     "language_model.lm_head": "affine8"
+  },
+  "input_artifacts": {
+    "calibration_baseline_accuracy": {
+      "filename": "calibration_baseline_accuracy.json",
+      "size": 5680,
+      "sha256": "ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6"
+    },
+    "calibration_reference_performance": {
+      "filename": "calibration_reference_performance.json",
+      "size": 2271,
+      "sha256": "98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4"
+    },
+    "calibration_bfloat16_accuracy": {
+      "filename": "calibration_bfloat16_accuracy.json",
+      "size": 5678,
+      "sha256": "5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0"
+    },
+    "calibration_bfloat16_performance": {
+      "filename": "calibration_bfloat16_performance.json",
+      "size": 2244,
+      "sha256": "dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0"
+    },
+    "calibration_mxfp8_accuracy": {
+      "filename": "calibration_mxfp8_accuracy.json",
+      "size": 5689,
+      "sha256": "77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26"
+    },
+    "calibration_mxfp8_performance": {
+      "filename": "calibration_mxfp8_performance.json",
+      "size": 2252,
+      "sha256": "e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b"
+    },
+    "calibration_affine8_accuracy": {
+      "filename": "calibration_affine8_accuracy.json",
+      "size": 5657,
+      "sha256": "c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19"
+    },
+    "calibration_affine8_performance": {
+      "filename": "calibration_affine8_performance.json",
+      "size": 2247,
+      "sha256": "f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f"
+    }
+  },
+  "dataset": {
+    "name": "calibration",
+    "num_samples": 10,
+    "num_digit_samples": 10,
+    "num_cjk_samples": 3,
+    "num_table_samples": 3,
+    "category_counts": {
+      "cjk_document": 3,
+      "english_document": 2,
+      "low_quality_scan": 2,
+      "table_numeric": 3
+    },
+    "manifest_sha256": "d4429669fe60a5c48ce82073294d20c845b85084644f9d4f62f7c8ca3923ba7a",
+    "content_sha256": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+    "files": [
+      {
+        "path": "images/0001.png",
+        "size": 243645,
+        "sha256": "8fad74c7e8f314075f34d5b3b3e6272a2c22a90e960dcf8074039b1f065895b4"
+      },
+      {
+        "path": "ground_truth/0001.txt",
+        "size": 1346,
+        "sha256": "a6f35a418836e8ba0667dc42d663219b07b65cbbf7a4627b2319f0fa44d11f8e"
+      },
+      {
+        "path": "images/0002.png",
+        "size": 208699,
+        "sha256": "31f2d9c651112e963366e5e39f3da4ff8198ec41e2187c3c8fb5fec0edd2f64c"
+      },
+      {
+        "path": "ground_truth/0002.txt",
+        "size": 838,
+        "sha256": "a05196b747f6898102fc56e23cae31c32754b985bc3eab0abe60d955bfadff68"
+      },
+      {
+        "path": "images/0003.png",
+        "size": 40039,
+        "sha256": "bbe26dc81fb4c5e6a143279dd1e0d0540df746f5e4e8f9fc7577e936882b190e"
+      },
+      {
+        "path": "ground_truth/0003.txt",
+        "size": 459,
+        "sha256": "39cf88aef99b6ce6b5d8af870e2feefca7a9991e5c835660bda60c5b5f80f481"
+      },
+      {
+        "path": "images/0004.png",
+        "size": 40356,
+        "sha256": "1e50bec7880f37e9e25eb8c4706998634b0c1f254d38fb75ccd6dbb9b89ed3a3"
+      },
+      {
+        "path": "ground_truth/0004.txt",
+        "size": 459,
+        "sha256": "2a2b723c4f1b4e14cdf5d70a74ae12586c12952ffac0d1bd295c448f1bf2d3fc"
+      },
+      {
+        "path": "images/0005.png",
+        "size": 40650,
+        "sha256": "7b7c9441ff17bb97ee76cb68eb6b9134f7d26d151c6ca1d18039959bf43de04c"
+      },
+      {
+        "path": "ground_truth/0005.txt",
+        "size": 459,
+        "sha256": "3dbec98a34736b424dafcf492691b21a1a36f629d83589cac5482cc1a135d9b7"
+      },
+      {
+        "path": "images/0006.png",
+        "size": 46036,
+        "sha256": "76006318e5aa3bb54182d7b3249ad016e7ab0dee387f580331c39b9ac49cb7ab"
+      },
+      {
+        "path": "ground_truth/0006.txt",
+        "size": 191,
+        "sha256": "75082837c3cbb8c41ba0747b98390a1f44577092c30f31f4c1e10ce52af83b61"
+      },
+      {
+        "path": "images/0007.png",
+        "size": 41828,
+        "sha256": "d642986c12233efe0bc7559096952e66eb175735f89ac91caedea0089c4adb19"
+      },
+      {
+        "path": "ground_truth/0007.txt",
+        "size": 195,
+        "sha256": "c47bf79076b7ab732c22dc1b023d9f6f12b98b724dd0d730c23e879ef7604e29"
+      },
+      {
+        "path": "images/0008.png",
+        "size": 20975,
+        "sha256": "852416613d0137144df77408c44e7d469249e7fbc650c684ad28911b96b28e76"
+      },
+      {
+        "path": "ground_truth/0008.txt",
+        "size": 201,
+        "sha256": "dc48f9c80e21438d5c2c3fb57b3bc471961708e2dd971df78a9ecc11c8c3b3ad"
+      },
+      {
+        "path": "images/0009.png",
+        "size": 487313,
+        "sha256": "87f977973bddf89f781762c87feb9c845e02731dae5cf39086db4cdb0588298e"
+      },
+      {
+        "path": "ground_truth/0009.txt",
+        "size": 121,
+        "sha256": "d9f04e21b8e73f964a9dee2776c65a76d509138f8abe694c46305a32833a52f1"
+      },
+      {
+        "path": "images/0010.png",
+        "size": 515820,
+        "sha256": "6fb40a4c63aed4cbb11346d0c80b917cf1ecd407b364bcfbc87483b5bd90d0b4"
+      },
+      {
+        "path": "ground_truth/0010.txt",
+        "size": 121,
+        "sha256": "b8e07f309069b395279995af437f82f3a9453f2add587bd9633b4ae97cc8419a"
+      }
+    ]
   }
 }

release/candidate_accuracy.json

@@ -1,5 +1,7 @@
 {
   "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+  "served_revision": null,
+  "backend": "mlx",
   "prompt": "<image>document parsing.",
   "max_tokens": 1024,
   "profile": "accurate",
@@ -12,183 +14,237 @@
   },
   "num_images": 12,
   "num_samples": 12,
-  "total_time_seconds": 31.473419997957535,
-  "mean_cer": 0.23738438172232493,
-  "mean_wer": 0.5707107552516416,
+  "total_time_seconds": 164.14354283601278,
+  "mean_cer": 0.21633091868789586,
+  "mean_wer": 0.6718835226038614,
   "num_digit_samples": 12,
   "num_cjk_samples": 3,
-  "mean_digit_cer": 0.08573253598528652,
-  "mean_cjk_cer": 0.3333333333333333,
+  "mean_digit_cer": 0.11193906322597019,
+  "mean_cjk_cer": 0.3412698412698412,
   "num_table_samples": 3,
   "mean_table_score": 1.0,
   "mean_repetition_rate": 0.0,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 0.2455035601103017,
+      "mean_digit_cer": 0.08888888888888889,
+      "mean_cjk_cer": 0.3412698412698412,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 4,
+      "mean_cer": 0.3333161110379089,
+      "mean_digit_cer": 0.12852552301124393,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 4,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.0,
+      "mean_digit_cer": 0.0,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
   "per_file": [
     {
       "file": "0001.png",
-      "cer": 0.020480854853072127,
-      "wer": 0.18385650224215247,
-      "digit_cer": 0.0,
+      "category": "english_document",
+      "cer": 0.3494191242180518,
+      "wer": 0.3317757009345794,
+      "digit_cer": 0.14814814814814814,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 3.815771791996667,
-      "ref_length": 1345,
-      "hyp_length": 1349,
-      "ref_digit_count": 86,
+      "elapsed_seconds": 23.19513687500148,
+      "ref_length": 1332,
+      "hyp_length": 1669,
+      "ref_digit_count": 27,
       "ref_cjk_count": 0
     },
     {
       "file": "0002.png",
-      "cer": 0.9327354260089686,
-      "wer": 0.5089820359281437,
-      "digit_cer": 0.17894736842105263,
+      "category": "english_document",
+      "cer": 0.37815884476534295,
+      "wer": 0.36486486486486486,
+      "digit_cer": 0.1016949152542373,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.953415083000436,
-      "ref_length": 835,
-      "hyp_length": 1310,
-      "ref_digit_count": 95,
+      "elapsed_seconds": 23.631959624995943,
+      "ref_length": 1329,
+      "hyp_length": 1708,
+      "ref_digit_count": 59,
       "ref_cjk_count": 0
     },
     {
       "file": "0003.png",
-      "cer": 0.3512064343163539,
-      "wer": 0.3317757009345794,
-      "digit_cer": 0.14814814814814814,
+      "category": "english_document",
+      "cer": 0.5393474088291746,
+      "wer": 0.5086206896551724,
+      "digit_cer": 0.136986301369863,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 3.5391229999950156,
-      "ref_length": 1332,
-      "hyp_length": 1672,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 25.065509457999724,
+      "ref_length": 636,
+      "hyp_length": 804,
+      "ref_digit_count": 73,
       "ref_cjk_count": 0
     },
     {
       "file": "0004.png",
-      "cer": 0.38176895306859204,
-      "wer": 0.36936936936936937,
-      "digit_cer": 0.1016949152542373,
+      "category": "english_document",
+      "cer": 0.06633906633906633,
+      "wer": 0.18309859154929578,
+      "digit_cer": 0.12727272727272726,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 3.735332540993113,
-      "ref_length": 1329,
-      "hyp_length": 1705,
-      "ref_digit_count": 59,
+      "elapsed_seconds": 18.39167737500975,
+      "ref_length": 477,
+      "hyp_length": 484,
+      "ref_digit_count": 55,
       "ref_cjk_count": 0
     },
     {
       "file": "0005.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 2.769828624994261,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 12.358092667011078,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0006.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 2.7706372079992434,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 12.517006292007864,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0007.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 2.726061583001865,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 12.23163116700016,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0008.png",
+      "category": "cjk_document",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.4497268339910079,
-      "ref_length": 92,
-      "hyp_length": 92,
-      "ref_digit_count": 26,
+      "elapsed_seconds": 7.415536249987781,
+      "ref_length": 98,
+      "hyp_length": 98,
+      "ref_digit_count": 30,
       "ref_cjk_count": 41
     },
     {
       "file": "0009.png",
-      "cer": 0.0,
-      "wer": 0.0,
+      "category": "cjk_document",
+      "cer": 0.011235955056179775,
+      "wer": 0.08333333333333333,
       "digit_cer": 0.0,
-      "cjk_cer": 0.0,
+      "cjk_cer": 0.023809523809523808,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.4638195829902543,
-      "ref_length": 93,
-      "hyp_length": 93,
-      "ref_digit_count": 25,
+      "elapsed_seconds": 8.263199041990447,
+      "ref_length": 100,
+      "hyp_length": 100,
+      "ref_digit_count": 30,
       "ref_cjk_count": 42
     },
     {
       "file": "0010.png",
-      "cer": 0.6309523809523809,
-      "wer": 0.9545454545454546,
-      "digit_cer": 0.0,
+      "category": "cjk_document",
+      "cer": 0.7252747252747253,
+      "wer": 2.090909090909091,
+      "digit_cer": 0.26666666666666666,
       "cjk_cer": 1.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.4698359579924727,
-      "ref_length": 105,
-      "hyp_length": 110,
-      "ref_digit_count": 25,
+      "elapsed_seconds": 7.9901230840041535,
+      "ref_length": 112,
+      "hyp_length": 147,
+      "ref_digit_count": 30,
       "ref_cjk_count": 44
     },
     {
       "file": "0011.png",
+      "category": "low_quality_scan",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.3830854999978328,
-      "ref_length": 118,
-      "hyp_length": 118,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 6.431838709002477,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
       "ref_cjk_count": 0
     },
     {
       "file": "0012.png",
+      "category": "low_quality_scan",
       "cer": 0.0,
       "wer": 0.0,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.3967822910053656,
-      "ref_length": 118,
-      "hyp_length": 118,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 6.651832292001927,
+      "ref_length": 120,
+      "hyp_length": 120,
+      "ref_digit_count": 29,
       "ref_cjk_count": 0
     }
   ]

release/candidate_performance.json

@@ -1,5 +1,6 @@
 {
   "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+  "served_revision": null,
   "image_path": "test_invoice.png",
   "prompt": "<image>document parsing.",
   "max_tokens": 256,
@@ -13,55 +14,55 @@
     "chip": "Apple M3 Max",
     "total_memory_gb": 128.0
   },
-  "model_load_time_seconds": 0.8421083340072073,
-  "memory_after_load_mb": 4008.296875,
+  "model_load_time_seconds": 0.9827843750099419,
+  "memory_after_load_mb": 4043.265625,
   "num_warmup": 1,
   "num_runs": 3,
-  "mean_tps": 297.2042493840031,
-  "std_tps": 3.7161843770763436,
-  "mean_elapsed_seconds": 1.2716583613364492,
-  "mean_peak_memory_mb": 5181.5234390000005,
+  "mean_tps": 35.14004923388335,
+  "std_tps": 0.8627581629341593,
+  "mean_elapsed_seconds": 8.941998055335716,
+  "mean_peak_memory_mb": 5184.062959,
   "runs": [
     {
       "run": 1,
-      "elapsed_seconds": 1.283014083004673,
+      "elapsed_seconds": 8.715698832995258,
       "tokens_generated": 256,
       "tokens_generated_source": "mlx-vlm token count",
-      "tokens_per_second": 293.6595631095472,
+      "tokens_per_second": 36.34797807809006,
       "tokens_per_second_source": "mlx-vlm generation_tps",
-      "peak_memory_mb": 5181.347799,
+      "peak_memory_mb": 5183.887319,
       "memory_delta_mb": 1337.1867550000002,
-      "process_peak_rss_mb": 4021.109375,
+      "process_peak_rss_mb": 4058.71875,
       "prompt_tokens": 697,
-      "prompt_tokens_per_second": 1793.8031467011067,
+      "prompt_tokens_per_second": 422.9274407258055,
       "finish_reason": "length"
     },
     {
       "run": 2,
-      "elapsed_seconds": 1.276783292007167,
+      "elapsed_seconds": 9.117345000006026,
       "tokens_generated": 256,
       "tokens_generated_source": "mlx-vlm token count",
-      "tokens_per_second": 295.6163188867776,
+      "tokens_per_second": 34.685109648852176,
       "tokens_per_second_source": "mlx-vlm generation_tps",
-      "peak_memory_mb": 5181.611259,
-      "memory_delta_mb": 1337.188071,
-      "process_peak_rss_mb": 4021.640625,
+      "peak_memory_mb": 5184.1507790000005,
+      "memory_delta_mb": 1337.1880710000005,
+      "process_peak_rss_mb": 4065.359375,
       "prompt_tokens": 697,
-      "prompt_tokens_per_second": 1787.3186918039762,
+      "prompt_tokens_per_second": 413.1939079712154,
       "finish_reason": "length"
     },
     {
       "run": 3,
-      "elapsed_seconds": 1.2551777089975076,
+      "elapsed_seconds": 8.992950333005865,
       "tokens_generated": 256,
       "tokens_generated_source": "mlx-vlm token count",
-      "tokens_per_second": 302.3368661556844,
+      "tokens_per_second": 34.38705997470784,
       "tokens_per_second_source": "mlx-vlm generation_tps",
-      "peak_memory_mb": 5181.611259,
-      "memory_delta_mb": 1337.188071,
-      "process_peak_rss_mb": 4021.90625,
+      "peak_memory_mb": 5184.1507790000005,
+      "memory_delta_mb": 1337.1880710000005,
+      "process_peak_rss_mb": 4065.5625,
       "prompt_tokens": 697,
-      "prompt_tokens_per_second": 1798.8601642526962,
+      "prompt_tokens_per_second": 463.0680271067021,
       "finish_reason": "length"
     }
   ]

release/candidate_rswa.json

@@ -3,23 +3,28 @@
   "image_path": "test_invoice.png",
   "prompt": "<image>document parsing.",
   "force_min_tokens": 8192,
+  "generation_settings": {
+    "temperature": 0.0,
+    "no_repeat_ngram_size": 35,
+    "ngram_window": 128
+  },
   "forced_eos_token_ids": [
     1
   ],
   "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
   "processor": "arm",
-  "baseline_memory_mb": 3844.161032,
+  "baseline_memory_mb": 3846.700552,
   "test_results": [
     {
       "max_tokens": 512,
       "tokens_generated": 512,
       "tokens_generated_source": "mlx-vlm token count",
       "tokens_generated_reliable": true,
-      "elapsed_seconds": 2.4322984160098713,
-      "mean_tps": 264.34124967086365,
+      "elapsed_seconds": 17.311166541010607,
+      "mean_tps": 32.76836342370835,
       "tps_source": "mlx-vlm generation_tps",
-      "peak_memory_mb": 5181.350593,
-      "memory_growth_mb": 1337.1895610000001,
+      "peak_memory_mb": 5183.890113,
+      "memory_growth_mb": 1337.1895610000006,
       "repetition_rate": 0.0,
       "completed_naturally": false,
       "finish_reason": "length",
@@ -31,12 +36,12 @@
       "tokens_generated": 2048,
       "tokens_generated_source": "mlx-vlm token count",
       "tokens_generated_reliable": true,
-      "elapsed_seconds": 8.234726583992597,
-      "mean_tps": 261.8525825264912,
+      "elapsed_seconds": 65.73962916599703,
+      "mean_tps": 32.0896585893278,
       "tps_source": "mlx-vlm generation_tps",
-      "peak_memory_mb": 5181.612737,
-      "memory_growth_mb": 1337.1895490000002,
-      "repetition_rate": 0.09565217391304348,
+      "peak_memory_mb": 5184.152257,
+      "memory_growth_mb": 1337.1895489999997,
+      "repetition_rate": 0.20496894409937888,
       "completed_naturally": false,
       "finish_reason": "length",
       "forced_minimum_tokens": 2048,
@@ -47,12 +52,12 @@
       "tokens_generated": 8192,
       "tokens_generated_source": "mlx-vlm token count",
       "tokens_generated_reliable": true,
-      "elapsed_seconds": 31.729143417003797,
-      "mean_tps": 261.56365755570715,
+      "elapsed_seconds": 268.4810879580036,
+      "mean_tps": 30.743604031277776,
       "tps_source": "mlx-vlm generation_tps",
-      "peak_memory_mb": 5181.612737,
-      "memory_growth_mb": 1337.1895490000002,
-      "repetition_rate": 0.09565217391304348,
+      "peak_memory_mb": 5184.152257,
+      "memory_growth_mb": 1337.1895489999997,
+      "repetition_rate": 0.20496894409937888,
       "completed_naturally": false,
       "finish_reason": "length",
       "forced_minimum_tokens": 8192,
@@ -64,7 +69,7 @@
     "cache_analysis_reason": "Peak memory stayed within 64 MB across a >=4x token span",
     "tps_stable_over_length": true,
     "max_successful_tokens": 8192,
-    "max_repetition_rate": 0.09565217391304348
+    "max_repetition_rate": 0.20496894409937888
   },
   "pass_conditions": {
     "cache_bounded": true,

release/generated_precision_map.json

@@ -1,6 +1,6 @@
 {
   "vision_model": "bfloat16",
-  "projector": "mxfp8",
+  "projector": "bfloat16",
   "language_model.model.embed_tokens": "mxfp8",
   "language_model.model.norm": "bfloat16",
   "language_model.lm_head": "affine8",
@@ -32,16 +32,16 @@
       "vision_encoder": {
         "precision": "bfloat16",
         "reasons": [
-          "cer_delta=2.423686",
-          "digit_cer_delta=1.358023",
+          "cer_delta=2.272233",
+          "digit_cer_delta=1.696454",
           "table_score_degradation=1.000000",
           "protected OCR-critical group"
         ]
       },
       "vision_projector": {
-        "precision": "mxfp8",
+        "precision": "bfloat16",
         "reasons": [
-          "all measured quality deltas were non-degrading"
+          "protected unless all measured quality deltas are non-degrading"
         ]
       },
       "token_embeddings": {

release/provenance.json

@@ -1,5 +1,5 @@
 {
-  "created_at": "2026-07-20T23:05:17.950531+00:00",
+  "created_at": "2026-07-21T03:08:54.605846+00:00",
   "python": "3.14.6",
   "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
   "machine": "arm64",
@@ -12,27 +12,29 @@
     "Pillow": "12.3.0"
   },
   "source_model": "baidu/Unlimited-OCR",
+  "source_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
   "source_local_name": "Unlimited-OCR",
   "source_config": {
     "path": "config.json",
     "sha256": "27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9"
   },
   "reference_model": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+  "reference_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
   "target_repo": "AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
-  "dataset": {
-    "name": "eval_dataset",
-    "num_samples": 12,
-    "num_digit_samples": 12,
+  "calibration_dataset": {
+    "name": "calibration",
+    "num_samples": 10,
+    "num_digit_samples": 10,
     "num_cjk_samples": 3,
     "num_table_samples": 3,
     "category_counts": {
       "cjk_document": 3,
-      "english_document": 4,
+      "english_document": 2,
       "low_quality_scan": 2,
       "table_numeric": 3
     },
-    "manifest_sha256": "94d96ec344b02b8c4b413eafe524d6844bddcb00705af4b3fccea4c297c22509",
-    "content_sha256": "78ca3e7c9c7584aea30b04176716a6dadf49ccf0f2bead959cbaeb9aeac68f30",
+    "manifest_sha256": "d4429669fe60a5c48ce82073294d20c845b85084644f9d4f62f7c8ca3923ba7a",
+    "content_sha256": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
     "files": [
       {
         "path": "images/0001.png",
@@ -56,103 +58,220 @@
       },
       {
         "path": "images/0003.png",
+        "size": 40039,
+        "sha256": "bbe26dc81fb4c5e6a143279dd1e0d0540df746f5e4e8f9fc7577e936882b190e"
+      },
+      {
+        "path": "ground_truth/0003.txt",
+        "size": 459,
+        "sha256": "39cf88aef99b6ce6b5d8af870e2feefca7a9991e5c835660bda60c5b5f80f481"
+      },
+      {
+        "path": "images/0004.png",
+        "size": 40356,
+        "sha256": "1e50bec7880f37e9e25eb8c4706998634b0c1f254d38fb75ccd6dbb9b89ed3a3"
+      },
+      {
+        "path": "ground_truth/0004.txt",
+        "size": 459,
+        "sha256": "2a2b723c4f1b4e14cdf5d70a74ae12586c12952ffac0d1bd295c448f1bf2d3fc"
+      },
+      {
+        "path": "images/0005.png",
+        "size": 40650,
+        "sha256": "7b7c9441ff17bb97ee76cb68eb6b9134f7d26d151c6ca1d18039959bf43de04c"
+      },
+      {
+        "path": "ground_truth/0005.txt",
+        "size": 459,
+        "sha256": "3dbec98a34736b424dafcf492691b21a1a36f629d83589cac5482cc1a135d9b7"
+      },
+      {
+        "path": "images/0006.png",
+        "size": 46036,
+        "sha256": "76006318e5aa3bb54182d7b3249ad016e7ab0dee387f580331c39b9ac49cb7ab"
+      },
+      {
+        "path": "ground_truth/0006.txt",
+        "size": 191,
+        "sha256": "75082837c3cbb8c41ba0747b98390a1f44577092c30f31f4c1e10ce52af83b61"
+      },
+      {
+        "path": "images/0007.png",
+        "size": 41828,
+        "sha256": "d642986c12233efe0bc7559096952e66eb175735f89ac91caedea0089c4adb19"
+      },
+      {
+        "path": "ground_truth/0007.txt",
+        "size": 195,
+        "sha256": "c47bf79076b7ab732c22dc1b023d9f6f12b98b724dd0d730c23e879ef7604e29"
+      },
+      {
+        "path": "images/0008.png",
+        "size": 20975,
+        "sha256": "852416613d0137144df77408c44e7d469249e7fbc650c684ad28911b96b28e76"
+      },
+      {
+        "path": "ground_truth/0008.txt",
+        "size": 201,
+        "sha256": "dc48f9c80e21438d5c2c3fb57b3bc471961708e2dd971df78a9ecc11c8c3b3ad"
+      },
+      {
+        "path": "images/0009.png",
+        "size": 487313,
+        "sha256": "87f977973bddf89f781762c87feb9c845e02731dae5cf39086db4cdb0588298e"
+      },
+      {
+        "path": "ground_truth/0009.txt",
+        "size": 121,
+        "sha256": "d9f04e21b8e73f964a9dee2776c65a76d509138f8abe694c46305a32833a52f1"
+      },
+      {
+        "path": "images/0010.png",
+        "size": 515820,
+        "sha256": "6fb40a4c63aed4cbb11346d0c80b917cf1ecd407b364bcfbc87483b5bd90d0b4"
+      },
+      {
+        "path": "ground_truth/0010.txt",
+        "size": 121,
+        "sha256": "b8e07f309069b395279995af437f82f3a9453f2add587bd9633b4ae97cc8419a"
+      }
+    ]
+  },
+  "evaluation_dataset": {
+    "name": "evaluation",
+    "num_samples": 12,
+    "num_digit_samples": 12,
+    "num_cjk_samples": 3,
+    "num_table_samples": 3,
+    "category_counts": {
+      "cjk_document": 3,
+      "english_document": 4,
+      "low_quality_scan": 2,
+      "table_numeric": 3
+    },
+    "manifest_sha256": "111646a6f59ed32689fbc658c1cb4ed60629e1bbd909d6137df469e7b489f121",
+    "content_sha256": "7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987",
+    "files": [
+      {
+        "path": "images/0001.png",
         "size": 181040,
         "sha256": "40330138b25a7ce9bf142a672ad0a6253517bf076af66591c4431490eff1ecfc"
       },
       {
-        "path": "ground_truth/0003.txt",
+        "path": "ground_truth/0001.txt",
         "size": 1333,
         "sha256": "517ab5af1c090227d4dc2473ac422ecd9301e2e21ba0cc1c54d08476b1903d7e"
       },
       {
-        "path": "images/0004.png",
+        "path": "images/0002.png",
         "size": 220208,
         "sha256": "4316ba5ebe86c62331361e6d23329adede5e9eecf50a70db16b5439d95f5e23e"
       },
       {
-        "path": "ground_truth/0004.txt",
+        "path": "ground_truth/0002.txt",
         "size": 1330,
         "sha256": "df3b4afaee0f3ffe0984c2f1d1201c1762bc70f4ffbc1bfe9b1242e17d794148"
       },
+      {
+        "path": "images/0003.png",
+        "size": 119613,
+        "sha256": "bcf22c94459b970f1ee11088d8a902dbc56975e487c489b4714dedf8f3a4dca1"
+      },
+      {
+        "path": "ground_truth/0003.txt",
+        "size": 637,
+        "sha256": "a035ed288c8cc0e73e6cddc08c804be9b1ed32eee98e7f85c0e027bd88221b66"
+      },
+      {
+        "path": "images/0004.png",
+        "size": 67701,
+        "sha256": "baea8365beb81cc9939e59d3c8a0b88ab17999122eb1729e33214450a4fba987"
+      },
+      {
+        "path": "ground_truth/0004.txt",
+        "size": 478,
+        "sha256": "af1c29967e55595362ab41e25000e00167474cbe38ea43b8816f1f1395ef6e72"
+      },
       {
         "path": "images/0005.png",
-        "size": 38984,
-        "sha256": "1532b86fe28d7331379c73b62cfb0b0ae1448824566684021df262e4741b06a4"
+        "size": 41300,
+        "sha256": "65662634fc3b5df7d006f40d276fb2869809a86a2027f464c1d6776094e16f16"
       },
       {
         "path": "ground_truth/0005.txt",
-        "size": 449,
-        "sha256": "fb8ebea089dacff057d98feb6450436f3f6db65f0bac29513d3417e94451686e"
+        "size": 459,
+        "sha256": "1d565459641aca5056fdcccabe2cdc6834107bef219645a54cd28db2e06b05ea"
       },
       {
         "path": "images/0006.png",
-        "size": 38512,
-        "sha256": "8c1bf51f86f0f22c8898cec8c77049b6f023b77aab77f55be40f75b477ab6383"
+        "size": 40574,
+        "sha256": "67dbd4d07cb6e3f971937dc93a35ac81775580ee4d8895373afe6ae7e6b328dd"
       },
       {
         "path": "ground_truth/0006.txt",
-        "size": 449,
-        "sha256": "f60ff098ac431b1021fb85f810dd6264c1c021fab0810ab97ce304c5eb802598"
+        "size": 459,
+        "sha256": "0e134815b0bee585e3841d5a22a296c136fb43f00a9e3d48d98d2df24e884b08"
       },
       {
         "path": "images/0007.png",
-        "size": 38911,
-        "sha256": "a87e7f53196e9d3f72af611394297ec64e9eb222d2872b62f0c5a2014d83ecfc"
+        "size": 41027,
+        "sha256": "2b54b5222174bf4d41d92a6ee97aa1b2a3e57daca987b55906793df807dfe08f"
       },
       {
         "path": "ground_truth/0007.txt",
-        "size": 449,
-        "sha256": "0381ec16f8895290dfc2b81f97933f5568a68bfe731cc4791272ff8269e57c0e"
+        "size": 459,
+        "sha256": "b63a7c471f4d36cc8bafddcbd7b4a8d92d6f92e73ab1e260505c2413d6e39146"
       },
       {
         "path": "images/0008.png",
-        "size": 45665,
-        "sha256": "d9ff0cc9ffa639e485b0d445e1f4856f479ed9e2e485b1ca1ece9a9a043bc6c8"
+        "size": 46025,
+        "sha256": "c8c8ed310cbdebf7e27adf3361e6ce5939922bd674af12205cea441a9b53399d"
       },
       {
         "path": "ground_truth/0008.txt",
-        "size": 185,
-        "sha256": "2dbeb40c0172dc618c7298a87e131153f51682dcaa33efefa67b7e50ecd7a8d1"
+        "size": 191,
+        "sha256": "246e79c3ef9faa52dd10c400eecf594314493c88ae5f7794a6129dd90e850107"
       },
       {
         "path": "images/0009.png",
-        "size": 40784,
-        "sha256": "f93ea216d77bfdb23f6c8c113e33c946318d6eac6ef3bdaf2e2f2c0d86c154c9"
+        "size": 41617,
+        "sha256": "a00eaebf40f9369b6851a949a0aedc82a57cf1104a957e42880b1f3aa65606f0"
       },
       {
         "path": "ground_truth/0009.txt",
-        "size": 188,
-        "sha256": "957232168770c2a15efeb33126a7d2d8141ce34782106b979e4b31acfa4f9bee"
+        "size": 195,
+        "sha256": "5653ef6df8a330496e73c6ddd71c9b2280818e13b11b6ff8842622ad53d24a5b"
       },
       {
         "path": "images/0010.png",
-        "size": 20512,
-        "sha256": "5ff52fd340da20784c4c934cbac3ae9a0788b7d1654d71934100624842ddf764"
+        "size": 21089,
+        "sha256": "59b6edec6b2ef0369f1d7fbe9fe96648fda2982db0eae377199d5edf26855900"
       },
       {
         "path": "ground_truth/0010.txt",
-        "size": 194,
-        "sha256": "2f9054687e5eb369e35d985bc2fe9d2d37ffeb8cafe204c3e81830f4431f5793"
+        "size": 201,
+        "sha256": "6d2f6474c94dea173288aebde046b6db40534548e8187cb776dc494abaa04ce2"
       },
       {
         "path": "images/0011.png",
-        "size": 486656,
-        "sha256": "74e878d54295c224481f0f1b51a670459b855b16492c246c27636d71f4666e68"
+        "size": 487332,
+        "sha256": "70f57ebf6f41479e443e4356422e6dd619ffdcf2f0df7837943f5427545cd266"
       },
       {
         "path": "ground_truth/0011.txt",
-        "size": 119,
-        "sha256": "8f8bc16447a12c17e4036c4c5fd936faef227310a564de983ebe2f4f22742d83"
+        "size": 121,
+        "sha256": "f550334b0b7d3508ef898d0e144593c6d005dd32d1bbcb837aeb650da56e9eb8"
       },
       {
         "path": "images/0012.png",
-        "size": 515472,
-        "sha256": "9a6e49af19e893000b8e96bece22fe99fedd3741e7ac3dede04a112ea77b5b5c"
+        "size": 516088,
+        "sha256": "fdd426f69928ba7bcc6008655fa0dbd950b6844eba8322491e7a5399bf052d59"
       },
       {
         "path": "ground_truth/0012.txt",
-        "size": 119,
-        "sha256": "1ccf00fe53da00ba3a2340dbe95e1692e87efccdc04b191c0f1d52dab23169a7"
+        "size": 121,
+        "sha256": "d040adc39553bc337df562486b3b5148727660214ec79a3af48b9cf5a2bc23f7"
       }
     ]
   },

release/reference_accuracy.json

@@ -1,5 +1,7 @@
 {
   "model_path": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+  "served_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
+  "backend": "mlx",
   "prompt": "<image>document parsing.",
   "max_tokens": 1024,
   "profile": "accurate",
@@ -12,183 +14,237 @@
   },
   "num_images": 12,
   "num_samples": 12,
-  "total_time_seconds": 26.09562641500088,
-  "mean_cer": 1.057248773567081,
-  "mean_wer": 0.9346679608165291,
+  "total_time_seconds": 162.6677352060069,
+  "mean_cer": 0.9073812810686629,
+  "mean_wer": 1.001372473311303,
   "num_digit_samples": 12,
   "num_cjk_samples": 3,
-  "mean_digit_cer": 0.20915311607827292,
+  "mean_digit_cer": 0.4065110611974061,
   "mean_cjk_cer": 1.0,
   "num_table_samples": 3,
   "mean_table_score": 1.0,
   "mean_repetition_rate": 0.00026371308016877635,
+  "by_category": {
+    "cjk_document": {
+      "num_samples": 3,
+      "mean_cer": 2.317035024900193,
+      "mean_digit_cer": 0.08888888888888889,
+      "mean_cjk_cer": 1.0,
+      "mean_table_score": null,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 3,
+      "num_table_samples": 0
+    },
+    "english_document": {
+      "num_samples": 4,
+      "mean_cer": 0.8171043138735056,
+      "mean_digit_cer": 0.9432759996841724,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 4,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "low_quality_scan": {
+      "num_samples": 2,
+      "mean_cer": 0.07142857142857142,
+      "mean_digit_cer": 0.13793103448275862,
+      "mean_cjk_cer": null,
+      "mean_table_score": null,
+      "num_digit_samples": 2,
+      "num_cjk_samples": 0,
+      "num_table_samples": 0
+    },
+    "table_numeric": {
+      "num_samples": 3,
+      "mean_cer": 0.1753986332574032,
+      "mean_digit_cer": 0.1875,
+      "mean_cjk_cer": null,
+      "mean_table_score": 1.0,
+      "num_digit_samples": 3,
+      "num_cjk_samples": 0,
+      "num_table_samples": 3
+    }
+  },
   "per_file": [
     {
       "file": "0001.png",
-      "cer": 0.06144256455921639,
-      "wer": 0.20179372197309417,
-      "digit_cer": 0.3023255813953488,
+      "category": "english_document",
+      "cer": 1.1751563896336015,
+      "wer": 1.439252336448598,
+      "digit_cer": 0.48148148148148145,
       "cjk_cer": 0.0,
-      "repetition_rate": 0.0,
+      "repetition_rate": 0.0031645569620253164,
       "table_score": null,
-      "elapsed_seconds": 3.459972665994428,
-      "ref_length": 1345,
-      "hyp_length": 1382,
-      "ref_digit_count": 86,
+      "elapsed_seconds": 32.467281207995256,
+      "ref_length": 1332,
+      "hyp_length": 2457,
+      "ref_digit_count": 27,
       "ref_cjk_count": 0
     },
     {
       "file": "0002.png",
-      "cer": 0.47234678624813153,
-      "wer": 0.5988023952095808,
-      "digit_cer": 0.5578947368421052,
+      "category": "english_document",
+      "cer": 0.7319494584837545,
+      "wer": 0.6216216216216216,
+      "digit_cer": 0.288135593220339,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.406020459005958,
-      "ref_length": 835,
-      "hyp_length": 771,
-      "ref_digit_count": 95,
+      "elapsed_seconds": 31.99284904099477,
+      "ref_length": 1329,
+      "hyp_length": 1794,
+      "ref_digit_count": 59,
       "ref_cjk_count": 0
     },
     {
       "file": "0003.png",
-      "cer": 1.1751563896336015,
-      "wer": 1.439252336448598,
-      "digit_cer": 0.48148148148148145,
+      "category": "english_document",
+      "cer": 1.2802303262955854,
+      "wer": 1.2586206896551724,
+      "digit_cer": 2.767123287671233,
       "cjk_cer": 0.0,
-      "repetition_rate": 0.0031645569620253164,
+      "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.409982333003427,
-      "ref_length": 1332,
-      "hyp_length": 2457,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 30.016402541004936,
+      "ref_length": 636,
+      "hyp_length": 1349,
+      "ref_digit_count": 73,
       "ref_cjk_count": 0
     },
     {
       "file": "0004.png",
-      "cer": 0.7319494584837545,
-      "wer": 0.6216216216216216,
-      "digit_cer": 0.288135593220339,
+      "category": "english_document",
+      "cer": 0.08108108108108109,
+      "wer": 0.15492957746478872,
+      "digit_cer": 0.23636363636363636,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 4.450652749990695,
-      "ref_length": 1329,
-      "hyp_length": 1794,
-      "ref_digit_count": 59,
+      "elapsed_seconds": 12.61737366700254,
+      "ref_length": 477,
+      "hyp_length": 478,
+      "ref_digit_count": 55,
       "ref_cjk_count": 0
     },
     {
       "file": "0005.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 1.2356766660086578,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 8.039810500005842,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0006.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 1.2516661249974277,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 8.472794083005283,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0007.png",
-      "cer": 0.17715617715617715,
+      "category": "table_numeric",
+      "cer": 0.17539863325740318,
       "wer": 1.5,
-      "digit_cer": 0.2,
+      "digit_cer": 0.1875,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": 1.0,
-      "elapsed_seconds": 1.274972874991363,
-      "ref_length": 436,
-      "hyp_length": 508,
-      "ref_digit_count": 70,
+      "elapsed_seconds": 7.832311457998003,
+      "ref_length": 446,
+      "hyp_length": 519,
+      "ref_digit_count": 80,
       "ref_cjk_count": 0
     },
     {
       "file": "0008.png",
-      "cer": 1.6867469879518073,
+      "category": "cjk_document",
+      "cer": 1.5730337078651686,
       "wer": 0.9,
       "digit_cer": 0.0,
       "cjk_cer": 1.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.1017809159966419,
-      "ref_length": 92,
-      "hyp_length": 187,
-      "ref_digit_count": 26,
+      "elapsed_seconds": 6.520762083993759,
+      "ref_length": 98,
+      "hyp_length": 193,
+      "ref_digit_count": 30,
       "ref_cjk_count": 41
     },
     {
       "file": "0009.png",
-      "cer": 5.670731707317073,
+      "category": "cjk_document",
+      "cer": 3.202247191011236,
       "wer": 1.0,
       "digit_cer": 0.0,
       "cjk_cer": 1.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.35718295800325,
-      "ref_length": 93,
-      "hyp_length": 509,
-      "ref_digit_count": 25,
+      "elapsed_seconds": 6.874465999993845,
+      "ref_length": 100,
+      "hyp_length": 334,
+      "ref_digit_count": 30,
       "ref_cjk_count": 42
     },
     {
       "file": "0010.png",
-      "cer": 2.357142857142857,
+      "category": "cjk_document",
+      "cer": 2.1758241758241756,
       "wer": 1.9545454545454546,
-      "digit_cer": 0.28,
+      "digit_cer": 0.26666666666666666,
       "cjk_cer": 1.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.1220285840099677,
-      "ref_length": 105,
-      "hyp_length": 269,
-      "ref_digit_count": 25,
+      "elapsed_seconds": 6.385247083002469,
+      "ref_length": 112,
+      "hyp_length": 275,
+      "ref_digit_count": 30,
       "ref_cjk_count": 44
     },
     {
       "file": "0011.png",
-      "cer": 0.0,
-      "wer": 0.0,
-      "digit_cer": 0.0,
+      "category": "low_quality_scan",
+      "cer": 0.12380952380952381,
+      "wer": 0.125,
+      "digit_cer": 0.27586206896551724,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.0181266669969773,
-      "ref_length": 118,
-      "hyp_length": 118,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 5.718108458007919,
+      "ref_length": 120,
+      "hyp_length": 135,
+      "ref_digit_count": 29,
       "ref_cjk_count": 0
     },
     {
       "file": "0012.png",
-      "cer": 0.0,
-      "wer": 0.0,
+      "category": "low_quality_scan",
+      "cer": 0.01904761904761905,
+      "wer": 0.0625,
       "digit_cer": 0.0,
       "cjk_cer": 0.0,
       "repetition_rate": 0.0,
       "table_score": null,
-      "elapsed_seconds": 1.0075634160020854,
-      "ref_length": 118,
-      "hyp_length": 118,
-      "ref_digit_count": 27,
+      "elapsed_seconds": 5.730329083002289,
+      "ref_length": 120,
+      "hyp_length": 123,
+      "ref_digit_count": 29,
       "ref_cjk_count": 0
     }
   ]

release/reference_performance.json

@@ -1,5 +1,6 @@
 {
   "model_path": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+  "served_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
   "image_path": "test_invoice.png",
   "prompt": "<image>document parsing.",
   "max_tokens": 256,
@@ -13,55 +14,55 @@
     "chip": "Apple M3 Max",
     "total_memory_gb": 128.0
   },
-  "model_load_time_seconds": 5.561058167004376,
-  "memory_after_load_mb": 3994.34375,
+  "model_load_time_seconds": 2.9305669579916866,
+  "memory_after_load_mb": 3992.859375,
   "num_warmup": 1,
   "num_runs": 3,
-  "mean_tps": 312.7284719764298,
-  "std_tps": 0.9770038029028864,
-  "mean_elapsed_seconds": 1.2253679166703175,
+  "mean_tps": 35.07146198590831,
+  "std_tps": 0.290807343078657,
+  "mean_elapsed_seconds": 8.953089208662277,
   "mean_peak_memory_mb": 5165.991419999999,
   "runs": [
     {
       "run": 1,
-      "elapsed_seconds": 1.226103625012911,
+      "elapsed_seconds": 9.042429541994352,
       "tokens_generated": 256,
       "tokens_generated_source": "mlx-vlm token count",
-      "tokens_per_second": 311.86183520486986,
+      "tokens_per_second": 34.74497597308309,
       "tokens_per_second_source": "mlx-vlm generation_tps",
       "peak_memory_mb": 5165.81578,
       "memory_delta_mb": 1337.186768,
-      "process_peak_rss_mb": 4015.96875,
+      "process_peak_rss_mb": 4022.6875,
       "prompt_tokens": 697,
-      "prompt_tokens_per_second": 1816.7391562081139,
+      "prompt_tokens_per_second": 429.97806661114964,
       "finish_reason": "length"
     },
     {
       "run": 2,
-      "elapsed_seconds": 1.2276696659973823,
+      "elapsed_seconds": 9.023512416999438,
       "tokens_generated": 256,
       "tokens_generated_source": "mlx-vlm token count",
-      "tokens_per_second": 312.2298514286067,
+      "tokens_per_second": 35.01812122387363,
       "tokens_per_second_source": "mlx-vlm generation_tps",
       "peak_memory_mb": 5166.07924,
       "memory_delta_mb": 1337.188084,
-      "process_peak_rss_mb": 4022.953125,
+      "process_peak_rss_mb": 4023.046875,
       "prompt_tokens": 697,
-      "prompt_tokens_per_second": 1801.9665867511062,
+      "prompt_tokens_per_second": 417.40764558358615,
       "finish_reason": "length"
     },
     {
       "run": 3,
-      "elapsed_seconds": 1.2223304590006592,
+      "elapsed_seconds": 8.793325666993042,
       "tokens_generated": 256,
       "tokens_generated_source": "mlx-vlm token count",
-      "tokens_per_second": 314.0937292958128,
+      "tokens_per_second": 35.451288760768186,
       "tokens_per_second_source": "mlx-vlm generation_tps",
       "peak_memory_mb": 5166.07924,
       "memory_delta_mb": 1337.188084,
-      "process_peak_rss_mb": 4023.265625,
+      "process_peak_rss_mb": 4023.3125,
       "prompt_tokens": 697,
-      "prompt_tokens_per_second": 1800.5791804452258,
+      "prompt_tokens_per_second": 449.69288066410945,
       "finish_reason": "length"
     }
   ]

release/release_manifest.json

@@ -1,10 +1,11 @@
 {
-  "schema_version": 1,
-  "created_at": "2026-07-20T23:31:41.909496+00:00",
+  "schema_version": 3,
+  "created_at": "2026-07-21T05:26:20.621136+00:00",
   "release_approved": true,
   "repo_id": "AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
   "source": {
     "id": "baidu/Unlimited-OCR",
+    "resolved_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
     "weights": {
       "files": [
         {
@@ -20,6 +21,7 @@
   },
   "reference": {
     "id": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+    "resolved_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
     "weights": {
       "files": [
         {
@@ -39,13 +41,63 @@
       "files": [
         {
           "name": "model.safetensors",
-          "size": 3843160559,
-          "sha256": "27568517565832e42daa563f6939e66a8ebb50cce0b01a24ecebb3ea6b4d0b10"
+          "size": 3845699993,
+          "sha256": "d9fe11fc8d2e333000be6a0e288681231efa69282c844f7775ca40f52db02f0c"
         }
       ],
-      "total_size_bytes": 3843160559,
-      "total_size_gb": 3.5792221864685416,
-      "aggregate_sha256": "a4f6e9934f90564906f42929542ad68493f0c6187ad431128e447debbf52b6f4"
+      "total_size_bytes": 3845699993,
+      "total_size_gb": 3.5815872186794877,
+      "aggregate_sha256": "5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549"
+    },
+    "files": {
+      "files": [
+        {
+          "path": "config.json",
+          "size": 33815,
+          "sha256": "2c3eeb61157f6b9a5c42aa0771771911516e3ac8b3cbd9b61d1532a33e84a2bd"
+        },
+        {
+          "path": "model.safetensors.index.json",
+          "size": 61701,
+          "sha256": "bb6693a47e22e596139e90d8a1442a64155900d27aa0e2419ec071ef4f0ad4a3"
+        },
+        {
+          "path": "processor_config.json",
+          "size": 454,
+          "sha256": "e9f224a7f0d2263bf62efc052e93681720e31f9e34b1c42134d2371e8708a426"
+        },
+        {
+          "path": "special_tokens_map.json",
+          "size": 801,
+          "sha256": "ab4bd57ce17d62e39e0a39e739de1e407484f090f0b2c7e391312bca7a5b061a"
+        },
+        {
+          "path": "tokenizer.json",
+          "size": 9979544,
+          "sha256": "a02f8fd5228c90256bb4f6554c34a579d48f909e5beb232dc4afad870b55a8b4"
+        },
+        {
+          "path": "tokenizer_config.json",
+          "size": 569,
+          "sha256": "86f9fcd3f04060ba47a616241f5fc3f65b53a54ee7260367089467cdca849568"
+        },
+        {
+          "path": "chat_template.jinja",
+          "size": 191,
+          "sha256": "b4e4771e69892ebc712ed6986bcce490c65041d1545e85599cd4a531cfab98a5"
+        },
+        {
+          "path": "precision_map.json",
+          "size": 3337,
+          "sha256": "91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75"
+        },
+        {
+          "path": "quantization_summary.json",
+          "size": 15572,
+          "sha256": "417dbf5b59f6064361f6096cbf59bbe1774e7562301a49f47100fe4ceb201c55"
+        }
+      ],
+      "aggregate_sha256": "cbd09caa91f51dd7abf2ebb16eb38d20a71ff00e21056a985f8bfd28269ac97c"
     },
     "metadata": {
       "passed": true,
@@ -55,25 +107,210 @@
         "mxfp8": true,
         "sliding_window": true,
         "processor_class": true,
-        "sft_format": true
+        "sft_format": true,
+        "precision_map": true,
+        "quantization_summary": true,
+        "protected_modules_preserved": true
       },
-      "sliding_window": 128
+      "sliding_window": 128,
+      "source_model": "Unlimited-OCR",
+      "source_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead"
     }
   },
-  "dataset": {
-    "name": "eval_dataset",
-    "num_samples": 12,
-    "num_digit_samples": 12,
+  "project_files": {
+    "files": [
+      {
+        "path": "LICENSE",
+        "size": 1067,
+        "sha256": "6268bc8a51423daeb65e5b0225d9a73fd87b464e235bebc02ceaccc1d1779f45"
+      },
+      {
+        "path": "pyproject.toml",
+        "size": 2060,
+        "sha256": "7441c069b03a90e26b1997e94d52edb2d8a7ce0d5c012093080d143e79d19ba2"
+      },
+      {
+        "path": "generation_config.json",
+        "size": 802,
+        "sha256": "69ac71fd1613d81cb83d07fb328ad6bf24eab5584fd5348665dad2497782954d"
+      },
+      {
+        "path": "preprocessing_config.json",
+        "size": 3033,
+        "sha256": "a269fce81f50bdabec9f7fac0219ccd9de62e2b7fd9d8a31f313b04e52b70432"
+      },
+      {
+        "path": "quantization_config.json",
+        "size": 1441,
+        "sha256": "cc5d461002cece610848d697aa293a41b57edcebd93020348451dbe4de1d2481"
+      },
+      {
+        "path": "docs/PRD.md",
+        "size": 6915,
+        "sha256": "c64527daf7d9b591a453be856d8a8733f8dd23f8a8e4c913cf47b49ff17786c1"
+      },
+      {
+        "path": "docs/TECHNICAL_SPEC.md",
+        "size": 10379,
+        "sha256": "3967f46ee59728e75563907ec8f0e93d08a5da320d28395c4e3084a1ff68a3db"
+      },
+      {
+        "path": "docs/adr/0001-verified-ocr-aware-quantization.md",
+        "size": 3312,
+        "sha256": "e5c4996d80263bcaf43a60cd91db45c214873486eee858fe0ac3f4762fde5e21"
+      },
+      {
+        "path": "docs/adr/0002-calibrated-affine8-lm-head.md",
+        "size": 2722,
+        "sha256": "4606afa17413791e4bd4acc2e959162fc98e6978a67b236d201add396e31d5ea"
+      },
+      {
+        "path": "src/unlimited_ocr/__init__.py",
+        "size": 533,
+        "sha256": "5f46da23ba70ec43fadd5e3b4b5f9335063e4fc86e2ef20d23e33f8722fb8d19"
+      },
+      {
+        "path": "src/unlimited_ocr/engine.py",
+        "size": 24024,
+        "sha256": "b66e97254d317bf4b8886daa3d01aecae71237590d6bb249af08fd6c9831da1c"
+      },
+      {
+        "path": "src/unlimited_ocr/pipeline.py",
+        "size": 26762,
+        "sha256": "baf259a4e46282b031cbdc71d1538dc68c2293935ff3581233d3d0b41e32d666"
+      },
+      {
+        "path": "src/unlimited_ocr/preprocessing.py",
+        "size": 28532,
+        "sha256": "d553b667a1c83ad1aedf63fdf642227a829dab66c705562e1c9d718f3ffe898d"
+      },
+      {
+        "path": "src/unlimited_ocr/output.py",
+        "size": 15061,
+        "sha256": "c0f42a4c454ad39ef62714575d8f0c391ba5572edb9a79b9d06c7049382964df"
+      },
+      {
+        "path": "src/unlimited_ocr/pdf.py",
+        "size": 8659,
+        "sha256": "3c23ea843bd06aa0712aabdf7c164270c8d363e32271a4e13a446ffdec05a460"
+      },
+      {
+        "path": "src/unlimited_ocr/cli.py",
+        "size": 20469,
+        "sha256": "c6b813ef2a01a960369bbff79d270cb335f41b92ad3435b24cf2665cc2d2c6f5"
+      },
+      {
+        "path": "src/unlimited_ocr/profiles.py",
+        "size": 2917,
+        "sha256": "ce31200aeb9c8e459b258e3258c0ded76ca40de0d78d07b8cb6bdd05ed45653f"
+      },
+      {
+        "path": "src/unlimited_ocr/server.py",
+        "size": 12153,
+        "sha256": "7723c3931e610815efc64477f3e7bd41ee851d9aee53798f42842e13ffdce2d1"
+      },
+      {
+        "path": "src/unlimited_ocr/adapter_registry.json",
+        "size": 880,
+        "sha256": "b06c2778707788f75a5f5c12097246c06126a6adca4bfbd8e4ba40fc6690ad6b"
+      },
+      {
+        "path": "quantization/layer_sensitivity.py",
+        "size": 14664,
+        "sha256": "06b954efcfa58eb00a02e3db77b46857a13c5fcab5b1422350cda062c577f7f3"
+      },
+      {
+        "path": "quantization/calibrate_precision.py",
+        "size": 10666,
+        "sha256": "0edaa2d14f435288fff96e876ea5eab8111232b34a802708bf354ee7a259c3c2"
+      },
+      {
+        "path": "quantization/mixed_precision_convert.py",
+        "size": 13954,
+        "sha256": "33b01c48b0f945871633c91e46e063549062730ae7792d2540a8a0a59dd193fe"
+      },
+      {
+        "path": "quantization/precision_map.json",
+        "size": 1743,
+        "sha256": "d7965210b5cab6afbde056cc91a33a41b651a254a9fa8db0b14dcdf85f7cbcbf"
+      },
+      {
+        "path": "quantization/release_gate.py",
+        "size": 83309,
+        "sha256": "87e9e2c4f3e94345009f698ec6222a4b78172693ac4c9444571cf34e100d5498"
+      },
+      {
+        "path": "quantization/run_pipeline.py",
+        "size": 23650,
+        "sha256": "35229b866f7433397d479e2795248d21b9d02b9d7e7000d01440ffd23c9162ec"
+      },
+      {
+        "path": "benchmarks/evaluate_cer.py",
+        "size": 9407,
+        "sha256": "e6c5010a5886e3e5342825f2b970d792bf02e4ef79e44284de74702f2745e423"
+      },
+      {
+        "path": "benchmarks/evaluate_tables.py",
+        "size": 9554,
+        "sha256": "a5ff42da5083eff0b70d7dc6e351395338198c1b860a3d6565e2352b159850a2"
+      },
+      {
+        "path": "benchmarks/normalize_output.py",
+        "size": 4059,
+        "sha256": "dfceeb07c0df48f6f4ed925f990a45b5f8f69fc0dfc37f0fd377cea6073196e4"
+      },
+      {
+        "path": "benchmarks/run_accuracy.py",
+        "size": 16808,
+        "sha256": "9e937e32b5c3599c07733f1046184624b2a0c40b68e307058c3fd141c383994e"
+      },
+      {
+        "path": "benchmarks/run_performance.py",
+        "size": 12345,
+        "sha256": "8f87f459b1f4778afdfc20c078154d954edf521699b273f610132d0371d1c57a"
+      },
+      {
+        "path": "benchmarks/rswa_validation.py",
+        "size": 21969,
+        "sha256": "0de31e66650604cfbdd5a48b85470002bbe447bca7caa7a0aa7ca48ac8512215"
+      },
+      {
+        "path": "benchmarks/datasets.md",
+        "size": 3185,
+        "sha256": "47902cdf95278dc9509057fe64018251dce43e775fa7582590d39a916483d983"
+      },
+      {
+        "path": "examples/single_image.py",
+        "size": 1123,
+        "sha256": "87afe046d745fe62e78eb584b35e8dffcd9579c13cf2f01b483b9d4ea9427cc5"
+      },
+      {
+        "path": "examples/multi_page_pdf.py",
+        "size": 1182,
+        "sha256": "cc335ed4d4e55f1b0818813d6fbd00319a1fb4675fda8b6c5803b5fdf137ae37"
+      },
+      {
+        "path": "examples/batch_directory.py",
+        "size": 1411,
+        "sha256": "d9bc32620e48fc5bcbb0504b3a16a90de9a35d1083d56584b2fce1e2591a928d"
+      }
+    ],
+    "aggregate_sha256": "68de2e9d2c503b3b2023d0b55ff0f7446d2046ac5a3285b5e9e23b8baeb6a370"
+  },
+  "calibration_dataset": {
+    "name": "calibration",
+    "num_samples": 10,
+    "num_digit_samples": 10,
     "num_cjk_samples": 3,
     "num_table_samples": 3,
     "category_counts": {
       "cjk_document": 3,
-      "english_document": 4,
+      "english_document": 2,
       "low_quality_scan": 2,
       "table_numeric": 3
     },
-    "manifest_sha256": "94d96ec344b02b8c4b413eafe524d6844bddcb00705af4b3fccea4c297c22509",
-    "content_sha256": "78ca3e7c9c7584aea30b04176716a6dadf49ccf0f2bead959cbaeb9aeac68f30",
+    "manifest_sha256": "d4429669fe60a5c48ce82073294d20c845b85084644f9d4f62f7c8ca3923ba7a",
+    "content_sha256": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
     "files": [
       {
         "path": "images/0001.png",
@@ -97,120 +334,239 @@
       },
       {
         "path": "images/0003.png",
+        "size": 40039,
+        "sha256": "bbe26dc81fb4c5e6a143279dd1e0d0540df746f5e4e8f9fc7577e936882b190e"
+      },
+      {
+        "path": "ground_truth/0003.txt",
+        "size": 459,
+        "sha256": "39cf88aef99b6ce6b5d8af870e2feefca7a9991e5c835660bda60c5b5f80f481"
+      },
+      {
+        "path": "images/0004.png",
+        "size": 40356,
+        "sha256": "1e50bec7880f37e9e25eb8c4706998634b0c1f254d38fb75ccd6dbb9b89ed3a3"
+      },
+      {
+        "path": "ground_truth/0004.txt",
+        "size": 459,
+        "sha256": "2a2b723c4f1b4e14cdf5d70a74ae12586c12952ffac0d1bd295c448f1bf2d3fc"
+      },
+      {
+        "path": "images/0005.png",
+        "size": 40650,
+        "sha256": "7b7c9441ff17bb97ee76cb68eb6b9134f7d26d151c6ca1d18039959bf43de04c"
+      },
+      {
+        "path": "ground_truth/0005.txt",
+        "size": 459,
+        "sha256": "3dbec98a34736b424dafcf492691b21a1a36f629d83589cac5482cc1a135d9b7"
+      },
+      {
+        "path": "images/0006.png",
+        "size": 46036,
+        "sha256": "76006318e5aa3bb54182d7b3249ad016e7ab0dee387f580331c39b9ac49cb7ab"
+      },
+      {
+        "path": "ground_truth/0006.txt",
+        "size": 191,
+        "sha256": "75082837c3cbb8c41ba0747b98390a1f44577092c30f31f4c1e10ce52af83b61"
+      },
+      {
+        "path": "images/0007.png",
+        "size": 41828,
+        "sha256": "d642986c12233efe0bc7559096952e66eb175735f89ac91caedea0089c4adb19"
+      },
+      {
+        "path": "ground_truth/0007.txt",
+        "size": 195,
+        "sha256": "c47bf79076b7ab732c22dc1b023d9f6f12b98b724dd0d730c23e879ef7604e29"
+      },
+      {
+        "path": "images/0008.png",
+        "size": 20975,
+        "sha256": "852416613d0137144df77408c44e7d469249e7fbc650c684ad28911b96b28e76"
+      },
+      {
+        "path": "ground_truth/0008.txt",
+        "size": 201,
+        "sha256": "dc48f9c80e21438d5c2c3fb57b3bc471961708e2dd971df78a9ecc11c8c3b3ad"
+      },
+      {
+        "path": "images/0009.png",
+        "size": 487313,
+        "sha256": "87f977973bddf89f781762c87feb9c845e02731dae5cf39086db4cdb0588298e"
+      },
+      {
+        "path": "ground_truth/0009.txt",
+        "size": 121,
+        "sha256": "d9f04e21b8e73f964a9dee2776c65a76d509138f8abe694c46305a32833a52f1"
+      },
+      {
+        "path": "images/0010.png",
+        "size": 515820,
+        "sha256": "6fb40a4c63aed4cbb11346d0c80b917cf1ecd407b364bcfbc87483b5bd90d0b4"
+      },
+      {
+        "path": "ground_truth/0010.txt",
+        "size": 121,
+        "sha256": "b8e07f309069b395279995af437f82f3a9453f2add587bd9633b4ae97cc8419a"
+      }
+    ]
+  },
+  "evaluation_dataset": {
+    "name": "evaluation",
+    "num_samples": 12,
+    "num_digit_samples": 12,
+    "num_cjk_samples": 3,
+    "num_table_samples": 3,
+    "category_counts": {
+      "cjk_document": 3,
+      "english_document": 4,
+      "low_quality_scan": 2,
+      "table_numeric": 3
+    },
+    "manifest_sha256": "111646a6f59ed32689fbc658c1cb4ed60629e1bbd909d6137df469e7b489f121",
+    "content_sha256": "7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987",
+    "files": [
+      {
+        "path": "images/0001.png",
         "size": 181040,
         "sha256": "40330138b25a7ce9bf142a672ad0a6253517bf076af66591c4431490eff1ecfc"
       },
       {
-        "path": "ground_truth/0003.txt",
+        "path": "ground_truth/0001.txt",
         "size": 1333,
         "sha256": "517ab5af1c090227d4dc2473ac422ecd9301e2e21ba0cc1c54d08476b1903d7e"
       },
       {
-        "path": "images/0004.png",
+        "path": "images/0002.png",
         "size": 220208,
         "sha256": "4316ba5ebe86c62331361e6d23329adede5e9eecf50a70db16b5439d95f5e23e"
       },
       {
-        "path": "ground_truth/0004.txt",
+        "path": "ground_truth/0002.txt",
         "size": 1330,
         "sha256": "df3b4afaee0f3ffe0984c2f1d1201c1762bc70f4ffbc1bfe9b1242e17d794148"
       },
+      {
+        "path": "images/0003.png",
+        "size": 119613,
+        "sha256": "bcf22c94459b970f1ee11088d8a902dbc56975e487c489b4714dedf8f3a4dca1"
+      },
+      {
+        "path": "ground_truth/0003.txt",
+        "size": 637,
+        "sha256": "a035ed288c8cc0e73e6cddc08c804be9b1ed32eee98e7f85c0e027bd88221b66"
+      },
+      {
+        "path": "images/0004.png",
+        "size": 67701,
+        "sha256": "baea8365beb81cc9939e59d3c8a0b88ab17999122eb1729e33214450a4fba987"
+      },
+      {
+        "path": "ground_truth/0004.txt",
+        "size": 478,
+        "sha256": "af1c29967e55595362ab41e25000e00167474cbe38ea43b8816f1f1395ef6e72"
+      },
       {
         "path": "images/0005.png",
-        "size": 38984,
-        "sha256": "1532b86fe28d7331379c73b62cfb0b0ae1448824566684021df262e4741b06a4"
+        "size": 41300,
+        "sha256": "65662634fc3b5df7d006f40d276fb2869809a86a2027f464c1d6776094e16f16"
       },
       {
         "path": "ground_truth/0005.txt",
-        "size": 449,
-        "sha256": "fb8ebea089dacff057d98feb6450436f3f6db65f0bac29513d3417e94451686e"
+        "size": 459,
+        "sha256": "1d565459641aca5056fdcccabe2cdc6834107bef219645a54cd28db2e06b05ea"
       },
       {
         "path": "images/0006.png",
-        "size": 38512,
-        "sha256": "8c1bf51f86f0f22c8898cec8c77049b6f023b77aab77f55be40f75b477ab6383"
+        "size": 40574,
+        "sha256": "67dbd4d07cb6e3f971937dc93a35ac81775580ee4d8895373afe6ae7e6b328dd"
       },
       {
         "path": "ground_truth/0006.txt",
-        "size": 449,
-        "sha256": "f60ff098ac431b1021fb85f810dd6264c1c021fab0810ab97ce304c5eb802598"
+        "size": 459,
+        "sha256": "0e134815b0bee585e3841d5a22a296c136fb43f00a9e3d48d98d2df24e884b08"
       },
       {
         "path": "images/0007.png",
-        "size": 38911,
-        "sha256": "a87e7f53196e9d3f72af611394297ec64e9eb222d2872b62f0c5a2014d83ecfc"
+        "size": 41027,
+        "sha256": "2b54b5222174bf4d41d92a6ee97aa1b2a3e57daca987b55906793df807dfe08f"
       },
       {
         "path": "ground_truth/0007.txt",
-        "size": 449,
-        "sha256": "0381ec16f8895290dfc2b81f97933f5568a68bfe731cc4791272ff8269e57c0e"
+        "size": 459,
+        "sha256": "b63a7c471f4d36cc8bafddcbd7b4a8d92d6f92e73ab1e260505c2413d6e39146"
       },
       {
         "path": "images/0008.png",
-        "size": 45665,
-        "sha256": "d9ff0cc9ffa639e485b0d445e1f4856f479ed9e2e485b1ca1ece9a9a043bc6c8"
+        "size": 46025,
+        "sha256": "c8c8ed310cbdebf7e27adf3361e6ce5939922bd674af12205cea441a9b53399d"
       },
       {
         "path": "ground_truth/0008.txt",
-        "size": 185,
-        "sha256": "2dbeb40c0172dc618c7298a87e131153f51682dcaa33efefa67b7e50ecd7a8d1"
+        "size": 191,
+        "sha256": "246e79c3ef9faa52dd10c400eecf594314493c88ae5f7794a6129dd90e850107"
       },
       {
         "path": "images/0009.png",
-        "size": 40784,
-        "sha256": "f93ea216d77bfdb23f6c8c113e33c946318d6eac6ef3bdaf2e2f2c0d86c154c9"
+        "size": 41617,
+        "sha256": "a00eaebf40f9369b6851a949a0aedc82a57cf1104a957e42880b1f3aa65606f0"
       },
       {
         "path": "ground_truth/0009.txt",
-        "size": 188,
-        "sha256": "957232168770c2a15efeb33126a7d2d8141ce34782106b979e4b31acfa4f9bee"
+        "size": 195,
+        "sha256": "5653ef6df8a330496e73c6ddd71c9b2280818e13b11b6ff8842622ad53d24a5b"
       },
       {
         "path": "images/0010.png",
-        "size": 20512,
-        "sha256": "5ff52fd340da20784c4c934cbac3ae9a0788b7d1654d71934100624842ddf764"
+        "size": 21089,
+        "sha256": "59b6edec6b2ef0369f1d7fbe9fe96648fda2982db0eae377199d5edf26855900"
       },
       {
         "path": "ground_truth/0010.txt",
-        "size": 194,
-        "sha256": "2f9054687e5eb369e35d985bc2fe9d2d37ffeb8cafe204c3e81830f4431f5793"
+        "size": 201,
+        "sha256": "6d2f6474c94dea173288aebde046b6db40534548e8187cb776dc494abaa04ce2"
       },
       {
         "path": "images/0011.png",
-        "size": 486656,
-        "sha256": "74e878d54295c224481f0f1b51a670459b855b16492c246c27636d71f4666e68"
+        "size": 487332,
+        "sha256": "70f57ebf6f41479e443e4356422e6dd619ffdcf2f0df7837943f5427545cd266"
       },
       {
         "path": "ground_truth/0011.txt",
-        "size": 119,
-        "sha256": "8f8bc16447a12c17e4036c4c5fd936faef227310a564de983ebe2f4f22742d83"
+        "size": 121,
+        "sha256": "f550334b0b7d3508ef898d0e144593c6d005dd32d1bbcb837aeb650da56e9eb8"
       },
       {
         "path": "images/0012.png",
-        "size": 515472,
-        "sha256": "9a6e49af19e893000b8e96bece22fe99fedd3741e7ac3dede04a112ea77b5b5c"
+        "size": 516088,
+        "sha256": "fdd426f69928ba7bcc6008655fa0dbd950b6844eba8322491e7a5399bf052d59"
       },
       {
         "path": "ground_truth/0012.txt",
-        "size": 119,
-        "sha256": "1ccf00fe53da00ba3a2340dbe95e1692e87efccdc04b191c0f1d52dab23169a7"
+        "size": 121,
+        "sha256": "d040adc39553bc337df562486b3b5148727660214ec79a3af48b9cf5a2bc23f7"
       }
     ]
   },
   "thresholds": {
-    "max_cer_delta_vs_bf16": 0.01,
+    "max_cer_delta_vs_bf16": 0.015,
     "max_cer_delta_vs_reference": 0.005,
     "max_digit_cer_delta_vs_bf16": 0.01,
     "max_table_score_degradation_vs_bf16": 0.01,
     "min_tps_ratio_vs_reference": 0.9,
-    "max_weight_size_gb": 4.5
+    "max_weight_size_gb": 4.5,
+    "min_rswa_tokens": 8192,
+    "max_rswa_repetition_rate": 0.25
   },
   "metrics": {
     "bf16_accuracy": {
       "model_path": "Unlimited-OCR",
       "num_samples": 12,
-      "mean_cer": 0.2391877132924964,
-      "mean_digit_cer": 0.08733996757027829,
+      "mean_cer": 0.205212265458972,
+      "mean_digit_cer": 0.11330969496189873,
       "mean_cjk_cer": 0.3333333333333333,
       "mean_table_score": 1.0,
       "max_tokens": 1024,
@@ -219,8 +575,8 @@
     "reference_accuracy": {
       "model_path": "sahilchachra/unlimited-ocr-mxfp8-mlx",
       "num_samples": 12,
-      "mean_cer": 1.057248773567081,
-      "mean_digit_cer": 0.20915311607827292,
+      "mean_cer": 0.9073812810686629,
+      "mean_digit_cer": 0.4065110611974061,
       "mean_cjk_cer": 1.0,
       "mean_table_score": 1.0,
       "max_tokens": 1024,
@@ -229,23 +585,23 @@
     "candidate_accuracy": {
       "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
       "num_samples": 12,
-      "mean_cer": 0.23738438172232493,
-      "mean_digit_cer": 0.08573253598528652,
-      "mean_cjk_cer": 0.3333333333333333,
+      "mean_cer": 0.21633091868789586,
+      "mean_digit_cer": 0.11193906322597019,
+      "mean_cjk_cer": 0.3412698412698412,
       "mean_table_score": 1.0,
       "max_tokens": 1024,
       "profile": "accurate"
     },
     "reference_performance": {
       "model_path": "sahilchachra/unlimited-ocr-mxfp8-mlx",
-      "mean_tps": 312.7284719764298,
+      "mean_tps": 35.07146198590831,
       "mean_peak_memory_mb": 5165.991419999999,
       "max_tokens": 256
     },
     "candidate_performance": {
       "model_path": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
-      "mean_tps": 297.2042493840031,
-      "mean_peak_memory_mb": 5181.5234390000005,
+      "mean_tps": 35.14004923388335,
+      "mean_peak_memory_mb": 5184.062959,
       "max_tokens": 256
     }
   },
@@ -254,23 +610,28 @@
     "image_path": "test_invoice.png",
     "prompt": "<image>document parsing.",
     "force_min_tokens": 8192,
+    "generation_settings": {
+      "temperature": 0.0,
+      "no_repeat_ngram_size": 35,
+      "ngram_window": 128
+    },
     "forced_eos_token_ids": [
       1
     ],
     "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
     "processor": "arm",
-    "baseline_memory_mb": 3844.161032,
+    "baseline_memory_mb": 3846.700552,
     "test_results": [
       {
         "max_tokens": 512,
         "tokens_generated": 512,
         "tokens_generated_source": "mlx-vlm token count",
         "tokens_generated_reliable": true,
-        "elapsed_seconds": 2.4322984160098713,
-        "mean_tps": 264.34124967086365,
+        "elapsed_seconds": 17.311166541010607,
+        "mean_tps": 32.76836342370835,
         "tps_source": "mlx-vlm generation_tps",
-        "peak_memory_mb": 5181.350593,
-        "memory_growth_mb": 1337.1895610000001,
+        "peak_memory_mb": 5183.890113,
+        "memory_growth_mb": 1337.1895610000006,
         "repetition_rate": 0.0,
         "completed_naturally": false,
         "finish_reason": "length",
@@ -282,12 +643,12 @@
         "tokens_generated": 2048,
         "tokens_generated_source": "mlx-vlm token count",
         "tokens_generated_reliable": true,
-        "elapsed_seconds": 8.234726583992597,
-        "mean_tps": 261.8525825264912,
+        "elapsed_seconds": 65.73962916599703,
+        "mean_tps": 32.0896585893278,
         "tps_source": "mlx-vlm generation_tps",
-        "peak_memory_mb": 5181.612737,
-        "memory_growth_mb": 1337.1895490000002,
-        "repetition_rate": 0.09565217391304348,
+        "peak_memory_mb": 5184.152257,
+        "memory_growth_mb": 1337.1895489999997,
+        "repetition_rate": 0.20496894409937888,
         "completed_naturally": false,
         "finish_reason": "length",
         "forced_minimum_tokens": 2048,
@@ -298,12 +659,12 @@
         "tokens_generated": 8192,
         "tokens_generated_source": "mlx-vlm token count",
         "tokens_generated_reliable": true,
-        "elapsed_seconds": 31.729143417003797,
-        "mean_tps": 261.56365755570715,
+        "elapsed_seconds": 268.4810879580036,
+        "mean_tps": 30.743604031277776,
         "tps_source": "mlx-vlm generation_tps",
-        "peak_memory_mb": 5181.612737,
-        "memory_growth_mb": 1337.1895490000002,
-        "repetition_rate": 0.09565217391304348,
+        "peak_memory_mb": 5184.152257,
+        "memory_growth_mb": 1337.1895489999997,
+        "repetition_rate": 0.20496894409937888,
         "completed_naturally": false,
         "finish_reason": "length",
         "forced_minimum_tokens": 8192,
@@ -315,7 +676,7 @@
       "cache_analysis_reason": "Peak memory stayed within 64 MB across a >=4x token span",
       "tps_stable_over_length": true,
       "max_successful_tokens": 8192,
-      "max_repetition_rate": 0.09565217391304348
+      "max_repetition_rate": 0.20496894409937888
     },
     "pass_conditions": {
       "cache_bounded": true,
@@ -326,14 +687,14 @@
   },
   "sensitivity": {
     "baseline": {
-      "num_samples": 12,
-      "num_digit_samples": 12,
+      "num_samples": 10,
+      "num_digit_samples": 10,
       "num_table_samples": 3,
-      "mean_cer": 0.2391877132924964,
-      "mean_digit_cer": 0.08733996757027829,
+      "mean_cer": 0.20230911688823577,
+      "mean_digit_cer": 0.07236995104039168,
       "mean_table_score": 1.0,
-      "total_inference_time": 37.96379516503657,
-      "avg_time_per_image": 3.163649597086381
+      "total_inference_time": 126.64344700002403,
+      "avg_time_per_image": 12.664344700002403
     },
     "groups": {
       "vision_encoder": {
@@ -487,17 +848,17 @@
           "vision_model.transformer.layers.9.self_attn.qkv_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 2.6628739674476654,
-          "mean_digit_cer": 1.445363129900987,
+          "mean_cer": 2.4745422809406823,
+          "mean_digit_cer": 1.7688242659575963,
           "mean_table_score": 0.0,
-          "total_inference_time": 82.4734385000047,
-          "avg_time_per_image": 6.872786541667058
+          "total_inference_time": 310.5234005409875,
+          "avg_time_per_image": 31.05234005409875
         },
-        "cer_delta": 2.423686254155169,
-        "digit_cer_delta": 1.3580231623307086,
+        "cer_delta": 2.2722331640524467,
+        "digit_cer_delta": 1.6964543149172047,
         "table_score_degradation": 1.0,
         "sensitivity_rank": 1
       },
@@ -508,19 +869,19 @@
           "projector.layers"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.23591319611894124,
-          "mean_digit_cer": 0.08425354781719185,
+          "mean_cer": 0.20560582018493906,
+          "mean_digit_cer": 0.07236995104039168,
           "mean_table_score": 1.0,
-          "total_inference_time": 37.98282545800612,
-          "avg_time_per_image": 3.165235454833843
+          "total_inference_time": 125.59270616796857,
+          "avg_time_per_image": 12.559270616796857
         },
-        "cer_delta": -0.003274517173555175,
-        "digit_cer_delta": -0.0030864197530864335,
+        "cer_delta": 0.003296703296703285,
+        "digit_cer_delta": 0.0,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 7
+        "sensitivity_rank": 3
       },
       "token_embeddings": {
         "status": "success",
@@ -529,17 +890,17 @@
           "language_model.model.embed_tokens"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.2326891302977425,
-          "mean_digit_cer": 0.08249916185227958,
+          "mean_cer": 0.19852784584239075,
+          "mean_digit_cer": 0.07026468788249693,
           "mean_table_score": 1.0,
-          "total_inference_time": 38.66650962494896,
-          "avg_time_per_image": 3.222209135412413
+          "total_inference_time": 125.69877116801217,
+          "avg_time_per_image": 12.569877116801218
         },
-        "cer_delta": -0.006498582994753921,
-        "digit_cer_delta": -0.004840805717998706,
+        "cer_delta": -0.003781271045845025,
+        "digit_cer_delta": -0.002105263157894749,
         "table_score_degradation": 0.0,
         "sensitivity_rank": 10
       },
@@ -561,17 +922,17 @@
           "language_model.model.layers.9.self_attn.q_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.21507560490000377,
-          "mean_digit_cer": 0.09653424957157782,
+          "mean_cer": 0.1810308435920253,
+          "mean_digit_cer": 0.08710679314565484,
           "mean_table_score": 1.0,
-          "total_inference_time": 38.38538125100604,
-          "avg_time_per_image": 3.19878177091717
+          "total_inference_time": 126.60937520701555,
+          "avg_time_per_image": 12.660937520701555
         },
-        "cer_delta": -0.024112108392492643,
-        "digit_cer_delta": 0.009194282001299528,
+        "cer_delta": -0.021278273296210487,
+        "digit_cer_delta": 0.01473684210526316,
         "table_score_degradation": 0.0,
         "sensitivity_rank": 11
       },
@@ -593,17 +954,17 @@
           "language_model.model.layers.9.self_attn.k_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.23424214947533464,
-          "mean_digit_cer": 0.08425354781719185,
+          "mean_cer": 0.1995659706473317,
+          "mean_digit_cer": 0.07236995104039168,
           "mean_table_score": 1.0,
-          "total_inference_time": 38.175402873021085,
-          "avg_time_per_image": 3.181283572751757
+          "total_inference_time": 125.29329520699685,
+          "avg_time_per_image": 12.529329520699685
         },
-        "cer_delta": -0.004945563817161769,
-        "digit_cer_delta": -0.0030864197530864335,
+        "cer_delta": -0.0027431462409040763,
+        "digit_cer_delta": 0.0,
         "table_score_degradation": 0.0,
         "sensitivity_rank": 9
       },
@@ -625,19 +986,19 @@
           "language_model.model.layers.9.self_attn.v_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.23642534441989824,
-          "mean_digit_cer": 0.08425354781719185,
+          "mean_cer": 0.20121021578933468,
+          "mean_digit_cer": 0.07236995104039168,
           "mean_table_score": 1.0,
-          "total_inference_time": 41.72621229101787,
-          "avg_time_per_image": 3.477184357584823
+          "total_inference_time": 126.87845074798679,
+          "avg_time_per_image": 12.687845074798679
         },
-        "cer_delta": -0.0027623688725981754,
-        "digit_cer_delta": -0.0030864197530864335,
+        "cer_delta": -0.001098901098901095,
+        "digit_cer_delta": 0.0,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 5
+        "sensitivity_rank": 6
       },
       "attention_o_proj": {
         "status": "success",
@@ -657,19 +1018,19 @@
           "language_model.model.layers.9.self_attn.o_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.2359826017166606,
-          "mean_digit_cer": 0.07714420469193685,
+          "mean_cer": 0.2058682899333709,
+          "mean_digit_cer": 0.06383873929008568,
           "mean_table_score": 1.0,
-          "total_inference_time": 41.9442965429771,
-          "avg_time_per_image": 3.4953580452480915
+          "total_inference_time": 128.61252554102975,
+          "avg_time_per_image": 12.861252554102975
         },
-        "cer_delta": -0.003205111575835806,
-        "digit_cer_delta": -0.010195762878341438,
+        "cer_delta": 0.0035591730451351167,
+        "digit_cer_delta": -0.008531211750305995,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 6
+        "sensitivity_rank": 2
       },
       "dense_mlp": {
         "status": "success",
@@ -680,19 +1041,19 @@
           "language_model.model.layers.0.mlp.up_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.23568756434998817,
-          "mean_digit_cer": 0.08425354781719185,
+          "mean_cer": 0.20121021578933468,
+          "mean_digit_cer": 0.07236995104039168,
           "mean_table_score": 1.0,
-          "total_inference_time": 41.424109207015135,
-          "avg_time_per_image": 3.4520091005845948
+          "total_inference_time": 127.19930095798918,
+          "avg_time_per_image": 12.719930095798919
         },
-        "cer_delta": -0.0035001489425082466,
-        "digit_cer_delta": -0.0030864197530864335,
+        "cer_delta": -0.001098901098901095,
+        "digit_cer_delta": 0.0,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 8
+        "sensitivity_rank": 7
       },
       "shared_experts": {
         "status": "success",
@@ -733,19 +1094,19 @@
           "language_model.model.layers.9.mlp.shared_experts.up_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.23649792801080774,
-          "mean_digit_cer": 0.09592829886362793,
+          "mean_cer": 0.20009091014419536,
+          "mean_digit_cer": 0.08267594859241126,
           "mean_table_score": 1.0,
-          "total_inference_time": 37.84185174998129,
-          "avg_time_per_image": 3.153487645831774
+          "total_inference_time": 128.06347554198874,
+          "avg_time_per_image": 12.806347554198874
         },
-        "cer_delta": -0.0026897852816886725,
-        "digit_cer_delta": 0.00858833129334964,
+        "cer_delta": -0.002218206744040413,
+        "digit_cer_delta": 0.01030599755201958,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 4
+        "sensitivity_rank": 8
       },
       "routed_experts": {
         "status": "success",
@@ -786,19 +1147,19 @@
           "language_model.model.layers.9.mlp.switch_mlp.up_proj"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.23911822127011442,
-          "mean_digit_cer": 0.08452479009169811,
+          "mean_cer": 0.2033268371469385,
+          "mean_digit_cer": 0.06899173806609546,
           "mean_table_score": 1.0,
-          "total_inference_time": 36.668351295011234,
-          "avg_time_per_image": 3.0556959412509364
+          "total_inference_time": 125.24760646000504,
+          "avg_time_per_image": 12.524760646000505
         },
-        "cer_delta": -6.949202238198948e-05,
-        "digit_cer_delta": -0.0028151774785801775,
+        "cer_delta": 0.0010177202587027256,
+        "digit_cer_delta": -0.0033782129742962186,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 3
+        "sensitivity_rank": 5
       },
       "lm_head": {
         "status": "success",
@@ -807,57 +1168,59 @@
           "language_model.lm_head"
         ],
         "metrics": {
-          "num_samples": 12,
-          "num_digit_samples": 12,
+          "num_samples": 10,
+          "num_digit_samples": 10,
           "num_table_samples": 3,
-          "mean_cer": 0.25282999056583605,
-          "mean_digit_cer": 0.12301897991595728,
+          "mean_cer": 0.20560582018493906,
+          "mean_digit_cer": 0.07236995104039168,
           "mean_table_score": 1.0,
-          "total_inference_time": 37.152046875009546,
-          "avg_time_per_image": 3.0960039062507954
+          "total_inference_time": 122.19146683299914,
+          "avg_time_per_image": 12.219146683299915
         },
-        "cer_delta": 0.013642277273339637,
-        "digit_cer_delta": 0.035679012345678995,
+        "cer_delta": 0.003296703296703285,
+        "digit_cer_delta": 0.0,
         "table_score_degradation": 0.0,
-        "sensitivity_rank": 2
+        "sensitivity_rank": 4
       }
     }
   },
   "calibration": {
-    "schema_version": 1,
-    "created_at": "2026-07-20T23:25:55.306169+00:00",
+    "schema_version": 3,
+    "created_at": "2026-07-21T05:26:12.430585+00:00",
     "target_pattern": "language_model.lm_head",
     "selection_policy": "fastest candidate passing existing quality and throughput limits",
     "thresholds": {
-      "max_cer_delta_vs_bf16": 0.01,
+      "max_cer_delta_vs_bf16": 0.015,
       "max_cer_delta_vs_reference": 0.005,
       "max_digit_cer_delta_vs_bf16": 0.01,
       "max_table_score_degradation_vs_bf16": 0.01,
       "min_tps_ratio_vs_reference": 0.9,
-      "max_weight_size_gb": 4.5
+      "max_weight_size_gb": 4.5,
+      "min_rswa_tokens": 8192,
+      "max_rswa_repetition_rate": 0.25
     },
     "experiments": [
       {
         "label": "bf16-head",
         "precision": "bfloat16",
-        "passed": false,
+        "passed": true,
         "checks": {
           "cer": true,
           "digit_cer": true,
           "table_score": true,
-          "throughput": false
+          "throughput": true
         },
         "metrics": {
-          "mean_cer": 0.23190356458231498,
-          "mean_digit_cer": 0.08222376405546196,
+          "mean_cer": 0.19010066585760363,
+          "mean_digit_cer": 0.06278610771113831,
           "mean_table_score": 1.0,
-          "mean_tps": 271.0950821181683
+          "mean_tps": 38.770309481806514
         },
         "deltas": {
-          "cer_vs_bf16": -0.007284148710181432,
-          "digit_cer_vs_bf16": -0.005116203514816328,
+          "cer_vs_bf16": -0.012208451030632145,
+          "digit_cer_vs_bf16": -0.00958384332925337,
           "table_degradation_vs_bf16": 0.0,
-          "tps_ratio_vs_reference": 0.8668704848166195
+          "tps_ratio_vs_reference": 0.9595978674779929
         }
       },
       {
@@ -865,22 +1228,22 @@
         "precision": "mxfp8",
         "passed": false,
         "checks": {
-          "cer": false,
-          "digit_cer": true,
+          "cer": true,
+          "digit_cer": false,
           "table_score": true,
           "throughput": true
         },
         "metrics": {
-          "mean_cer": 0.26017003915600867,
-          "mean_digit_cer": 0.0769606061607251,
+          "mean_cer": 0.21607021200652196,
+          "mean_digit_cer": 0.1052422480620155,
           "mean_table_score": 1.0,
-          "mean_tps": 307.3938836867251
+          "mean_tps": 41.326731942543184
         },
         "deltas": {
-          "cer_vs_bf16": 0.020982325863512258,
-          "digit_cer_vs_bf16": -0.010379361409553187,
+          "cer_vs_bf16": 0.013761095118286187,
+          "digit_cer_vs_bf16": 0.03287229702162382,
           "table_degradation_vs_bf16": 0.0,
-          "tps_ratio_vs_reference": 0.9829417889071937
+          "tps_ratio_vs_reference": 1.0228714800563738
         }
       },
       {
@@ -894,16 +1257,16 @@
           "throughput": true
         },
         "metrics": {
-          "mean_cer": 0.23738438172232493,
-          "mean_digit_cer": 0.08573253598528652,
+          "mean_cer": 0.19105830313811087,
+          "mean_digit_cer": 0.07857558139534884,
           "mean_table_score": 1.0,
-          "mean_tps": 301.24796539578256
+          "mean_tps": 39.58614991640946
         },
         "deltas": {
-          "cer_vs_bf16": -0.0018033315701714847,
-          "digit_cer_vs_bf16": -0.0016074315849917697,
+          "cer_vs_bf16": -0.011250813750124905,
+          "digit_cer_vs_bf16": 0.0062056303549571645,
           "table_degradation_vs_bf16": 0.0,
-          "tps_ratio_vs_reference": 0.9632892185732532
+          "tps_ratio_vs_reference": 0.979790606502031
         }
       }
     ],
@@ -913,11 +1276,170 @@
     },
     "precision_overrides": {
       "language_model.lm_head": "affine8"
+    },
+    "input_artifacts": {
+      "calibration_baseline_accuracy": {
+        "filename": "calibration_baseline_accuracy.json",
+        "size": 5680,
+        "sha256": "ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6"
+      },
+      "calibration_reference_performance": {
+        "filename": "calibration_reference_performance.json",
+        "size": 2271,
+        "sha256": "98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4"
+      },
+      "calibration_bfloat16_accuracy": {
+        "filename": "calibration_bfloat16_accuracy.json",
+        "size": 5678,
+        "sha256": "5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0"
+      },
+      "calibration_bfloat16_performance": {
+        "filename": "calibration_bfloat16_performance.json",
+        "size": 2244,
+        "sha256": "dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0"
+      },
+      "calibration_mxfp8_accuracy": {
+        "filename": "calibration_mxfp8_accuracy.json",
+        "size": 5689,
+        "sha256": "77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26"
+      },
+      "calibration_mxfp8_performance": {
+        "filename": "calibration_mxfp8_performance.json",
+        "size": 2252,
+        "sha256": "e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b"
+      },
+      "calibration_affine8_accuracy": {
+        "filename": "calibration_affine8_accuracy.json",
+        "size": 5657,
+        "sha256": "c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19"
+      },
+      "calibration_affine8_performance": {
+        "filename": "calibration_affine8_performance.json",
+        "size": 2247,
+        "sha256": "f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f"
+      }
+    },
+    "dataset": {
+      "name": "calibration",
+      "num_samples": 10,
+      "num_digit_samples": 10,
+      "num_cjk_samples": 3,
+      "num_table_samples": 3,
+      "category_counts": {
+        "cjk_document": 3,
+        "english_document": 2,
+        "low_quality_scan": 2,
+        "table_numeric": 3
+      },
+      "manifest_sha256": "d4429669fe60a5c48ce82073294d20c845b85084644f9d4f62f7c8ca3923ba7a",
+      "content_sha256": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+      "files": [
+        {
+          "path": "images/0001.png",
+          "size": 243645,
+          "sha256": "8fad74c7e8f314075f34d5b3b3e6272a2c22a90e960dcf8074039b1f065895b4"
+        },
+        {
+          "path": "ground_truth/0001.txt",
+          "size": 1346,
+          "sha256": "a6f35a418836e8ba0667dc42d663219b07b65cbbf7a4627b2319f0fa44d11f8e"
+        },
+        {
+          "path": "images/0002.png",
+          "size": 208699,
+          "sha256": "31f2d9c651112e963366e5e39f3da4ff8198ec41e2187c3c8fb5fec0edd2f64c"
+        },
+        {
+          "path": "ground_truth/0002.txt",
+          "size": 838,
+          "sha256": "a05196b747f6898102fc56e23cae31c32754b985bc3eab0abe60d955bfadff68"
+        },
+        {
+          "path": "images/0003.png",
+          "size": 40039,
+          "sha256": "bbe26dc81fb4c5e6a143279dd1e0d0540df746f5e4e8f9fc7577e936882b190e"
+        },
+        {
+          "path": "ground_truth/0003.txt",
+          "size": 459,
+          "sha256": "39cf88aef99b6ce6b5d8af870e2feefca7a9991e5c835660bda60c5b5f80f481"
+        },
+        {
+          "path": "images/0004.png",
+          "size": 40356,
+          "sha256": "1e50bec7880f37e9e25eb8c4706998634b0c1f254d38fb75ccd6dbb9b89ed3a3"
+        },
+        {
+          "path": "ground_truth/0004.txt",
+          "size": 459,
+          "sha256": "2a2b723c4f1b4e14cdf5d70a74ae12586c12952ffac0d1bd295c448f1bf2d3fc"
+        },
+        {
+          "path": "images/0005.png",
+          "size": 40650,
+          "sha256": "7b7c9441ff17bb97ee76cb68eb6b9134f7d26d151c6ca1d18039959bf43de04c"
+        },
+        {
+          "path": "ground_truth/0005.txt",
+          "size": 459,
+          "sha256": "3dbec98a34736b424dafcf492691b21a1a36f629d83589cac5482cc1a135d9b7"
+        },
+        {
+          "path": "images/0006.png",
+          "size": 46036,
+          "sha256": "76006318e5aa3bb54182d7b3249ad016e7ab0dee387f580331c39b9ac49cb7ab"
+        },
+        {
+          "path": "ground_truth/0006.txt",
+          "size": 191,
+          "sha256": "75082837c3cbb8c41ba0747b98390a1f44577092c30f31f4c1e10ce52af83b61"
+        },
+        {
+          "path": "images/0007.png",
+          "size": 41828,
+          "sha256": "d642986c12233efe0bc7559096952e66eb175735f89ac91caedea0089c4adb19"
+        },
+        {
+          "path": "ground_truth/0007.txt",
+          "size": 195,
+          "sha256": "c47bf79076b7ab732c22dc1b023d9f6f12b98b724dd0d730c23e879ef7604e29"
+        },
+        {
+          "path": "images/0008.png",
+          "size": 20975,
+          "sha256": "852416613d0137144df77408c44e7d469249e7fbc650c684ad28911b96b28e76"
+        },
+        {
+          "path": "ground_truth/0008.txt",
+          "size": 201,
+          "sha256": "dc48f9c80e21438d5c2c3fb57b3bc471961708e2dd971df78a9ecc11c8c3b3ad"
+        },
+        {
+          "path": "images/0009.png",
+          "size": 487313,
+          "sha256": "87f977973bddf89f781762c87feb9c845e02731dae5cf39086db4cdb0588298e"
+        },
+        {
+          "path": "ground_truth/0009.txt",
+          "size": 121,
+          "sha256": "d9f04e21b8e73f964a9dee2776c65a76d509138f8abe694c46305a32833a52f1"
+        },
+        {
+          "path": "images/0010.png",
+          "size": 515820,
+          "sha256": "6fb40a4c63aed4cbb11346d0c80b917cf1ecd407b364bcfbc87483b5bd90d0b4"
+        },
+        {
+          "path": "ground_truth/0010.txt",
+          "size": 121,
+          "sha256": "b8e07f309069b395279995af437f82f3a9453f2add587bd9633b4ae97cc8419a"
+        }
+      ]
     }
   },
   "precision_map": {
     "vision_model": "bfloat16",
-    "projector": "mxfp8",
+    "projector": "bfloat16",
     "language_model.model.embed_tokens": "mxfp8",
     "language_model.model.norm": "bfloat16",
     "language_model.lm_head": "affine8",
@@ -949,16 +1471,16 @@
         "vision_encoder": {
           "precision": "bfloat16",
           "reasons": [
-            "cer_delta=2.423686",
-            "digit_cer_delta=1.358023",
+            "cer_delta=2.272233",
+            "digit_cer_delta=1.696454",
             "table_score_degradation=1.000000",
             "protected OCR-critical group"
           ]
         },
         "vision_projector": {
-          "precision": "mxfp8",
+          "precision": "bfloat16",
           "reasons": [
-            "all measured quality deltas were non-degrading"
+            "protected unless all measured quality deltas are non-degrading"
           ]
         },
         "token_embeddings": {
@@ -1026,69 +1548,109 @@
     }
   },
   "artifacts": {
-    "bf16_accuracy": {
-      "filename": "bf16_accuracy.json",
-      "size": 4857,
-      "sha256": "0de8511f1ba98f362d3ab712522115e534d7f129a8b587751ba5d5d8c95f272d"
-    },
     "reference_accuracy": {
       "filename": "reference_accuracy.json",
-      "size": 4923,
-      "sha256": "354d3c17451bc9e1c6ec645d36ba2e133cd1b2393dae2d7825628832e11dc365"
+      "size": 6664,
+      "sha256": "2cd131108b70f0252995c0c096bd203bd7f7e6d06ce8921e1585b8a0a4523ad7"
     },
-    "candidate_accuracy": {
-      "filename": "candidate_accuracy.json",
-      "size": 4863,
-      "sha256": "20dbd578c93afa885a2b98627b5e8335b3c68ef14e84afa66bc9392f7e6c5daf"
-    },
-    "reference_performance": {
-      "filename": "reference_performance.json",
-      "size": 2211,
-      "sha256": "74a5983677fcedab1cfe8193cc4138abfb011c6e9f2fa90d954cf3b52086888a"
+    "generated_precision_map": {
+      "filename": "generated_precision_map.json",
+      "size": 3337,
+      "sha256": "91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75"
     },
     "candidate_performance": {
       "filename": "candidate_performance.json",
-      "size": 2219,
-      "sha256": "52a7760cfcc1ce1f588fbce0642951874800fcb7952bed53644d81d91170862c"
-    },
-    "candidate_rswa": {
-      "filename": "candidate_rswa.json",
-      "size": 2374,
-      "sha256": "612e0250c99eae03d4746f3687af57ab4ee43500fab376cd59d69be460ffa429"
+      "size": 2261,
+      "sha256": "71034ab96428b1ff5f7d700166aa393bacca91a465bb265bfe33af96d91dde52"
     },
     "sensitivity_results": {
       "filename": "sensitivity_results.json",
-      "size": 22710,
-      "sha256": "63435bddcfe5d8e96921ed53eb1741ca731c9d362ab549391a436342b147d415"
+      "size": 26432,
+      "sha256": "170e57762b46a880360c3c6b89e2ed96e6ceb08454541a0baa7432e53f60ce49"
     },
     "calibration_results": {
       "filename": "calibration_results.json",
-      "size": 2542,
-      "sha256": "416c1f8e332d69d9a753bea140e264a5573875f283c9eadc79b3ea61addc3d7b"
+      "size": 8019,
+      "sha256": "b077712a0521f50e0b8c6ba1ca20e66d0ee64f750d6a545a1b3ab378af50adf7"
     },
-    "generated_precision_map": {
-      "filename": "generated_precision_map.json",
-      "size": 3315,
-      "sha256": "df9d9f6b2490d75e8589350b481fb1916a6dbf8a141c5f6958b0525d72fb1a46"
+    "candidate_accuracy": {
+      "filename": "candidate_accuracy.json",
+      "size": 6562,
+      "sha256": "60bbb1f0ac5ea87e64d57889c5ade08aa5341e9558c530c009cb7fa546e68c25"
+    },
+    "bf16_accuracy": {
+      "filename": "bf16_accuracy.json",
+      "size": 6499,
+      "sha256": "04289ab8cd0ee4019a341eaab83617eeca3cca99f2f96f236cb8725036789659"
+    },
+    "reference_performance": {
+      "filename": "reference_performance.json",
+      "size": 2271,
+      "sha256": "1ac6b646a090cd052eea48ff50cc96d666204e986273a98f43daf263cecb270d"
     },
     "provenance": {
       "filename": "provenance.json",
-      "size": 5389,
-      "sha256": "03c82c24826fdd6fdc0f5031ba67394979753584ff8949482a739dfef1719b47"
+      "size": 9280,
+      "sha256": "958930c6a480798922a88287ba53d75de13df2fa7710d387c7138bbf300601e8"
+    },
+    "candidate_rswa": {
+      "filename": "candidate_rswa.json",
+      "size": 2483,
+      "sha256": "1516b35616be2947d5595a3c9779f7b960adee4ad472336f50f5de88a151ebe0"
+    },
+    "calibration_affine8_accuracy": {
+      "filename": "calibration_affine8_accuracy.json",
+      "size": 5657,
+      "sha256": "c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19"
+    },
+    "calibration_bfloat16_accuracy": {
+      "filename": "calibration_bfloat16_accuracy.json",
+      "size": 5678,
+      "sha256": "5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0"
+    },
+    "calibration_mxfp8_performance": {
+      "filename": "calibration_mxfp8_performance.json",
+      "size": 2252,
+      "sha256": "e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b"
+    },
+    "calibration_reference_performance": {
+      "filename": "calibration_reference_performance.json",
+      "size": 2271,
+      "sha256": "98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4"
+    },
+    "calibration_mxfp8_accuracy": {
+      "filename": "calibration_mxfp8_accuracy.json",
+      "size": 5689,
+      "sha256": "77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26"
+    },
+    "calibration_affine8_performance": {
+      "filename": "calibration_affine8_performance.json",
+      "size": 2247,
+      "sha256": "f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f"
+    },
+    "calibration_bfloat16_performance": {
+      "filename": "calibration_bfloat16_performance.json",
+      "size": 2244,
+      "sha256": "dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0"
+    },
+    "calibration_baseline_accuracy": {
+      "filename": "calibration_baseline_accuracy.json",
+      "size": 5680,
+      "sha256": "ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6"
     }
   },
   "gates": [
     {
       "name": "weights_are_distinct",
       "passed": true,
-      "actual": "a4f6e9934f90564906f42929542ad68493f0c6187ad431128e447debbf52b6f4",
+      "actual": "5ff9be1dc0a833b608f6cece5b2e257c5f152eab7de160c5f07d3e5cd9462549",
       "limit": "different from 439c6acfe5e277537dfe3368b94a145a6b0da4c39e4bd00582977d030f292ad1",
       "detail": "Candidate aggregate digest must differ from the Sahil reference"
     },
     {
       "name": "weight_size_gb",
       "passed": true,
-      "actual": 3.5792221864685416,
+      "actual": 3.5815872186794877,
       "limit": 4.5,
       "detail": "Candidate Safetensors size"
     },
@@ -1101,11 +1663,30 @@
         "mxfp8": true,
         "sliding_window": true,
         "processor_class": true,
-        "sft_format": true
+        "sft_format": true,
+        "precision_map": true,
+        "quantization_summary": true,
+        "protected_modules_preserved": true
       },
       "limit": true,
       "detail": "Native Unlimited-OCR, MXFP8, and R-SWA metadata"
     },
+    {
+      "name": "candidate_source_provenance",
+      "passed": true,
+      "actual": {
+        "model": "Unlimited-OCR",
+        "revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead"
+      },
+      "limit": {
+        "model": [
+          "baidu/Unlimited-OCR",
+          "Unlimited-OCR"
+        ],
+        "revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead"
+      },
+      "detail": "Converted metadata must bind the exact BF16 source commit"
+    },
     {
       "name": "evaluation_coverage",
       "passed": true,
@@ -1119,34 +1700,168 @@
       "detail": "Dataset must cover ordinary text, digits, CJK, and tables"
     },
     {
-      "name": "same_evaluation_samples",
+      "name": "held_out_evaluation_dataset",
+      "passed": true,
+      "actual": {
+        "passed": true,
+        "calibration_digest": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+        "evaluation_digest": "7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987",
+        "overlapping_image_sha256": [],
+        "overlapping_ground_truth_sha256": []
+      },
+      "limit": "distinct dataset digests and no shared image or ground-truth hashes",
+      "detail": "Final quality evidence must not reuse model-selection samples"
+    },
+    {
+      "name": "accuracy_aggregates_recomputed",
       "passed": true,
       "actual": [
-        12,
-        12,
-        12
+        true,
+        true,
+        true
+      ],
+      "limit": [
+        true,
+        true,
+        true
       ],
-      "limit": 12,
-      "detail": "All three checkpoints must run the complete evaluation set"
+      "detail": "Every reported accuracy aggregate must recompute from per-file rows"
+    },
+    {
+      "name": "same_evaluation_samples",
+      "passed": true,
+      "actual": {
+        "counts": [
+          12,
+          12,
+          12
+        ],
+        "file_counts": [
+          12,
+          12,
+          12
+        ]
+      },
+      "limit": {
+        "count": 12,
+        "files": [
+          "0001.png",
+          "0002.png",
+          "0003.png",
+          "0004.png",
+          "0005.png",
+          "0006.png",
+          "0007.png",
+          "0008.png",
+          "0009.png",
+          "0010.png",
+          "0011.png",
+          "0012.png"
+        ]
+      },
+      "detail": "All three checkpoints must run every identical evaluation file"
+    },
+    {
+      "name": "model_identities",
+      "passed": true,
+      "actual": {
+        "bf16": "Unlimited-OCR",
+        "reference": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+        "candidate": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+        "reference_performance": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+        "candidate_performance": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+        "rswa": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8"
+      },
+      "limit": {
+        "bf16": "Unlimited-OCR",
+        "reference": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+        "candidate": "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8"
+      },
+      "detail": "Accuracy, performance, and R-SWA evidence must identify exact models"
+    },
+    {
+      "name": "immutable_model_revisions",
+      "passed": true,
+      "actual": {
+        "bf16_accuracy": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+        "reference_accuracy": "55b8031a6c867de675279d9604e38cc94b9882a4",
+        "candidate_accuracy": null,
+        "reference_performance": "55b8031a6c867de675279d9604e38cc94b9882a4",
+        "candidate_performance": null
+      },
+      "limit": {
+        "bf16_accuracy": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+        "reference_accuracy": "55b8031a6c867de675279d9604e38cc94b9882a4",
+        "candidate_accuracy": null,
+        "reference_performance": "55b8031a6c867de675279d9604e38cc94b9882a4",
+        "candidate_performance": null
+      },
+      "detail": "Remote source and reference evidence must name exact Hub commits"
+    },
+    {
+      "name": "same_accuracy_recipe",
+      "passed": true,
+      "actual": {
+        "prompt": [
+          "<image>document parsing.",
+          "<image>document parsing.",
+          "<image>document parsing."
+        ],
+        "max_tokens": [
+          1024,
+          1024,
+          1024
+        ],
+        "profile": [
+          "accurate",
+          "accurate",
+          "accurate"
+        ],
+        "generation_settings": [
+          {
+            "temperature": 0.0,
+            "top_p": 1.0,
+            "repetition_penalty": 1.0,
+            "no_repeat_ngram_size": 35,
+            "ngram_window": 128
+          },
+          {
+            "temperature": 0.0,
+            "top_p": 1.0,
+            "repetition_penalty": 1.0,
+            "no_repeat_ngram_size": 35,
+            "ngram_window": 128
+          },
+          {
+            "temperature": 0.0,
+            "top_p": 1.0,
+            "repetition_penalty": 1.0,
+            "no_repeat_ngram_size": 35,
+            "ngram_window": 128
+          }
+        ]
+      },
+      "limit": "identical official MLX OCR recipe",
+      "detail": "Accuracy runs must use the same deterministic OCR generation settings"
     },
     {
       "name": "candidate_cer_vs_bf16",
       "passed": true,
-      "actual": -0.0018033315701714847,
-      "limit": 0.01,
+      "actual": 0.011118653228923858,
+      "limit": 0.015,
       "detail": "Candidate minus BF16 absolute mean CER"
     },
     {
       "name": "candidate_cer_vs_reference",
       "passed": true,
-      "actual": -0.8198643918447561,
+      "actual": -0.691050362380767,
       "limit": 0.005,
       "detail": "Candidate minus Sahil-reference absolute mean CER"
     },
     {
       "name": "candidate_digit_cer_vs_bf16",
       "passed": true,
-      "actual": -0.0016074315849917697,
+      "actual": -0.0013706317359285375,
       "limit": 0.01,
       "detail": "Candidate minus BF16 digit CER"
     },
@@ -1157,10 +1872,73 @@
       "limit": 0.01,
       "detail": "BF16 minus candidate mean table score"
     },
+    {
+      "name": "performance_aggregates_recomputed",
+      "passed": true,
+      "actual": {
+        "reference": true,
+        "candidate": true
+      },
+      "limit": {
+        "reference": true,
+        "candidate": true
+      },
+      "detail": "Performance means must recompute from complete benchmark runs"
+    },
+    {
+      "name": "same_performance_setup",
+      "passed": true,
+      "actual": {
+        "image_path": [
+          "test_invoice.png",
+          "test_invoice.png"
+        ],
+        "prompt": [
+          "<image>document parsing.",
+          "<image>document parsing."
+        ],
+        "max_tokens": [
+          256,
+          256
+        ],
+        "num_warmup": [
+          1,
+          1
+        ],
+        "num_runs": [
+          3,
+          3
+        ],
+        "system": [
+          {
+            "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
+            "processor": "arm",
+            "python_version": "3.14.6",
+            "machine": "arm64",
+            "mlx_version": "0.32.0",
+            "mlx_vlm_version": "0.6.6",
+            "chip": "Apple M3 Max",
+            "total_memory_gb": 128.0
+          },
+          {
+            "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O",
+            "processor": "arm",
+            "python_version": "3.14.6",
+            "machine": "arm64",
+            "mlx_version": "0.32.0",
+            "mlx_vlm_version": "0.6.6",
+            "chip": "Apple M3 Max",
+            "total_memory_gb": 128.0
+          }
+        ]
+      },
+      "limit": "identical setup with at least three complete runs",
+      "detail": "Performance comparisons must use the same host and benchmark recipe"
+    },
     {
       "name": "candidate_tps_vs_reference",
       "passed": true,
-      "actual": 0.9503587809120344,
+      "actual": 1.001955642681865,
       "limit": 0.9,
       "detail": "Candidate decode throughput divided by Sahil-reference throughput"
     },
@@ -1168,12 +1946,134 @@
       "name": "rswa_8k_bounded",
       "passed": true,
       "actual": {
-        "cache_bounded": true,
-        "tps_stable": true,
-        "8k_test_passed": true
+        "pass_conditions": {
+          "cache_bounded": true,
+          "tps_stable": true,
+          "8k_test_passed": true
+        },
+        "tokens": 8192,
+        "repetition_rate": 0.20496894409937888
       },
-      "limit": true,
-      "detail": "8K generation, bounded cache, and stable throughput"
+      "limit": {
+        "min_tokens": 8192,
+        "max_repetition_rate": 0.25
+      },
+      "detail": "8K generation must have bounded cache, stable throughput, and repetition"
+    },
+    {
+      "name": "provenance_matches_release",
+      "passed": true,
+      "actual": {
+        "source": "baidu/Unlimited-OCR",
+        "reference": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+        "target": "AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+        "source_config": {
+          "path": "config.json",
+          "sha256": "27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9"
+        },
+        "source_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+        "reference_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
+        "dataset_digest": "7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987",
+        "calibration_dataset_digest": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb"
+      },
+      "limit": {
+        "source": "baidu/Unlimited-OCR",
+        "reference": "sahilchachra/unlimited-ocr-mxfp8-mlx",
+        "target": "AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+        "source_config_sha256": "27246d03fd670904ec9601b1cb0861fbb79ec076830771daa8d943d6229946f9",
+        "source_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+        "reference_revision": "55b8031a6c867de675279d9604e38cc94b9882a4",
+        "dataset_digest": "7becb54d81bae0dc69092ff69bae4a324ab63dd672123e5de5d19b6baebbc987",
+        "calibration_dataset_digest": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb"
+      },
+      "detail": "Provenance must bind the source, selection/evaluation datasets, and target"
+    },
+    {
+      "name": "calibration_recomputed",
+      "passed": true,
+      "actual": {
+        "selected": {
+          "label": "affine8-head",
+          "precision": "affine8"
+        },
+        "models": [
+          "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-affine8",
+          "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-bfloat16",
+          "AX-Unlimited-OCR-3B-MoE-MLX-MXFP8-cal-mxfp8"
+        ],
+        "dataset_digest": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+        "input_artifacts": {
+          "calibration_baseline_accuracy": {
+            "filename": "calibration_baseline_accuracy.json",
+            "size": 5680,
+            "sha256": "ea66cc3190566173dd29c3da3c624b3c950487ebe7b80375d0b159fe46f58cd6"
+          },
+          "calibration_reference_performance": {
+            "filename": "calibration_reference_performance.json",
+            "size": 2271,
+            "sha256": "98b8082733af986c054d2d888722bdc28eb92c6accbb3d1d0b70cd2cffc0aae4"
+          },
+          "calibration_bfloat16_accuracy": {
+            "filename": "calibration_bfloat16_accuracy.json",
+            "size": 5678,
+            "sha256": "5c124b0ffbc8cda32bc941e54b86d8e83f48f9ab05377e1d63b262909b8b1de0"
+          },
+          "calibration_bfloat16_performance": {
+            "filename": "calibration_bfloat16_performance.json",
+            "size": 2244,
+            "sha256": "dba85444f023def084fcd41fb1a404a18010a003557f85bb2070cfda413ec2b0"
+          },
+          "calibration_mxfp8_accuracy": {
+            "filename": "calibration_mxfp8_accuracy.json",
+            "size": 5689,
+            "sha256": "77cbe174170ea41896d26176b30a0320754eef8f62977c08f03f41e161659d26"
+          },
+          "calibration_mxfp8_performance": {
+            "filename": "calibration_mxfp8_performance.json",
+            "size": 2252,
+            "sha256": "e6ad66839447d8b24b7c549fb644a547863a6277ba80d392b9d74572c420099b"
+          },
+          "calibration_affine8_accuracy": {
+            "filename": "calibration_affine8_accuracy.json",
+            "size": 5657,
+            "sha256": "c1f7d2ba7a447c2fb14156e3fabda97b7577035d79bf114ea8ccacec3771ac19"
+          },
+          "calibration_affine8_performance": {
+            "filename": "calibration_affine8_performance.json",
+            "size": 2247,
+            "sha256": "f1c8b7d22cc691797f9e221784f2a747386ea81c5ee1286bcd2860a1670bf89f"
+          }
+        }
+      },
+      "limit": "content-addressed inputs and fastest passing experiment",
+      "detail": "Calibration claims must recompute from raw benchmark evidence"
+    },
+    {
+      "name": "sensitivity_matches_calibration_dataset",
+      "passed": true,
+      "actual": {
+        "dataset_digest": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+        "baseline_metrics_match": true
+      },
+      "limit": {
+        "dataset_digest": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+        "baseline_metrics_match": true
+      },
+      "detail": "Sensitivity decisions must use the recorded selection dataset and baseline"
+    },
+    {
+      "name": "candidate_precision_map_matches_evidence",
+      "passed": true,
+      "actual": "91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75",
+      "limit": "91163d306379c513935c877d53344c628d333cf4eba2f75f0f5f00889affcd75",
+      "detail": "The executable candidate precision map must equal approved evidence"
+    },
+    {
+      "name": "precision_map_reproducible",
+      "passed": true,
+      "actual": "c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a",
+      "limit": "c5f8b260d2e1ff2fc0405c6ef3857829ca85f49b1980a8c599fb03e06788eb1a",
+      "detail": "Sensitivity and calibration evidence must reproduce the executable map"
     }
   ],
   "environment": {

release/sensitivity_results.json

@@ -1,16 +1,134 @@
 {
   "baseline": {
-    "num_samples": 12,
-    "num_digit_samples": 12,
+    "num_samples": 10,
+    "num_digit_samples": 10,
     "num_table_samples": 3,
-    "mean_cer": 0.2391877132924964,
-    "mean_digit_cer": 0.08733996757027829,
+    "mean_cer": 0.20230911688823577,
+    "mean_digit_cer": 0.07236995104039168,
     "mean_table_score": 1.0,
-    "total_inference_time": 37.96379516503657,
-    "avg_time_per_image": 3.163649597086381
+    "total_inference_time": 126.64344700002403,
+    "avg_time_per_image": 12.664344700002403
   },
   "model_path": "baidu/Unlimited-OCR",
-  "eval_dir": "eval_dataset",
+  "source_revision": "ee63731b6461c8afcdcc7b15352e7d2ffecc2ead",
+  "eval_dir": "calibration",
+  "dataset": {
+    "name": "calibration",
+    "num_samples": 10,
+    "num_digit_samples": 10,
+    "num_cjk_samples": 3,
+    "num_table_samples": 3,
+    "category_counts": {
+      "cjk_document": 3,
+      "english_document": 2,
+      "low_quality_scan": 2,
+      "table_numeric": 3
+    },
+    "manifest_sha256": "d4429669fe60a5c48ce82073294d20c845b85084644f9d4f62f7c8ca3923ba7a",
+    "content_sha256": "40260fc715daa552e7b7dcd75d82441065710eccc8b90892b18826cad6f8dfcb",
+    "files": [
+      {
+        "path": "images/0001.png",
+        "size": 243645,
+        "sha256": "8fad74c7e8f314075f34d5b3b3e6272a2c22a90e960dcf8074039b1f065895b4"
+      },
+      {
+        "path": "ground_truth/0001.txt",
+        "size": 1346,
+        "sha256": "a6f35a418836e8ba0667dc42d663219b07b65cbbf7a4627b2319f0fa44d11f8e"
+      },
+      {
+        "path": "images/0002.png",
+        "size": 208699,
+        "sha256": "31f2d9c651112e963366e5e39f3da4ff8198ec41e2187c3c8fb5fec0edd2f64c"
+      },
+      {
+        "path": "ground_truth/0002.txt",
+        "size": 838,
+        "sha256": "a05196b747f6898102fc56e23cae31c32754b985bc3eab0abe60d955bfadff68"
+      },
+      {
+        "path": "images/0003.png",
+        "size": 40039,
+        "sha256": "bbe26dc81fb4c5e6a143279dd1e0d0540df746f5e4e8f9fc7577e936882b190e"
+      },
+      {
+        "path": "ground_truth/0003.txt",
+        "size": 459,
+        "sha256": "39cf88aef99b6ce6b5d8af870e2feefca7a9991e5c835660bda60c5b5f80f481"
+      },
+      {
+        "path": "images/0004.png",
+        "size": 40356,
+        "sha256": "1e50bec7880f37e9e25eb8c4706998634b0c1f254d38fb75ccd6dbb9b89ed3a3"
+      },
+      {
+        "path": "ground_truth/0004.txt",
+        "size": 459,
+        "sha256": "2a2b723c4f1b4e14cdf5d70a74ae12586c12952ffac0d1bd295c448f1bf2d3fc"
+      },
+      {
+        "path": "images/0005.png",
+        "size": 40650,
+        "sha256": "7b7c9441ff17bb97ee76cb68eb6b9134f7d26d151c6ca1d18039959bf43de04c"
+      },
+      {
+        "path": "ground_truth/0005.txt",
+        "size": 459,
+        "sha256": "3dbec98a34736b424dafcf492691b21a1a36f629d83589cac5482cc1a135d9b7"
+      },
+      {
+        "path": "images/0006.png",
+        "size": 46036,
+        "sha256": "76006318e5aa3bb54182d7b3249ad016e7ab0dee387f580331c39b9ac49cb7ab"
+      },
+      {
+        "path": "ground_truth/0006.txt",
+        "size": 191,
+        "sha256": "75082837c3cbb8c41ba0747b98390a1f44577092c30f31f4c1e10ce52af83b61"
+      },
+      {
+        "path": "images/0007.png",
+        "size": 41828,
+        "sha256": "d642986c12233efe0bc7559096952e66eb175735f89ac91caedea0089c4adb19"
+      },
+      {
+        "path": "ground_truth/0007.txt",
+        "size": 195,
+        "sha256": "c47bf79076b7ab732c22dc1b023d9f6f12b98b724dd0d730c23e879ef7604e29"
+      },
+      {
+        "path": "images/0008.png",
+        "size": 20975,
+        "sha256": "852416613d0137144df77408c44e7d469249e7fbc650c684ad28911b96b28e76"
+      },
+      {
+        "path": "ground_truth/0008.txt",
+        "size": 201,
+        "sha256": "dc48f9c80e21438d5c2c3fb57b3bc471961708e2dd971df78a9ecc11c8c3b3ad"
+      },
+      {
+        "path": "images/0009.png",
+        "size": 487313,
+        "sha256": "87f977973bddf89f781762c87feb9c845e02731dae5cf39086db4cdb0588298e"
+      },
+      {
+        "path": "ground_truth/0009.txt",
+        "size": 121,
+        "sha256": "d9f04e21b8e73f964a9dee2776c65a76d509138f8abe694c46305a32833a52f1"
+      },
+      {
+        "path": "images/0010.png",
+        "size": 515820,
+        "sha256": "6fb40a4c63aed4cbb11346d0c80b917cf1ecd407b364bcfbc87483b5bd90d0b4"
+      },
+      {
+        "path": "ground_truth/0010.txt",
+        "size": 121,
+        "sha256": "b8e07f309069b395279995af437f82f3a9453f2add587bd9633b4ae97cc8419a"
+      }
+    ]
+  },
   "prompt": "<image>document parsing.",
   "max_tokens": 1024,
   "groups_tested": [
@@ -178,17 +296,17 @@
         "vision_model.transformer.layers.9.self_attn.qkv_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 2.6628739674476654,
-        "mean_digit_cer": 1.445363129900987,
+        "mean_cer": 2.4745422809406823,
+        "mean_digit_cer": 1.7688242659575963,
         "mean_table_score": 0.0,
-        "total_inference_time": 82.4734385000047,
-        "avg_time_per_image": 6.872786541667058
+        "total_inference_time": 310.5234005409875,
+        "avg_time_per_image": 31.05234005409875
       },
-      "cer_delta": 2.423686254155169,
-      "digit_cer_delta": 1.3580231623307086,
+      "cer_delta": 2.2722331640524467,
+      "digit_cer_delta": 1.6964543149172047,
       "table_score_degradation": 1.0,
       "sensitivity_rank": 1
     },
@@ -199,19 +317,19 @@
         "projector.layers"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.23591319611894124,
-        "mean_digit_cer": 0.08425354781719185,
+        "mean_cer": 0.20560582018493906,
+        "mean_digit_cer": 0.07236995104039168,
         "mean_table_score": 1.0,
-        "total_inference_time": 37.98282545800612,
-        "avg_time_per_image": 3.165235454833843
+        "total_inference_time": 125.59270616796857,
+        "avg_time_per_image": 12.559270616796857
       },
-      "cer_delta": -0.003274517173555175,
-      "digit_cer_delta": -0.0030864197530864335,
+      "cer_delta": 0.003296703296703285,
+      "digit_cer_delta": 0.0,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 7
+      "sensitivity_rank": 3
     },
     "token_embeddings": {
       "status": "success",
@@ -220,17 +338,17 @@
         "language_model.model.embed_tokens"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.2326891302977425,
-        "mean_digit_cer": 0.08249916185227958,
+        "mean_cer": 0.19852784584239075,
+        "mean_digit_cer": 0.07026468788249693,
         "mean_table_score": 1.0,
-        "total_inference_time": 38.66650962494896,
-        "avg_time_per_image": 3.222209135412413
+        "total_inference_time": 125.69877116801217,
+        "avg_time_per_image": 12.569877116801218
       },
-      "cer_delta": -0.006498582994753921,
-      "digit_cer_delta": -0.004840805717998706,
+      "cer_delta": -0.003781271045845025,
+      "digit_cer_delta": -0.002105263157894749,
       "table_score_degradation": 0.0,
       "sensitivity_rank": 10
     },
@@ -252,17 +370,17 @@
         "language_model.model.layers.9.self_attn.q_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.21507560490000377,
-        "mean_digit_cer": 0.09653424957157782,
+        "mean_cer": 0.1810308435920253,
+        "mean_digit_cer": 0.08710679314565484,
         "mean_table_score": 1.0,
-        "total_inference_time": 38.38538125100604,
-        "avg_time_per_image": 3.19878177091717
+        "total_inference_time": 126.60937520701555,
+        "avg_time_per_image": 12.660937520701555
       },
-      "cer_delta": -0.024112108392492643,
-      "digit_cer_delta": 0.009194282001299528,
+      "cer_delta": -0.021278273296210487,
+      "digit_cer_delta": 0.01473684210526316,
       "table_score_degradation": 0.0,
       "sensitivity_rank": 11
     },
@@ -284,17 +402,17 @@
         "language_model.model.layers.9.self_attn.k_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.23424214947533464,
-        "mean_digit_cer": 0.08425354781719185,
+        "mean_cer": 0.1995659706473317,
+        "mean_digit_cer": 0.07236995104039168,
         "mean_table_score": 1.0,
-        "total_inference_time": 38.175402873021085,
-        "avg_time_per_image": 3.181283572751757
+        "total_inference_time": 125.29329520699685,
+        "avg_time_per_image": 12.529329520699685
       },
-      "cer_delta": -0.004945563817161769,
-      "digit_cer_delta": -0.0030864197530864335,
+      "cer_delta": -0.0027431462409040763,
+      "digit_cer_delta": 0.0,
       "table_score_degradation": 0.0,
       "sensitivity_rank": 9
     },
@@ -316,19 +434,19 @@
         "language_model.model.layers.9.self_attn.v_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.23642534441989824,
-        "mean_digit_cer": 0.08425354781719185,
+        "mean_cer": 0.20121021578933468,
+        "mean_digit_cer": 0.07236995104039168,
         "mean_table_score": 1.0,
-        "total_inference_time": 41.72621229101787,
-        "avg_time_per_image": 3.477184357584823
+        "total_inference_time": 126.87845074798679,
+        "avg_time_per_image": 12.687845074798679
       },
-      "cer_delta": -0.0027623688725981754,
-      "digit_cer_delta": -0.0030864197530864335,
+      "cer_delta": -0.001098901098901095,
+      "digit_cer_delta": 0.0,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 5
+      "sensitivity_rank": 6
     },
     "attention_o_proj": {
       "status": "success",
@@ -348,19 +466,19 @@
         "language_model.model.layers.9.self_attn.o_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.2359826017166606,
-        "mean_digit_cer": 0.07714420469193685,
+        "mean_cer": 0.2058682899333709,
+        "mean_digit_cer": 0.06383873929008568,
         "mean_table_score": 1.0,
-        "total_inference_time": 41.9442965429771,
-        "avg_time_per_image": 3.4953580452480915
+        "total_inference_time": 128.61252554102975,
+        "avg_time_per_image": 12.861252554102975
       },
-      "cer_delta": -0.003205111575835806,
-      "digit_cer_delta": -0.010195762878341438,
+      "cer_delta": 0.0035591730451351167,
+      "digit_cer_delta": -0.008531211750305995,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 6
+      "sensitivity_rank": 2
     },
     "dense_mlp": {
       "status": "success",
@@ -371,19 +489,19 @@
         "language_model.model.layers.0.mlp.up_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.23568756434998817,
-        "mean_digit_cer": 0.08425354781719185,
+        "mean_cer": 0.20121021578933468,
+        "mean_digit_cer": 0.07236995104039168,
         "mean_table_score": 1.0,
-        "total_inference_time": 41.424109207015135,
-        "avg_time_per_image": 3.4520091005845948
+        "total_inference_time": 127.19930095798918,
+        "avg_time_per_image": 12.719930095798919
       },
-      "cer_delta": -0.0035001489425082466,
-      "digit_cer_delta": -0.0030864197530864335,
+      "cer_delta": -0.001098901098901095,
+      "digit_cer_delta": 0.0,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 8
+      "sensitivity_rank": 7
     },
     "shared_experts": {
       "status": "success",
@@ -424,19 +542,19 @@
         "language_model.model.layers.9.mlp.shared_experts.up_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.23649792801080774,
-        "mean_digit_cer": 0.09592829886362793,
+        "mean_cer": 0.20009091014419536,
+        "mean_digit_cer": 0.08267594859241126,
         "mean_table_score": 1.0,
-        "total_inference_time": 37.84185174998129,
-        "avg_time_per_image": 3.153487645831774
+        "total_inference_time": 128.06347554198874,
+        "avg_time_per_image": 12.806347554198874
       },
-      "cer_delta": -0.0026897852816886725,
-      "digit_cer_delta": 0.00858833129334964,
+      "cer_delta": -0.002218206744040413,
+      "digit_cer_delta": 0.01030599755201958,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 4
+      "sensitivity_rank": 8
     },
     "routed_experts": {
       "status": "success",
@@ -477,19 +595,19 @@
         "language_model.model.layers.9.mlp.switch_mlp.up_proj"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.23911822127011442,
-        "mean_digit_cer": 0.08452479009169811,
+        "mean_cer": 0.2033268371469385,
+        "mean_digit_cer": 0.06899173806609546,
         "mean_table_score": 1.0,
-        "total_inference_time": 36.668351295011234,
-        "avg_time_per_image": 3.0556959412509364
+        "total_inference_time": 125.24760646000504,
+        "avg_time_per_image": 12.524760646000505
       },
-      "cer_delta": -6.949202238198948e-05,
-      "digit_cer_delta": -0.0028151774785801775,
+      "cer_delta": 0.0010177202587027256,
+      "digit_cer_delta": -0.0033782129742962186,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 3
+      "sensitivity_rank": 5
     },
     "lm_head": {
       "status": "success",
@@ -498,19 +616,19 @@
         "language_model.lm_head"
       ],
       "metrics": {
-        "num_samples": 12,
-        "num_digit_samples": 12,
+        "num_samples": 10,
+        "num_digit_samples": 10,
         "num_table_samples": 3,
-        "mean_cer": 0.25282999056583605,
-        "mean_digit_cer": 0.12301897991595728,
+        "mean_cer": 0.20560582018493906,
+        "mean_digit_cer": 0.07236995104039168,
         "mean_table_score": 1.0,
-        "total_inference_time": 37.152046875009546,
-        "avg_time_per_image": 3.0960039062507954
+        "total_inference_time": 122.19146683299914,
+        "avg_time_per_image": 12.219146683299915
       },
-      "cer_delta": 0.013642277273339637,
-      "digit_cer_delta": 0.035679012345678995,
+      "cer_delta": 0.003296703296703285,
+      "digit_cer_delta": 0.0,
       "table_score_degradation": 0.0,
-      "sensitivity_rank": 2
+      "sensitivity_rank": 4
     }
   }
-}
+}

src/unlimited_ocr/__init__.py

@@ -1,15 +1,18 @@
-"""Unlimited-OCR MLX — Improved inference pipeline for Apple Silicon."""
+"""Unlimited-OCR inference toolkit for MLX and CUDA/vLLM."""
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"
 
 from unlimited_ocr.pipeline import BatchProcessingError, OCRPipeline
 from unlimited_ocr.engine import OCREngine
 from unlimited_ocr.profiles import get_profile, list_profiles, PROFILES
+from unlimited_ocr.vllm_backend import VLLMEngine, VLLMEngineConfig
 
 __all__ = [
     "OCRPipeline",
     "BatchProcessingError",
     "OCREngine",
+    "VLLMEngine",
+    "VLLMEngineConfig",
     "get_profile",
     "list_profiles",
     "PROFILES",

src/unlimited_ocr/adapter_registry.json

@@ -5,12 +5,20 @@
     "table": {
       "path": "adapters/table-v1",
       "hf_repo": "AutomatosX/unlimited-ocr-lora-table",
-      "description": "Table structure recognition adapter"
+      "description": "Table structure recognition adapter",
+      "base_model": "AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+      "target_scope": "all supported language-model linear modules except lm_head",
+      "lora_rank": 16,
+      "scene": "table_numeric"
     },
     "receipt": {
       "path": "adapters/receipt-v1",
       "hf_repo": "AutomatosX/unlimited-ocr-lora-receipt",
-      "description": "Receipt and invoice parsing adapter"
+      "description": "Receipt and invoice parsing adapter",
+      "base_model": "AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8",
+      "target_scope": "all supported language-model linear modules except lm_head",
+      "lora_rank": 16,
+      "scene": "receipt"
     }
   },
   "default": null

src/unlimited_ocr/cli.py

@@ -1,4 +1,4 @@
-"""Command-line interface for Unlimited-OCR MLX."""
+"""Command-line interface for Unlimited-OCR MLX and vLLM backends."""
 
 from __future__ import annotations
 
@@ -12,6 +12,7 @@ from unlimited_ocr.engine import DEFAULT_MODEL
 from unlimited_ocr.pdf import DEFAULT_PDF_DPI
 from unlimited_ocr.preprocessing import PREPROCESS_MODES
 from unlimited_ocr.profiles import PROFILES
+from unlimited_ocr.vllm_backend import DEFAULT_VLLM_BASE_URL, DEFAULT_VLLM_MODEL
 
 console = Console()
 status_console = Console(stderr=True)
@@ -65,7 +66,11 @@ def _resolve_adapter(name_or_path: str) -> str:
                     from huggingface_hub import snapshot_download
                     return snapshot_download(
                         hf_repo,
-                        allow_patterns=["adapter_config.json", "adapters.safetensors"],
+                        allow_patterns=[
+                            "adapter_config.json",
+                            "adapters.safetensors",
+                            "training_config.json",
+                        ],
                     )
                 except Exception as exc:
                     raise click.ClickException(
@@ -84,10 +89,11 @@ def _resolve_adapter(name_or_path: str) -> str:
 @click.pass_context
 @click.version_option(version=__version__, prog_name="unlimited-ocr")
 def main(ctx):
-    """Unlimited-OCR MLX — One-shot document parsing on Apple Silicon.
+    """Unlimited-OCR — document parsing with MLX or CUDA/vLLM.
 
     Run OCR on images, PDFs, or entire directories using the Unlimited-OCR
-    vision-language model optimized for MLX (MX FP8 quantized, ~3.6 GB).
+    vision-language model through local MLX or an OpenAI-compatible vLLM
+    server on NVIDIA CUDA.
 
     \b
     Examples:
@@ -123,18 +129,32 @@ def main(ctx):
               help="DPI for PDF rendering (150 is usually enough; model input is 1024px).")
 @click.option("--max-tokens", type=click.IntRange(min=1), default=None,
               help="Maximum tokens per page (overrides profile default).")
-@click.option("--model-path", type=str, default=DEFAULT_MODEL,
-              help="Model path (HF repo ID or local directory).")
+@click.option("--backend", type=click.Choice(["mlx", "vllm"]), default="mlx",
+              show_default=True, help="Inference backend.")
+@click.option("--model-path", type=str, default=None,
+              help="Model path/served name. Defaults depend on --backend.")
+@click.option("--vllm-base-url", type=str, default=DEFAULT_VLLM_BASE_URL,
+              show_default=True, help="OpenAI-compatible vLLM API root.")
+@click.option("--vllm-api-key", type=str, default=None, envvar="VLLM_API_KEY",
+              help="vLLM bearer token (or set VLLM_API_KEY).")
+@click.option("--vllm-timeout", type=click.FloatRange(min=0.001), default=3600.0,
+              show_default=True, help="vLLM request timeout in seconds.")
+@click.option("--vllm-verify-tls/--no-vllm-verify-tls", default=True,
+              help="Verify TLS certificates for the vLLM endpoint.")
 @click.option("--verbose", is_flag=True, default=False,
               help="Enable verbose output.")
 @click.option("--stream", is_flag=True, default=False,
               help="Stream output token-by-token (single image only).")
 @click.option("--jobs", type=click.IntRange(min=1), default=1,
               help="Parallel preprocessing workers for batch mode.")
+@click.option("--request-concurrency", type=click.IntRange(min=1), default=1,
+              help="Concurrent vLLM requests for directory batch mode.")
 @click.option("--adapter", type=str, default=None,
               help="LoRA adapter name or path (e.g. 'table' or 'adapters/table-v1').")
 def run(input_path, output_path, fmt, task, profile, grounding, preprocess,
-        preprocess_mode, dpi, max_tokens, model_path, verbose, stream, jobs, adapter):
+        preprocess_mode, dpi, max_tokens, backend, model_path, vllm_base_url,
+        vllm_api_key, vllm_timeout, vllm_verify_tls, verbose, stream, jobs,
+        request_concurrency, adapter):
     """Run OCR on an image, PDF, or directory of images.
 
     INPUT_PATH can be an image file, a PDF file, or a directory of images.
@@ -154,12 +174,30 @@ def run(input_path, output_path, fmt, task, profile, grounding, preprocess,
             raise click.UsageError("--stream does not support PDF files")
         if jobs != 1:
             raise click.UsageError("--jobs only applies to directory batch processing")
-    elif not input_path.is_dir() and jobs != 1:
-        raise click.UsageError("--jobs only applies to directory batch processing")
+        if request_concurrency != 1:
+            raise click.UsageError(
+                "--request-concurrency only applies to directory batch processing"
+            )
+    elif not input_path.is_dir() and (jobs != 1 or request_concurrency != 1):
+        raise click.UsageError(
+            "--jobs and --request-concurrency only apply to directory batch processing"
+        )
+    if request_concurrency > 1 and backend != "vllm":
+        raise click.UsageError(
+            "--request-concurrency > 1 requires --backend vllm"
+        )
+    if backend == "vllm" and adapter:
+        raise click.UsageError(
+            "--adapter is MLX-only; configure LoRA modules when starting vLLM"
+        )
+
+    resolved_model = model_path or (
+        DEFAULT_VLLM_MODEL if backend == "vllm" else DEFAULT_MODEL
+    )
 
     if verbose:
-        status_console.print(f"[bold]Unlimited-OCR MLX[/] v{__version__}")
-        status_console.print(f"[dim]Model:[/] {model_path}")
+        status_console.print(f"[bold]Unlimited-OCR {backend.upper()}[/] v{__version__}")
+        status_console.print(f"[dim]Model:[/] {resolved_model}")
         status_console.print(f"[dim]Input:[/] {input_path}")
         status_console.print(f"[dim]Profile:[/] {profile} | Format: {fmt} | Task: {task}")
         if preprocess:
@@ -174,17 +212,23 @@ def run(input_path, output_path, fmt, task, profile, grounding, preprocess,
         adapter_path = _resolve_adapter(adapter) if adapter else None
 
         pipeline = OCRPipeline(
-            model_path=model_path,
+            backend=backend,
+            model_path=resolved_model,
             verbose=verbose,
             preprocess=preprocess,
             preprocess_mode=preprocess_mode,
             profile=profile,
             adapter_path=adapter_path,
+            vllm_base_url=vllm_base_url,
+            vllm_api_key=vllm_api_key,
+            vllm_timeout=vllm_timeout,
+            vllm_verify_tls=vllm_verify_tls,
         )
         if input_path.is_dir():
             # Batch mode
             _run_batch(pipeline, input_path, output_path, fmt, task,
-                       grounding, max_tokens, profile, jobs)
+                       grounding, max_tokens, profile, jobs,
+                       request_concurrency)
         elif stream:
             # Streaming mode (single file, stdout only)
             _run_stream(pipeline, input_path, task, grounding, dpi, max_tokens, profile)
@@ -242,7 +286,8 @@ def _run_single(pipeline, input_path, output_path, fmt, task, grounding, dpi, ma
         click.echo(result)
 
 
-def _run_batch(pipeline, input_dir, output_path, fmt, task, grounding, max_tokens, profile, jobs=1):
+def _run_batch(pipeline, input_dir, output_path, fmt, task, grounding,
+               max_tokens, profile, jobs=1, request_concurrency=1):
     """Handle batch directory processing."""
     results = pipeline.run_batch(
         input_dir=input_dir,
@@ -255,6 +300,7 @@ def _run_batch(pipeline, input_dir, output_path, fmt, task, grounding, max_token
         profile=profile,
         raise_on_error=True,
         jobs=jobs,
+        inference_concurrency=request_concurrency,
     )
 
     if output_path:
@@ -305,6 +351,9 @@ def models():
     console.print(f"  [green]•[/] {DEFAULT_MODEL}")
     console.print("    MX FP8 quantized (~3.6 GB) — recommended for Apple Silicon")
     console.print()
+    console.print(f"  [green]•[/] {DEFAULT_VLLM_MODEL}")
+    console.print("    BF16 CUDA/vLLM baseline — use an AWQ repo with --model-path when available")
+    console.print()
     console.print("[dim]Use --model-path to specify a different model.[/]")
 
 
@@ -322,6 +371,94 @@ def profiles():
         console.print()
 
 
+@main.command("vllm-serve")
+@click.option("--model-path", type=str, default=DEFAULT_VLLM_MODEL,
+              show_default=True, help="Hugging Face repo ID or local CUDA checkpoint.")
+@click.option("--served-model-name", type=str, default=None,
+              help="Model name exposed by the OpenAI API (defaults to model path).")
+@click.option("--host", type=str, default="127.0.0.1", show_default=True,
+              help="Bind host; protect externally exposed endpoints with --api-key.")
+@click.option("--port", type=click.IntRange(1, 65535), default=8000,
+              show_default=True, help="Bind port.")
+@click.option("--tensor-parallel-size", type=click.IntRange(min=1), default=1,
+              show_default=True, help="Number of CUDA GPUs used for tensor parallelism.")
+@click.option("--gpu-memory-utilization", type=click.FloatRange(min=0.01, max=1.0),
+              default=0.90, show_default=True, help="Per-GPU memory utilization target.")
+@click.option("--max-model-len", type=click.IntRange(min=1), default=32768,
+              show_default=True, help="Maximum context length.")
+@click.option("--max-num-seqs", type=click.IntRange(min=1), default=4,
+              show_default=True, help="Maximum concurrent sequences.")
+@click.option("--max-images", type=click.IntRange(min=1), default=40,
+              show_default=True, help="Maximum images accepted by one request.")
+@click.option("--dtype", type=click.Choice(["auto", "bfloat16", "float16"]),
+              default="bfloat16", show_default=True, help="Model compute dtype.")
+@click.option("--revision", type=str, default=None,
+              help="Immutable Hugging Face commit revision (recommended for releases).")
+@click.option("--api-key", type=str, default=None, envvar="VLLM_API_KEY",
+              help="Protect the OpenAI API with a bearer token (or VLLM_API_KEY).")
+@click.option("--trust-remote-code/--no-trust-remote-code", default=False,
+              help="Allow model repository code (not needed for native Unlimited-OCR).")
+@click.option("--dry-run", is_flag=True, default=False,
+              help="Print the redacted vLLM command without starting it.")
+def vllm_serve(model_path, served_model_name, host, port,
+               tensor_parallel_size, gpu_memory_utilization, max_model_len,
+               max_num_seqs, max_images, dtype, revision, api_key,
+               trust_remote_code, dry_run):
+    """Start the native CUDA vLLM OpenAI-compatible server.
+
+    Requires Linux, NVIDIA CUDA, and vLLM 0.25.1+. The command installs the
+    Unlimited-OCR R-SWA logits processor and required cache settings.
+    """
+    import shlex
+
+    from unlimited_ocr.vllm_serve import (
+        VLLMServeConfig,
+        build_vllm_command,
+        redact_vllm_command,
+        run_vllm_server,
+    )
+
+    config = VLLMServeConfig(
+        model_path=model_path,
+        served_model_name=served_model_name,
+        host=host,
+        port=port,
+        tensor_parallel_size=tensor_parallel_size,
+        gpu_memory_utilization=gpu_memory_utilization,
+        max_model_len=max_model_len,
+        max_num_seqs=max_num_seqs,
+        max_images_per_prompt=max_images,
+        dtype=dtype,
+        revision=revision,
+        api_key=api_key,
+        trust_remote_code=trust_remote_code,
+    )
+    try:
+        command = build_vllm_command(config)
+    except (TypeError, ValueError) as exc:
+        raise click.UsageError(str(exc)) from exc
+
+    if dry_run:
+        rendered = shlex.join(redact_vllm_command(command))
+        if api_key:
+            rendered = f"VLLM_API_KEY=*** {rendered}"
+        click.echo(rendered)
+        return
+
+    if host not in {"127.0.0.1", "localhost", "::1"} and not api_key:
+        status_console.print(
+            "[yellow]Warning:[/] vLLM is exposed without an API key."
+        )
+    console.print(f"[bold]Starting Unlimited-OCR CUDA/vLLM[/] on {host}:{port}")
+    console.print(f"[dim]Model:[/] {model_path}")
+    try:
+        exit_code = run_vllm_server(config)
+    except Exception as exc:
+        raise click.ClickException(str(exc)) from exc
+    if exit_code != 0:
+        raise click.ClickException(f"vLLM exited with status {exit_code}")
+
+
 @main.command()
 @click.option("--host", type=str, default="127.0.0.1", show_default=True,
               help="Bind host. Use 0.0.0.0 only to expose the API on your network.")

src/unlimited_ocr/engine.py

@@ -2,9 +2,10 @@
 
 from __future__ import annotations
 
+import json
+import math
 import re
 import time
-import math
 from numbers import Real
 from dataclasses import dataclass, field
 from pathlib import Path
@@ -34,6 +35,90 @@ PROMPTS = {
 GROUNDING_PREFIX = "<|grounding|>"
 
 
+def _same_model_reference(recorded: str, requested: str) -> bool:
+    if recorded == requested:
+        return True
+    recorded_path = Path(recorded).expanduser()
+    requested_path = Path(requested).expanduser()
+    return (
+        recorded_path.exists()
+        and requested_path.exists()
+        and recorded_path.resolve() == requested_path.resolve()
+    )
+
+
+def _validate_adapter_base_model(adapter_path: str, model_path: str) -> None:
+    """Fail before MLX load when a local adapter records another base model."""
+    adapter_dir = Path(adapter_path).expanduser()
+    if not adapter_dir.is_dir():
+        return
+    config_path = adapter_dir / "training_config.json"
+    if not config_path.exists() and not config_path.is_symlink():
+        return
+    if not config_path.is_file():
+        raise ValueError(
+            f"Adapter training config must be a readable file: {config_path}"
+        )
+    try:
+        training_config = json.loads(config_path.read_text(encoding="utf-8"))
+    except (OSError, UnicodeError, json.JSONDecodeError) as exc:
+        raise ValueError(f"Adapter has invalid training config: {config_path}") from exc
+    if not isinstance(training_config, dict):
+        raise ValueError(f"Adapter training config must be an object: {config_path}")
+    recorded_base_model = training_config.get("base_model")
+    if (
+        not isinstance(recorded_base_model, str)
+        or not recorded_base_model
+        or recorded_base_model != recorded_base_model.strip()
+    ):
+        raise ValueError(
+            f"Adapter training config requires a normalized base_model: {config_path}"
+        )
+    if not _same_model_reference(recorded_base_model, model_path):
+        raise ValueError(
+            "Adapter base model mismatch: training config records "
+            f"{recorded_base_model!r}, but inference requested {model_path!r}"
+        )
+
+
+def build_ocr_prompt(
+    task: str = "document",
+    grounding: bool = False,
+    custom_prompt: Optional[str] = None,
+) -> str:
+    """Build a backend-independent Unlimited-OCR prompt.
+
+    Both the local MLX engine and the remote vLLM client must use the same
+    trained prompt vocabulary. Keeping prompt normalization here prevents the
+    two backends from silently producing different requests.
+    """
+    if custom_prompt is not None:
+        if not isinstance(custom_prompt, str):
+            raise TypeError("custom_prompt must be a string")
+        image_count = custom_prompt.count("<image>")
+        if image_count != 1:
+            raise ValueError(
+                "custom_prompt must contain exactly one literal '<image>' token"
+            )
+        prompt = custom_prompt
+    else:
+        if task not in PROMPTS:
+            available = ", ".join(PROMPTS)
+            raise ValueError(f"Unknown task '{task}'. Available: {available}")
+        prompt = PROMPTS[task]
+
+    wants_grounding = grounding or GROUNDING_PREFIX in prompt
+    if wants_grounding:
+        # Upstream DeepSeek/Unlimited-OCR places grounding after the image
+        # marker: ``<image>\n<|grounding|>task``.
+        prompt = prompt.replace(GROUNDING_PREFIX, "")
+        before, after = prompt.split("<image>", 1)
+        after = after.lstrip("\r\n")
+        prompt = f"{before}<image>\n{GROUNDING_PREFIX}{after}"
+
+    return prompt
+
+
 @dataclass
 class OCRResult:
     """Result from a single OCR inference."""
@@ -132,24 +217,30 @@ def parse_grounding(text: str) -> list[dict]:
                 "text": "",
             }))
 
-    # Current direct form. The first block may omit its opening det token.
+    # Current direct form. The first block may omit its opening det token, and
+    # a single det span may list several boxes for the same label.
     direct_pattern = re.compile(
         rf"(?:^|<\|det\|>)([^\[\n<]*?)\s*"
-        rf"({coordinate_pattern.pattern})\s*<\|/det\|>"
+        rf"((?:{coordinate_pattern.pattern}\s*)+)\s*<\|/det\|>"
         r"([^\n<]*)",
         re.MULTILINE,
     )
     for match in direct_pattern.finditer(normalized):
         if any(start <= match.start() < end for start, end in ref_spans):
             continue
-        coordinate_match = coordinate_pattern.search(match.group(2))
-        if coordinate_match is None:
+        boxes = list(coordinate_pattern.finditer(match.group(2)))
+        if not boxes:
             continue
-        parsed.append((match.start(), {
-            "label": match.group(1).strip(),
-            "bbox": coordinate_from_match(coordinate_match),
-            "text": match.group(7).strip(),
-        }))
+        label = match.group(1).strip()
+        # group(2) is the coordinate run (nested number groups inside it);
+        # the trailing content capture is the last group.
+        text = match.group(match.lastindex).strip() if match.lastindex else ""
+        for coordinate_match in boxes:
+            parsed.append((match.start() + coordinate_match.start(), {
+                "label": label,
+                "bbox": coordinate_from_match(coordinate_match),
+                "text": text,
+            }))
 
     parsed.sort(key=lambda item: item[0])
     return [item for _, item in parsed]
@@ -206,6 +297,9 @@ class OCREngine:
         self._model = None
         self._processor = None
 
+    backend_name = "mlx"
+    supports_multi_image = False
+
     @property
     def model(self):
         """Lazy-load the MLX model."""
@@ -222,8 +316,6 @@ class OCREngine:
 
     def _load_model(self):
         """Load model and processor from mlx-vlm."""
-        from mlx_vlm import load
-
         if self.config.verbose:
             console.print(f"[bold blue]Loading model:[/] {self.config.model_path}")
 
@@ -231,9 +323,15 @@ class OCREngine:
         if self.config.adapter_path is not None:
             if not isinstance(self.config.adapter_path, str) or not self.config.adapter_path:
                 raise ValueError("adapter_path must be a non-empty string or None")
+            _validate_adapter_base_model(
+                self.config.adapter_path,
+                self.config.model_path,
+            )
             # mlx-vlm applies adapters to the nested language model correctly;
             # mlx-lm's helper alone can target the wrong module tree for VLMs.
             load_kwargs["adapter_path"] = self.config.adapter_path
+        from mlx_vlm import load
+
         self._model, self._processor = load(self.config.model_path, **load_kwargs)
 
         if self.config.verbose:
@@ -252,32 +350,11 @@ class OCREngine:
             grounding: If True, prefix with grounding token for bounding boxes.
             custom_prompt: Override with a fully custom prompt (must contain <image>).
         """
-        if custom_prompt is not None:
-            if not isinstance(custom_prompt, str):
-                raise TypeError("custom_prompt must be a string")
-            image_count = custom_prompt.count("<image>")
-            if image_count != 1:
-                raise ValueError(
-                    "custom_prompt must contain exactly one literal '<image>' token"
-                )
-            prompt = custom_prompt
-        else:
-            if task not in PROMPTS:
-                available = ", ".join(PROMPTS)
-                raise ValueError(f"Unknown task '{task}'. Available: {available}")
-            prompt = PROMPTS[task]
-
-        wants_grounding = grounding or GROUNDING_PREFIX in prompt
-        if wants_grounding:
-            # Upstream DeepSeek/Unlimited-OCR places grounding after the image
-            # marker: ``<image>\n<|grounding|>task``. Normalize misplaced or
-            # duplicate copies in custom prompts to that trained format.
-            prompt = prompt.replace(GROUNDING_PREFIX, "")
-            before, after = prompt.split("<image>", 1)
-            after = after.lstrip("\r\n")
-            prompt = f"{before}<image>\n{GROUNDING_PREFIX}{after}"
-
-        return prompt
+        return build_ocr_prompt(
+            task=task,
+            grounding=grounding,
+            custom_prompt=custom_prompt,
+        )
 
     def _validate_generation_settings(
         self,
@@ -560,6 +637,11 @@ class OCREngine:
         """Check if the model is currently loaded."""
         return self._model is not None
 
+    def ensure_ready(self) -> None:
+        """Load the local model and processor before accepting work."""
+        _ = self.model
+        _ = self.processor
+
     def unload(self):
         """Release model from memory."""
         self._model = None

src/unlimited_ocr/pdf.py

@@ -48,47 +48,53 @@ def _pixmap_to_pil(pixmap: fitz.Pixmap) -> Image.Image:
 
     Pixmaps may be CMYK/gray or carry premultiplied alpha. Reading samples as
     raw RGB packing (or dropping alpha without compositing) produces near-black
-    garbage. Prefer PyMuPDF's own conversion, then composite onto white so
-    document pages keep a paper background.
+    garbage. PyMuPDF stores alpha-bearing samples premultiplied, so compositing
+    must use ``premul + 255 * (1 - a)`` rather than treating channels as straight
+    alpha (PIL's ``paste(..., mask=)`` path). Composite onto white so document
+    pages keep a paper background.
     """
-    if hasattr(pixmap, "pil_image"):
-        image = pixmap.pil_image()
-    else:
-        # Older PyMuPDF fallback: force DeviceRGB and copy sample bytes.
+    if pixmap.alpha:
+        # Premultiplied alpha → opaque RGB on white. Do not use pil_image()+paste:
+        # that path assumes straight alpha and darkens translucent ink.
         if pixmap.colorspace is None or pixmap.n - pixmap.alpha != 3:
             pixmap = fitz.Pixmap(fitz.csRGB, pixmap)
-        if pixmap.alpha:
-            # Premultiplied alpha → composite onto white, then drop A.
-            import array
-
-            samples = array.array("B", pixmap.samples)
-            width, height = pixmap.width, pixmap.height
-            stride = pixmap.n
-            rgb = bytearray(width * height * 3)
-            for index in range(width * height):
-                base = index * stride
-                alpha = samples[base + 3] / 255.0
-                for channel in range(3):
-                    value = samples[base + channel] + 255.0 * (1.0 - alpha)
-                    rgb[index * 3 + channel] = 0 if value < 0 else (
-                        255 if value > 255 else int(value)
-                    )
-            image = Image.frombytes("RGB", (width, height), bytes(rgb))
-        else:
-            image = Image.frombytes(
-                "RGB",
-                (pixmap.width, pixmap.height),
-                bytes(pixmap.samples),
-            )
-
-    if image.mode == "RGB":
-        # Copy so the image outlives the temporary pixmap.
-        return image.copy()
-    if image.mode == "RGBA":
-        background = Image.new("RGB", image.size, (255, 255, 255))
-        background.paste(image, mask=image.getchannel("A"))
-        return background
-    return image.convert("RGB")
+        import array
+
+        samples = array.array("B", pixmap.samples)
+        width, height = pixmap.width, pixmap.height
+        stride = pixmap.n
+        alpha_index = stride - 1
+        rgb = bytearray(width * height * 3)
+        for index in range(width * height):
+            base = index * stride
+            alpha = samples[base + alpha_index] / 255.0
+            inv = 1.0 - alpha
+            for channel in range(3):
+                value = samples[base + channel] + 255.0 * inv
+                rgb[index * 3 + channel] = 0 if value < 0 else (
+                    255 if value > 255 else int(value)
+                )
+        return Image.frombytes("RGB", (width, height), bytes(rgb))
+
+    # Opaque path: prefer pil_image so CMYK/gray conversion stays with PyMuPDF.
+    if hasattr(pixmap, "pil_image"):
+        image = pixmap.pil_image()
+        if image.mode == "RGB":
+            # Copy so the image outlives the temporary pixmap.
+            return image.copy()
+        if image.mode == "RGBA":
+            background = Image.new("RGB", image.size, (255, 255, 255))
+            background.paste(image, mask=image.getchannel("A"))
+            return background
+        return image.convert("RGB")
+
+    if pixmap.colorspace is None or pixmap.n != 3:
+        pixmap = fitz.Pixmap(fitz.csRGB, pixmap)
+    return Image.frombytes(
+        "RGB",
+        (pixmap.width, pixmap.height),
+        bytes(pixmap.samples),
+    )
 
 
 def iter_pdf_pages(
@@ -185,9 +191,10 @@ def pdf_to_images(
             for i, page_idx in enumerate(range(start, end + 1)):
                 page = doc[page_idx]
                 pixmap = page.get_pixmap(matrix=mat, alpha=False)
-
+                # Route through the same RGB/alpha normalizer as the in-memory
+                # path so unusual page colorspaces cannot fail PNG encoding.
                 out_path = output_dir / f"page_{page_idx + 1:04d}.png"
-                pixmap.save(str(out_path))
+                _pixmap_to_pil(pixmap).save(str(out_path))
                 image_paths.append(out_path)
 
                 if progress_callback:

src/unlimited_ocr/pipeline.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import tempfile
 from collections import Counter
-from concurrent.futures import ThreadPoolExecutor
+from concurrent.futures import ThreadPoolExecutor, as_completed
 from dataclasses import replace
 from pathlib import Path
 from threading import Lock
@@ -19,6 +19,12 @@ from unlimited_ocr.output import DocumentResult, format_result, save_result
 from unlimited_ocr.pdf import DEFAULT_PDF_DPI, get_pdf_info, iter_pdf_pages, is_pdf
 from unlimited_ocr.preprocessing import PreprocessConfig, preprocess_to_pil, config_for_mode
 from unlimited_ocr.profiles import get_profile
+from unlimited_ocr.vllm_backend import (
+    DEFAULT_VLLM_BASE_URL,
+    DEFAULT_VLLM_MODEL,
+    VLLMEngine,
+    VLLMEngineConfig,
+)
 
 console = Console(stderr=True)
 
@@ -52,7 +58,7 @@ class OCRPipeline:
 
     def __init__(
         self,
-        model_path: str = DEFAULT_MODEL,
+        model_path: Optional[str] = None,
         max_tokens: Optional[int] = None,
         temperature: Optional[float] = None,
         verbose: bool = False,
@@ -61,11 +67,18 @@ class OCRPipeline:
         preprocess_config: Optional[PreprocessConfig] = None,
         profile: str = "accurate",
         adapter_path: Optional[str] = None,
+        backend: str = "mlx",
+        vllm_base_url: str = DEFAULT_VLLM_BASE_URL,
+        vllm_api_key: Optional[str] = None,
+        vllm_timeout: float = 3600.0,
+        vllm_verify_tls: bool = True,
+        vllm_max_images: int = 40,
     ):
         """Initialize the OCR pipeline.
 
         Args:
-            model_path: HuggingFace model ID or local path.
+            model_path: HuggingFace model ID or local path. Defaults to the
+                AutomatosX MLX model for ``mlx`` and upstream BF16 for ``vllm``.
             max_tokens: Optional per-page override for the selected profile.
             temperature: Optional sampling-temperature override.
             verbose: Enable verbose output.
@@ -74,18 +87,51 @@ class OCRPipeline:
             preprocess_config: Custom preprocessing configuration (overrides mode).
             profile: Generation profile (accurate/fast/long-document/plain-text/markdown).
             adapter_path: Optional LoRA adapter path for scene-specific inference.
+            backend: Inference backend (``mlx`` or ``vllm``).
+            vllm_base_url: OpenAI-compatible vLLM API root.
+            vllm_api_key: Optional vLLM bearer token.
+            vllm_timeout: Remote inference timeout in seconds.
+            vllm_verify_tls: Verify HTTPS certificates for vLLM requests.
+            vllm_max_images: Maximum images in one vLLM request.
         """
         # Initialize first so cleanup remains safe if later validation raises.
         self._tmp_dir: Optional[tempfile.TemporaryDirectory] = None
         self._tmp_lock = Lock()
-        self.engine = OCREngine(EngineConfig(
-            model_path=model_path,
-            max_tokens=max_tokens,
-            temperature=temperature,
-            verbose=verbose,
-            profile=profile,
-            adapter_path=adapter_path,
-        ))
+        if not isinstance(backend, str):
+            raise TypeError("backend must be a string")
+        backend_name = backend.strip().lower()
+        if backend_name not in {"mlx", "vllm"}:
+            raise ValueError("backend must be one of: mlx, vllm")
+        self.backend = backend_name
+        if backend_name == "mlx":
+            resolved_model = model_path or DEFAULT_MODEL
+            self.engine = OCREngine(EngineConfig(
+                model_path=resolved_model,
+                max_tokens=max_tokens,
+                temperature=temperature,
+                verbose=verbose,
+                profile=profile,
+                adapter_path=adapter_path,
+            ))
+        else:
+            if adapter_path is not None:
+                raise ValueError(
+                    "adapter_path is not supported by the vLLM client; "
+                    "configure LoRA modules when starting vLLM"
+                )
+            resolved_model = model_path or DEFAULT_VLLM_MODEL
+            self.engine = VLLMEngine(VLLMEngineConfig(
+                model_path=resolved_model,
+                base_url=vllm_base_url,
+                api_key=vllm_api_key,
+                timeout_seconds=vllm_timeout,
+                verify_tls=vllm_verify_tls,
+                max_tokens=max_tokens,
+                temperature=temperature,
+                verbose=verbose,
+                profile=profile,
+                max_images_per_request=vllm_max_images,
+            ))
         self.preprocess_enabled = preprocess
         self.preprocess_mode = preprocess_mode
         self.preprocess_config = preprocess_config or config_for_mode(preprocess_mode)
@@ -269,6 +315,7 @@ class OCRPipeline:
         profile: Optional[str] = None,
         raise_on_error: bool = False,
         jobs: int = 1,
+        inference_concurrency: int = 1,
         memory_reset_interval: int = 50,
     ) -> dict[str, str]:
         """Run OCR on all images in a directory.
@@ -286,6 +333,8 @@ class OCRPipeline:
             raise_on_error: Raise BatchProcessingError after processing if any
                 files failed. Successful results remain attached to the error.
             jobs: Number of parallel workers for image preprocessing (1=sequential).
+            inference_concurrency: Concurrent remote inference requests. Values
+                above 1 are supported only by the vLLM backend.
             memory_reset_interval: Reset the reported Metal peak-memory
                 counter every N successful images (0=never).
     
@@ -298,6 +347,16 @@ class OCRPipeline:
         self._validate_run_options(task, format, profile, max_tokens)
         if not isinstance(jobs, int) or isinstance(jobs, bool) or jobs < 1:
             raise ValueError("jobs must be a positive integer")
+        if (
+            not isinstance(inference_concurrency, int)
+            or isinstance(inference_concurrency, bool)
+            or inference_concurrency < 1
+        ):
+            raise ValueError("inference_concurrency must be a positive integer")
+        if inference_concurrency > 1 and self.backend != "vllm":
+            raise ValueError(
+                "inference_concurrency > 1 is supported only by the vLLM backend"
+            )
         if (
             not isinstance(memory_reset_interval, int)
             or isinstance(memory_reset_interval, bool)
@@ -356,6 +415,7 @@ class OCRPipeline:
                         preparation_errors[img_file.name] = exc
     
         processed_count = 0
+        processed_count_lock = Lock()
     
         def _process_one(img_file: Path) -> tuple[str, str]:
             """Process a single image, returns (filename, output_text)."""
@@ -387,14 +447,47 @@ class OCRPipeline:
                 out_path = output_path_for(img_file)
                 save_result(doc_result, out_path, format)
     
-            processed_count += 1
+            with processed_count_lock:
+                processed_count += 1
+                reset_memory = (
+                    memory_reset_interval > 0
+                    and processed_count % memory_reset_interval == 0
+                )
             # Start a fresh peak-memory reporting window. This does not free
             # model memory; MLX manages allocations itself.
-            if memory_reset_interval > 0 and processed_count % memory_reset_interval == 0:
+            if reset_memory:
                 self._reset_memory_stats()
     
             return img_file.name, output
     
+        def _run_all(advance: Callable[[], None]) -> None:
+            if inference_concurrency > 1:
+                # Resolve readiness once before worker threads issue requests.
+                self.engine.ensure_ready()
+                with ThreadPoolExecutor(
+                    max_workers=inference_concurrency
+                ) as executor:
+                    futures = {
+                        executor.submit(_process_one, img_file): img_file
+                        for img_file in image_files
+                    }
+                    for future in as_completed(futures):
+                        img_file = futures[future]
+                        try:
+                            name, output = future.result()
+                            results[name] = output
+                        except Exception as exc:
+                            errors.append((img_file.name, str(exc)))
+                        advance()
+                return
+            for img_file in image_files:
+                try:
+                    name, output = _process_one(img_file)
+                    results[name] = output
+                except Exception as exc:
+                    errors.append((img_file.name, str(exc)))
+                advance()
+
         if show_progress:
             with Progress(
                 SpinnerColumn(),
@@ -404,20 +497,9 @@ class OCRPipeline:
                 console=console,
             ) as progress:
                 task_id = progress.add_task("Processing images...", total=len(image_files))
-                for img_file in image_files:
-                    try:
-                        name, output = _process_one(img_file)
-                        results[name] = output
-                    except Exception as e:
-                        errors.append((img_file.name, str(e)))
-                    progress.advance(task_id)
+                _run_all(lambda: progress.advance(task_id))
         else:
-            for img_file in image_files:
-                try:
-                    name, output = _process_one(img_file)
-                    results[name] = output
-                except Exception as e:
-                    errors.append((img_file.name, str(e)))
+            _run_all(lambda: None)
     
         # Report errors
         if errors:
@@ -427,10 +509,18 @@ class OCRPipeline:
             if raise_on_error:
                 raise BatchProcessingError(errors, results)
     
-        return results
+        # Preserve deterministic input order even when remote requests finish
+        # out of order.
+        return {
+            image_file.name: results[image_file.name]
+            for image_file in image_files
+            if image_file.name in results
+        }
     
     def _reset_memory_stats(self):
         """Reset MLX's peak-memory statistic for subsequent measurements."""
+        if getattr(self.engine, "backend_name", "mlx") != "mlx":
+            return
         # Avoid importing/initializing MLX merely to reset a statistic when no
         # model has been loaded (important on non-Metal and headless hosts).
         if not self.engine.is_loaded():
@@ -529,7 +619,7 @@ class OCRPipeline:
         )
 
     def cleanup(self):
-        """Clean up temporary files."""
+        """Clean up temporary files and release the selected backend."""
         lock = getattr(self, "_tmp_lock", None)
         if lock is None:
             return
@@ -537,6 +627,10 @@ class OCRPipeline:
             if getattr(self, "_tmp_dir", None) is not None:
                 self._tmp_dir.cleanup()
                 self._tmp_dir = None
+        engine = getattr(self, "engine", None)
+        unload = getattr(engine, "unload", None)
+        if callable(unload):
+            unload()
 
     def _validate_run_options(
         self,

src/unlimited_ocr/preprocessing.py

@@ -544,6 +544,11 @@ def apply_gamma(img: np.ndarray, gamma: float = 1.5) -> np.ndarray:
         raise ValueError("gamma must be a positive finite number")
     # Quantize the cache key so near-identical values share a LUT.
     cache_key = round(float(gamma), 4)
+    # Very small positive values round to zero at the cache precision. Keep
+    # the exact value in that case so valid input cannot become a division by
+    # zero merely because of the LUT cache key.
+    if cache_key == 0:
+        cache_key = float(gamma)
     table = _GAMMA_LUT_CACHE.get(cache_key)
     if table is None:
         inv_gamma = 1.0 / cache_key
@@ -680,6 +685,11 @@ def preprocess_image(
         img, rot_angle = auto_rotate(img)
         if rot_angle:
             config.applied_steps.append(f"rotate_{rot_angle}")
+        if deskew:
+            deskewed = deskew_image(img)
+            if deskewed is not img:
+                config.applied_steps.append("deskew")
+            img = deskewed
         img = sauvola_binarize(img)
         config.applied_steps.append("sauvola_binarize")
         if config.target_size:

tokenizer_config.json

@@ -13,7 +13,7 @@
   "local_files_only": false,
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "<｜▁pad▁｜>",
-  "processor_class": "UnlimitedOCRProcessor",
+  "processor_class": "UnlimitedOCRHFProcessor",
   "tokenizer_class": "TokenizersBackend",
   "unk_token": null,
   "use_default_system_prompt": false