Upload int8 MLX quantization

README.md

@@ -79,12 +79,13 @@ regions in reading order. Each variant ran the model's own documented OCR prompt
 truth with two metrics: character-level similarity (`difflib.SequenceMatcher`) and word
 recall (fraction of ground-truth words present in the output).
 
-| Variant | Mean similarity | Mean word recall | Degenerate outputs | Agreement with FP16 |
-|---|---|---|---|---|
-| fp16 | 63.4% | 92.9% | 0/15 | — |
-| mxfp4 | 63.2% | 93.1% | 0/15 | 96.5% |
-| mxfp8 | 63.0% | 92.9% | 0/15 | 97.7% |
-| int8 | 63.9% | 93.1% | 0/15 | 99.2% |
+| Variant | Bits/weight | Disk size | Mean similarity | Mean word recall | Degenerate outputs | Agreement with FP16 | Eval time (15 pages) |
+|---|---|---|---|---|---|---|---|
+| fp16 | 16 | 1.6 GB | 63.4% | 92.9% | 0/15 | — | 212.4s |
+| mxfp4 | 5.643 | 599 MB | 63.2% | 93.1% | 0/15 | 96.5% | 138.1s |
+| int4 | 5.863 | 622 MB | 63.5% | 93.2% | 0/15 | 96.1% | 140.7s |
+| mxfp8 | 9.168 | 958 MB | 63.0% | 92.9% | 0/15 | 97.7% | 166.8s |
+| int8 | 9.389 | 980 MB | 63.9% | 93.1% | 0/15 | 99.2% | 165.1s |
 
 - **Word recall (~93%) is the more reliable signal here** — character-level similarity is
   pulled down by cosmetic LaTeX-formatting differences (e.g. the model emits `$...$` compact
@@ -93,16 +94,20 @@ recall (fraction of ground-truth words present in the output).
   equation block, sim≈3%) confirmed the model's output was in fact a faithful, correctly
   ordered transcription — the metric penalizes LaTeX style, not content.
 - **Zero degenerate outputs** (no repetition loops, no empty/near-empty generations) across
-  all 60 generations (15 pages × 4 variants, including the fp16 baseline).
-- All three quantized variants agree with FP16's output at the character level 96–99% of the
-  time — quantization barely perturbs the transcription, even at 4-bit.
-- mxfp4 is also the fastest of the three on this 15-page batch (138s vs 213s for fp16),
-  consistent with its smaller memory footprint.
+  all 75 generations (15 pages × 5 variants, including the fp16 baseline).
+- **All four quantized variants agree with FP16's output at the character level 96–99% of the
+  time** — quantization barely perturbs the transcription, even at 4-bit (both mxfp4 and int4).
+- **4-bit vs 8-bit is a wash on quality here**: mxfp4/int4 score essentially the same word
+  recall as mxfp8/int8 on this OCR task, while being ~35% faster and roughly 40% smaller on
+  disk — for OvisOCR2 specifically, the 4-bit variants are the better default.
+- mxfp4 and int4 are nearly identical to each other in every metric; pick whichever fits your
+  serving stack (MX FP4 vs plain affine int4) — there's no accuracy reason to prefer one.
 
 
 ## Other MLX variants
 
 - [ovisocr2-mxfp4-mlx](https://huggingface.co/sahilchachra/ovisocr2-mxfp4-mlx)
+- [ovisocr2-int4-mlx](https://huggingface.co/sahilchachra/ovisocr2-int4-mlx)
 - [ovisocr2-mxfp8-mlx](https://huggingface.co/sahilchachra/ovisocr2-mxfp8-mlx)
 - [ovisocr2-int8-mlx](https://huggingface.co/sahilchachra/ovisocr2-int8-mlx)