Image-Text-to-Text
MLX
Safetensors
unlimited-ocr
ax-engine
mlx-vlm
ocr
mxfp8
int8
apple-silicon
automatosx
conversational
8-bit precision
Instructions to use AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8") config = load_config("AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Model card: AX Engine primary, mlx-vlm alternative only
Browse files
README.md
CHANGED
|
@@ -28,8 +28,11 @@ checkpoint was used only as a comparison reference; these are not repackaged
|
|
| 28 |
Sahil weights.
|
| 29 |
|
| 30 |
**Primary inference engine:** [AX Engine](https://github.com/defai-digital/ax-engine)
|
| 31 |
-
(`model_family=unlimited_ocr`)
|
| 32 |
-
MXFP8 / dual-vision loads.
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
**Hybrid precision (name is shorthand):** MXFP8 (group size 32) on eligible
|
| 35 |
language linears, **affine INT8** on the LM head, **BF16** on the vision
|
|
@@ -42,7 +45,7 @@ active MoE capacity; the marketing name refers to the upstream 3B MoE family.
|
|
| 42 |
- **Hub pin:** `revision="v0.2.0"` or the commit SHA after publish
|
| 43 |
- **Repo:** [`AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8`](https://huggingface.co/AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8)
|
| 44 |
- **Runtime:** [AX Engine](https://github.com/defai-digital/ax-engine)
|
| 45 |
-
- **Release notes:** Sensitivity-guided MXFP8 with calibrated affine INT8 LM head; primary inference via AX Engine
|
| 46 |
|
| 47 |
Immutable upstream source revision: `ee63731b6461c8afcdcc7b15352e7d2ffecc2ead`.
|
| 48 |
Reference checkpoint revision: `55b8031a6c867de675279d9604e38cc94b9882a4`.
|
|
@@ -87,11 +90,9 @@ CJK, and 3 with tables), digest
|
|
| 87 |
- Small local set; some samples are synthetic. Do not treat mean CER as field accuracy.
|
| 88 |
- The community reference may use a different `model_type` / load path; it is a
|
| 89 |
throughput and packaging baseline, not an identical-recipe peer.
|
| 90 |
-
- AX Engine
|
| 91 |
-
|
| 92 |
-
revision when comparing stacks.
|
| 93 |
-
- Generation profiles in AX-OCR live in `profiles.py`; raw mlx-vlm loads use
|
| 94 |
-
library defaults unless you pass matching decode settings.
|
| 95 |
|
| 96 |
### Weight integrity
|
| 97 |
|
|
@@ -133,14 +134,18 @@ Full per-sample evidence, sensitivity results, precision rules, PRD, ADR, and
|
|
| 133 |
technical specification are included under `release/`, `quantization/`, and
|
| 134 |
`docs/`.
|
| 135 |
|
| 136 |
-
## Usage
|
|
|
|
|
|
|
| 137 |
|
| 138 |
-
|
| 139 |
-
[AX Engine](https://github.com/defai-digital/ax-engine) (native `unlimited_ocr`).
|
| 140 |
|
| 141 |
-
|
|
|
|
| 142 |
|
| 143 |
-
|
|
|
|
|
|
|
| 144 |
|
| 145 |
```bash
|
| 146 |
# Homebrew (recommended on macOS)
|
|
@@ -152,7 +157,7 @@ ax-engine doctor
|
|
| 152 |
python3 -m pip install --upgrade "ax-engine[download]>=6.11.0,<7"
|
| 153 |
```
|
| 154 |
|
| 155 |
-
### Download and prepare
|
| 156 |
|
| 157 |
```bash
|
| 158 |
# Managed download (writes model-manifest.json when needed)
|
|
@@ -170,7 +175,7 @@ ax-engine-bench generate-manifest ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --validate
|
|
| 170 |
This repository ships `model-manifest.json` so downloads are AX-ready without a
|
| 171 |
manual convert step when the file is present.
|
| 172 |
|
| 173 |
-
### Serve
|
| 174 |
|
| 175 |
```bash
|
| 176 |
ax-engine serve ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --port 31418
|
|
@@ -180,12 +185,12 @@ ax-engine serve ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --port 31418
|
|
| 180 |
|
| 181 |
OpenAI-compatible routes are exposed on the local server. See the
|
| 182 |
[AX Engine docs](https://github.com/defai-digital/ax-engine/blob/main/docs/GETTING-STARTED.md) for
|
| 183 |
-
install, doctor checks, and API details.
|
| 184 |
-
native dual-vision prefill in AX Engine (`model_family=unlimited_ocr`).
|
| 185 |
|
| 186 |
-
##
|
| 187 |
|
| 188 |
-
|
|
|
|
| 189 |
|
| 190 |
```bash
|
| 191 |
pip install 'mlx-vlm>=0.6.4'
|
|
@@ -197,7 +202,10 @@ python -m mlx_vlm.generate \
|
|
| 197 |
--max-tokens 4096
|
| 198 |
```
|
| 199 |
|
| 200 |
-
### AX-OCR
|
|
|
|
|
|
|
|
|
|
| 201 |
|
| 202 |
```bash
|
| 203 |
pip install 'ax-ocr[mlx]'
|
|
|
|
| 28 |
Sahil weights.
|
| 29 |
|
| 30 |
**Primary inference engine:** [AX Engine](https://github.com/defai-digital/ax-engine)
|
| 31 |
+
(`model_family=unlimited_ocr`) — use this for Apple Silicon serving and native
|
| 32 |
+
MXFP8 / dual-vision loads.
|
| 33 |
+
|
| 34 |
+
**Alternative runtime:** `mlx-vlm` (weight-compatible; not the recommended
|
| 35 |
+
production path for AutomatosX packs).
|
| 36 |
|
| 37 |
**Hybrid precision (name is shorthand):** MXFP8 (group size 32) on eligible
|
| 38 |
language linears, **affine INT8** on the LM head, **BF16** on the vision
|
|
|
|
| 45 |
- **Hub pin:** `revision="v0.2.0"` or the commit SHA after publish
|
| 46 |
- **Repo:** [`AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8`](https://huggingface.co/AutomatosX/AX-Unlimited-OCR-3B-MoE-MLX-MXFP8)
|
| 47 |
- **Runtime:** [AX Engine](https://github.com/defai-digital/ax-engine)
|
| 48 |
+
- **Release notes:** Sensitivity-guided MXFP8 with calibrated affine INT8 LM head; primary inference via AX Engine; mlx-vlm alternative only
|
| 49 |
|
| 50 |
Immutable upstream source revision: `ee63731b6461c8afcdcc7b15352e7d2ffecc2ead`.
|
| 51 |
Reference checkpoint revision: `55b8031a6c867de675279d9604e38cc94b9882a4`.
|
|
|
|
| 90 |
- Small local set; some samples are synthetic. Do not treat mean CER as field accuracy.
|
| 91 |
- The community reference may use a different `model_type` / load path; it is a
|
| 92 |
throughput and packaging baseline, not an identical-recipe peer.
|
| 93 |
+
- Prefer **AX Engine** for inference. `mlx-vlm` is an alternative load path only.
|
| 94 |
+
- Document helpers (e.g. AX-OCR PDF/profiles) may still call mlx-vlm under the
|
| 95 |
+
hood; pin the same Hub revision when comparing stacks.
|
|
|
|
|
|
|
| 96 |
|
| 97 |
### Weight integrity
|
| 98 |
|
|
|
|
| 134 |
technical specification are included under `release/`, `quantization/`, and
|
| 135 |
`docs/`.
|
| 136 |
|
| 137 |
+
## Usage
|
| 138 |
+
|
| 139 |
+
Requires **Apple Silicon**.
|
| 140 |
|
| 141 |
+
### Primary — AX Engine
|
|
|
|
| 142 |
|
| 143 |
+
**Use [AX Engine](https://github.com/defai-digital/ax-engine) as the primary inference engine** for this
|
| 144 |
+
pack (native `unlimited_ocr`: dual vision + SWA MoE + MXFP8 + R-SWA).
|
| 145 |
|
| 146 |
+
Quantization/release measurements for these weights used: mlx-vlm `0.6.6`, mlx `0.32.0`, python `3.14.6`.
|
| 147 |
+
|
| 148 |
+
#### Install
|
| 149 |
|
| 150 |
```bash
|
| 151 |
# Homebrew (recommended on macOS)
|
|
|
|
| 157 |
python3 -m pip install --upgrade "ax-engine[download]>=6.11.0,<7"
|
| 158 |
```
|
| 159 |
|
| 160 |
+
#### Download and prepare
|
| 161 |
|
| 162 |
```bash
|
| 163 |
# Managed download (writes model-manifest.json when needed)
|
|
|
|
| 175 |
This repository ships `model-manifest.json` so downloads are AX-ready without a
|
| 176 |
manual convert step when the file is present.
|
| 177 |
|
| 178 |
+
#### Serve
|
| 179 |
|
| 180 |
```bash
|
| 181 |
ax-engine serve ./AX-Unlimited-OCR-3B-MoE-MLX-MXFP8 --port 31418
|
|
|
|
| 185 |
|
| 186 |
OpenAI-compatible routes are exposed on the local server. See the
|
| 187 |
[AX Engine docs](https://github.com/defai-digital/ax-engine/blob/main/docs/GETTING-STARTED.md) for
|
| 188 |
+
install, doctor checks, and API details.
|
|
|
|
| 189 |
|
| 190 |
+
### Alternative — mlx-vlm
|
| 191 |
|
| 192 |
+
`mlx-vlm` can load these weights as a **secondary / alternative** path. Prefer
|
| 193 |
+
AX Engine for AutomatosX deployments.
|
| 194 |
|
| 195 |
```bash
|
| 196 |
pip install 'mlx-vlm>=0.6.4'
|
|
|
|
| 202 |
--max-tokens 4096
|
| 203 |
```
|
| 204 |
|
| 205 |
+
### Optional — AX-OCR document toolkit
|
| 206 |
+
|
| 207 |
+
AX-OCR adds deskew, PDF, and generation profiles on top of the same Hub weights.
|
| 208 |
+
It is a toolkit, not a replacement for AX Engine as the primary runtime.
|
| 209 |
|
| 210 |
```bash
|
| 211 |
pip install 'ax-ocr[mlx]'
|