---


license: mit
base_model: baidu/Unlimited-OCR
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: vllm
tags:
  - ocr
  - vllm
  - cuda
  - awq
  - compressed-tensors
  - preview
  - jetson-thor
  - ax-engine
  - automatosx
---

# AX-Unlimited-OCR-3B-MoE-CUDA-AWQ-W4A16

> **CUDA PREVIEW — not a full multi-architecture GA release.**
> NVIDIA Thor `aarch64` passed formal native and immutable OCI acceptance.
> `x86_64` remains experimental until the exact Production A100 OCI gate passes.

This is one shared AWQ W4A16 checkpoint intended for CUDA/vLLM. The same model
files may be downloaded on Thor and x86 Linux, but this preview makes a formal
support claim only for NVIDIA Thor.

## Architecture status

| Target | Status | Release claim |
|---|---|---|
| NVIDIA Thor (`aarch64`) | Formally validated | Supported by this preview |
| A100-SXM4-80GB (`x86_64`, SM80) | Experimental | Native evidence exists; formal Production OCI pending |
| RTX A6000 (`x86_64`, SM86) | Experimental | No formal compatibility or performance claim |

The available x86 development hosts currently report Thunder
`mode=prototyping`, Docker `fastvfs`, and `/etc/thunder/libthunder.so`
injection. Increasing their disks to 200 GB solved storage capacity but did not
turn them into standard Production NVIDIA runtimes. This is an evidence-host
limitation, not a claim that the checkpoint itself failed on x86.

`native_x86_64_acceptance` remains recorded as `false`; it was not removed or
changed to pass. The existing full CUDA publisher still requires both
architectures and all 31 GA gates. No immutable SemVer GA tag is created for
this preview.

## Thor evidence

- Formal OCI child: `127.0.0.1:5000/axocr-cuda@sha256:73257ded4e5a0b763ce7c117dca846c6796b786b8933c236ab479e4f58845575`
- Image config: `sha256:d6418adfcb11c371a13595d5df1af3d76f78542fc5f4050ac2534ae8285a0565`
- OCI receipt: `7ef7bb054547ad3ec479ace48a2ec22d18ded21fd6738017fadee055d461f6d4`
- Frozen runtime source: `9b71932f621ee2939b40507a8ab942559851e757c4e54fe59528b498072fce10`
- Upstream source: `baidu/Unlimited-OCR@ee63731b6461c8afcdcc7b15352e7d2ffecc2ead`
- Evaluation set: `ax-ocr-accuracy-v3-20260723` (208 samples)

| Thor metric | BF16 | AWQ |
|---|---:|---:|
| Aggregate throughput (completion tok/s) | 30.9081 | 31.0139 |
| p95 latency (s) | 23.2570 | 23.0460 |
| Mean CER | 0.089845 | 0.088245 |
| Digit CER | 0.076077 | 0.076705 |

The exact public-safe evidence summary is in `preview_manifest.json`.

## Serving

Use AX-OCR 0.5.0 source matching the frozen runtime identity and public vLLM
0.25.1 with CUDA 13.0. On Thor the AX-OCR vLLM plugin selects the validated
eager compatibility path.

```bash
ax-ocr vllm-serve \
  --model-path AutomatosX/AX-Unlimited-OCR-3B-MoE-CUDA-AWQ-W4A16 \
  --host 127.0.0.1 \
  --port 8000
```

Requests must preserve the literal `<image>` prompt, set
`skip_special_tokens=false`, and use the Unlimited-OCR no-repeat/R-SWA request
settings recorded by AX-OCR.

## Limitations

- This preview is not a Production support statement for x86 CUDA.
- A6000 and other NVIDIA GPUs may require different kernels or fallbacks.
- The model is not MLX or GGUF.
- OCR can hallucinate or omit content; verify high-stakes documents.