DeepEncoder ONNX fp32 (parity 1.0000000) + extras + parity fixtures + model card

README.md

@@ -0,0 +1,77 @@
+---
+license: mit
+base_model: baidu/Unlimited-OCR
+tags:
+  - onnx
+  - webgpu
+  - ocr
+  - vision
+  - document-parsing
+  - onnxruntime-web
+  - browser
+library_name: onnxruntime
+---
+
+# Unlimited-OCR DeepEncoder — ONNX (browser-ready vision stack)
+
+The **complete vision encoder of [baidu/Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR)**
+(the DeepSeek-OCR family DeepEncoder: **SAM ViT-B → CLIP-L fusion → linear projector**) exported as a
+single ONNX graph, verified numerically identical to the PyTorch reference
+(**torch-vs-onnxruntime cosine 1.0000000**, maxAbsDiff 5e-5).
+
+Paired with the language-model GGUF, this runs the full Unlimited-OCR pipeline **entirely in a
+browser tab** — image in, det-boxed markdown out, no server. Reference implementation:
+[NakliTechie/gemma4-webgpu](https://github.com/NakliTechie/gemma4-webgpu) (hand-written WGSL
+WebGPU engine with DeepSeek-V2-MoE decoder support; see `ocr-demo.html`). Measured on Apple
+Metal-3: vision 1.5 s (onnxruntime-web WebGPU EP) + decode at ~125 tok/s → **single page OCR'd
+in < 3 s in-tab**.
+
+## Files
+
+| file | what |
+|---|---|
+| `deepencoder_fp32.onnx` | SAM→CLIP→projector, opset 18, static `[1,3,1024,1024]` → `[1,256,1280]` (fp32, 1.6 GB) |
+| `deepencoder_extras.npz` | `image_newline`, `view_seperator` embeddings (`[1280]` f32 each) — spliced host-side, not part of the graph |
+| `deepencoder_ref_in.npy` / `deepencoder_ref_out.npy` | parity fixtures (seed-42 input + reference output) — verify your runtime reproduces cosine ≈ 1.0 |
+| `control_doc.png` | the ground-truth test document used in the e2e verification |
+
+## I/O contract
+
+- **Input** `pixel_values` `[1,3,1024,1024]` f32 — RGB, normalized `(x/255 − 0.5)/0.5` (mean/std 0.5, per the upstream processor config).
+- **Output** `vision_embeds` `[1,256,1280]` f32 — 256 vision tokens (16×16 grid) already projected to the decoder's hidden size.
+
+### Splicing into the decoder sequence (single 1024² global view)
+
+```
+[BOS(0)]
++ 16 rows × ( 16 patch embeds [row-major] + image_newline )
++ view_seperator                       → 273 embedding rows total
++ prompt token ids
+```
+
+The 273 rows replace `<image>` placeholder tokens (id 128815) — feed them as `inputs_embeds`.
+
+### ⚠ The prompt matters more than you think
+
+Use this model family's canonical prompt: **`<image>document parsing.`**
+DeepSeek-OCR-v1 phrasings fail on Unlimited-OCR — `<image>\nFree OCR.` produces an **immediate EOS
+even in the bf16 reference**, and `<|grounding|>Convert the document to markdown.` makes it recite
+instruction boilerplate. (Verified against the full-precision HF stack; see the engine repo's
+`reference/pytorch/hf_image_control.py`.)
+
+## The rest of the pipeline
+
+- **Decoder GGUF** (DeepSeek-V2 MoE, 64×550M, 12 layers — *no MLA despite the family name*:
+  `use_mla: false`, plain Llama MHA): community K-quants at
+  [sahilchachra/Unlimited-OCR-GGUF](https://huggingface.co/sahilchachra/Unlimited-OCR-GGUF)
+  (Q4_K_M 1.95 GB works with the engine's in-shader q4k/q8 storage).
+- **Browser engine**: [NakliTechie/gemma4-webgpu](https://github.com/NakliTechie/gemma4-webgpu) —
+  WGSL kernels incl. on-GPU top-6 MoE routing and batched expert GEMVs; crossLabDiff-verified
+  against the HF bf16 reference (per-layer sweep, argmax match).
+
+## Provenance & license
+
+Weights are a mechanical export of [baidu/Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR)
+(MIT, © 2026 Baidu — notice retained per license). Export script:
+`reference/pytorch/export_deepencoder_onnx.py` in the engine repo. fp32; an fp16 (~800 MB) pass is
+planned — mind LayerNorm precision if you convert yourself.

deepencoder_extras.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8ad4355b1a1db528afdbd2560d628656b9fbd8fa3ec2d9a2c99b216baeda522f
+size 10780

deepencoder_fp32.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a88301420fa7f306193aef1fca2bd229c2e97382c3d6c1f468adba686c4a5a2
+size 1604047237

deepencoder_ref_in.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37b05b437b993b85c8675d24e9134e8612db2c6f6df758496d186e53cba56e83
+size 12583040

deepencoder_ref_out.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f26707d3018950a9f6877e348cecd39613bb995ef74a1f41e9e5572461d2c750
+size 1310848