sahilchachra

/

Unlimited-OCR-GGUF

like 138

Let me take a look and get back!

The model ran fine for me. Sharing steps. Please cross-check it.

1. Build llama.cpp with DeepSeek-OCR support (PR #17400) — mandatory
   Stock llama.cpp / Ollama / LM Studio cannot load this model's vision tower. You must build the PR:

git clone --depth 1 https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch --depth 1 origin pull/17400/head:pr17400 && git checkout pr17400
cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF # add -DGGML_CUDA=ON for GPU
cmake --build build -j --target llama-mtmd-cli

2. Download a quant + the projector (you always need both)
   huggingface-cli download sahilchachra/Unlimited-OCR-GGUF
   --include "Unlimited-OCR-Q4_K_M.gguf" "mmproj-Unlimited-OCR-F16.gguf" --local-dir ./uocr

3. Run it (the command I verified)
   ./build/bin/llama-mtmd-cli
   -m ./uocr/Unlimited-OCR-Q4_K_M.gguf
   --mmproj ./uocr/mmproj-Unlimited-OCR-F16.gguf
   --image /path/to/your/document.png
   -p "<|grounding|>Convert the document to markdown."
   --chat-template deepseek-ocr --temp 0 -n 4096

Similar problem, after successful OCR/Markdown, endless repetition :

llama.cpp/build/bin/llama-mtmd-cli -m models/Unlimited-OCR-BF16.gguf --mmproj models/mmproj-Unlimited-OCR-F16.gguf --image /home/alerant/models/szamla.png -p "Free OCR" --chat-template deepseek-ocr --temp 0 --repeat-penalty 1.0

Tested with llama.cpp(5c7c22c3e) and Q8 model, I also find it can not handle complex image content like below. It works for simple images. And I tested this complex image with the official demo page(https://huggingface.co/spaces/baidu/Unlimited-OCR), it's parsed without issue. So that means we may have some precision issue in llama.cpp porting.

And using "<|grounding|>Convert the document to markdown." I get nothing output. The prompt "document parsing." works. But it fails into repeat output after half page content parsed. I'm using Mac M4.

Similar problem, after successful OCR/Markdown, endless repetition :

llama.cpp/build/bin/llama-mtmd-cli -m models/Unlimited-OCR-BF16.gguf --mmproj models/mmproj-Unlimited-OCR-F16.gguf --image /home/alerant/models/szamla.png -p "Free OCR" --chat-template deepseek-ocr --temp 0 --repeat-penalty 1.0

Solved:
llama.cpp//build/bin/llama-mtmd-cli -m models/Unlimited-OCR-Q8_0.gguf --mmproj models/mmproj-Unlimited-OCR-F16.gguf --image models/szamla.png -p "<|grounding|>Convert the document to markdown." --chat-template deepseek-ocr --temp 0 --flash-attn off --no-warmup -n 4096 -c 16384 --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --dry-penalty-last-n -1 --dry-sequence-breaker none

But the quality is bad, does not understand Hungarian language

Tested with llama.cpp(5c7c22c3e) and Q8 model, I also find it can not handle complex image content like below. It works for simple images. And I tested this complex image with the official demo page(https://huggingface.co/spaces/baidu/Unlimited-OCR), it's parsed without issue. So that means we may have some precision issue in llama.cpp porting.

From sabafallah/Unlimited-OCR-GGUF model card, I see we must use the branch at https://github.com/ggml-org/llama.cpp/pull/24975 and it works for this image.
So it seems this model repository model card is not correctly describing the latest dev state.

Thanks, will update it!

The PR referenced in your model card is labeled as Merged. Is building from git checkout pr17400still required? I’ll try both.

https://github.com/ggml-org/llama.cpp/pull/17400

The PR referenced in your model card is labeled as Merged. Is building from git checkout pr17400still required? I’ll try both.

https://github.com/ggml-org/llama.cpp/pull/17400

I will also give a try once! Thanks!

Basic OCR is working fine with a standard build from main via llama-server.
version: 0.1.2-dev (build 10496, commit af5172627)

Here are the startup parameters:

./bin/llama-server --verbose --host 0.0.0.0 --port 4077 \
--temp 0  \
-a Unlimited-OCR \
--model ./gguf/Unlimited-OCR-Q8_0.gguf \
--mmproj. //gguf/mmproj-Unlimited-OCR-F16.gguf

Here's the output of a simple screenshot with no prompt:

title [33, 24, 611, 77]Bluetooth & devices > Cameras
image [62, 164, 94, 205]
text [119, 171, 320, 202]Add a network camera
text [736, 171, 905, 201]Search for cameras
image [61, 294, 96, 334]
text [120, 284, 367, 345]UNKNOWN (SNH-V6414BN)
192.168.1.28
image [62, 426, 97, 468]
text [121, 421, 368, 482]UNKNOWN (SNH-V6414BN)
192.168.1.27
text [117, 530, 207, 558]Username
text [116, 620, 203, 650]Password
text [751, 711, 860, 740]Add camera
image [62, 845, 96, 887]
text [119, 838, 367, 899]UNKNOWN (SNH-V6414BN)
192.168.1.32

here's the image parsed:

Larger English PDFs have been falling back to Chinese. I'm experimenting with different context sizes, other parameters and prompts.