sabafallah/DeepSeek-OCR-2-GGUF

DeepSeek-OCR-2 GGUF (for llama.cpp PR #20975)

This repository provides GGUF model files required for running the DeepSeek-OCR-2 / MTMD support introduced in the following llama.cpp pull request:

👉 llama.cpp PR #20975 https://github.com/ggml-org/llama.cpp/pull/20975

These models are only compatible with the PR branch and will not run on upstream llama.cpp main.

---

📥 Download

You can download the model files directly using:

huggingface-cli download <this-repo> --include "deepseek-ocr-2-bf16.gguf" --local-dir gguf_models/deepseek-ai
huggingface-cli download <this-repo> --include "mmproj-deepseek-ocr-2-bf16.gguf" --local-dir gguf_models/deepseek-ai

---

🚀 Run Example

Use with llama-server

build/bin/llama-server \
  -hf sabafallah/DeepSeek-OCR-2-GGUF:bf16 \
  --chat-template deepseek-ocr --no-jinja \
  --temp 0 \
  --flash-attn off \
  --no-warmup \
  -n 2048 \
  --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 \
  --dry-penalty-last-n -1 --dry-sequence-breaker none

Use the llama-mtmd-cli executable from the PR:

build/bin/llama-mtmd-cli \        
-m gguf_models/deepseek-ai/deepseek-ocr-2-bf16.gguf \
--mmproj gguf_models/deepseek-ai/mmproj-deepseek-ocr-2-bf16.gguf \
--image tmp/mtmd_test_data/webpage_tall.png -p "Free OCR." \
--chat-template deepseek-ocr --no-jinja \
--temp 0 \
--flash-attn off \
--no-warmup \
-n 2048 \
--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 \
--dry-penalty-last-n -1 --dry-sequence-breaker none