---
title: ERP-DocIQ
emoji: 📄
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: 6.9.0
app_file: gradio_app.py
pinned: false
license: mit
short_description: OCR/IDP + ERP NLQ chatbot on small models (MiniCPM)
tags:
  - build-small-hackathon
  - backyard-ai
  - best-minicpm-build
  - best-agent
  - tiny-titan
  - minicpm
  - ocr
  - idp
  - nlq
  - rag
  - agent
  - fine-tuning
---

# ERP-DocIQ — back-office automation on small models

> **Build Small Hackathon · Practical track (Backyard AI).** A useful, problem-solving app
> that runs on hardware you own — built entirely on **open models ≤32B**, with **OpenBMB
> MiniCPM** as the load-bearing model (and fine-tuned for the domain).

## 💡 The idea
Retail back-offices drown in paperwork and report requests. **ERP-DocIQ** is an open-source,
UiPath-style assistant that **reads your documents**, **answers questions about your ERP in
plain English**, and **automates the boring clicks** — all on small models, no per-robot
license, no data leaving your tenancy.

## 🧩 The business problem
A retail brand's IT team runs **UiPath** for invoice/PO processing and portal automation. It's
**expensive** (per-robot / AI-Unit licensing, six figures/yr at volume), **brittle** (selector
recorders break on every UI change), **locked-in** (closed IDP models you can't swap or
fine-tune), and **slow to change** (new layouts wait on a closed retraining toolchain). On top
of that, every "what did we spend in Q2?" becomes a **BI ticket**. They wanted an AI-native,
open, cheaper alternative they control.

## 🛠️ The solution approach — one Gradio app
1. **Read any document (OCR + IDP).** A hybrid pipeline — OCR → classify → extract → normalize →
   enrich (RAG) → validate → post / human-review — reads orders, receipts, invoices, contracts
   and complex multi-layer forms, **even messy scans/photos**, into structured ERP records.
2. **Ask your ERP reports (ERP DocIQ chatbot).** Natural-language **NLQ → SQL**, analytics,
   summaries and **"why"** reasoning over a simulated retail ERP (vendors · POs · invoices · GL ·
   inventory · returns). Every figure comes from **real SQL over the data** — the model only
   *phrases* the answer, it never invents numbers.
3. **Automate the clicks (agentic browser).** A self-correcting, multi-step browser agent drives
   a portal (dashboard → Procurement → Create Order → read the complex form) — selector-free and
   self-healing, replacing fragile RPA recorders.

## 🤖 Models used — small (≤32B), the right one for each job
Three labs, eight models, all under the cap. **MiniCPM is the core.**

| Lab | Model | Params | Role in ERP-DocIQ |
|---|---|---:|---|
| **OpenBMB** | **MiniCPM-V-4.6** | 8B | **primary OCR + vision extraction** (reads messy/rotated/scanned docs → JSON) |
| **OpenBMB** | MiniCPM-o-4.5 | 8B | alt omni VLM |
| **OpenBMB** | **MiniCPM3-4B** | **4B** | **ERP reasoning · NLQ→SQL · summarization — and the fine-tune target** |
| **Cohere** | Aya-Vision-8B / 32B | 8–32B | alt OCR / VQA backend (23 languages) |
| **Cohere** | Command R7B | 7B | alt RAG · NLQ · grounded reasoning |
| **Black Forest Labs** | FLUX.1 [dev]/[schnell] | 12B | image *generation* → synthetic OCR stress docs (honestly, not an OCR model) |

A single ~8B MiniCPM-V powers OCR **and** vision extraction **and** the grounded chat phrasing —
so the whole product runs on small, swappable, open weights. `GET /api/models` lists which are live.

## 🎯 Fine-tuning — adapting a small model to the ERP domain
We fine-tune **OpenBMB MiniCPM3-4B (4B)** on an instruction dataset built from the ERP
knowledgebase (`results/erp_sft.jsonl`):
- **Production:** a LoRA (PEFT/TRL `SFTTrainer`) recipe — `scripts/finetune_erp.py --backend hf`.
- **Offline CPU demo (runs anywhere):** trains the ERP NLQ-routing head on the same data with a
  real train/test split — **untrained 8.3% → fine-tuned 91.7% (+83 pts)**, 100% routed-SQL
  execution. See `results/erp_finetune_report.json`.

## 📈 Benefits / value delivered
- **Significantly lower inference cost** vs per-document RPA licensing (model routing + prompt caching).
- **No vendor lock-in, data stays on-prem** — open weights you can run, read and fine-tune.
- **Self-service ERP analytics** (NLQ) deflects routine BI/report tickets off the queue.
- **Reads documents classic OCR can't** — MiniCPM-V cuts character error **~5.6× vs Tesseract**
  (CER **2.6%** vs 14.7%; see `results/ocr_quality_report.json`); field-exact **90.7%**.
- **Honest, measured** — every claim is backed by a published eval/quality/fine-tune report.

## ✅ How it meets the Build Small criteria
- **≤32B params** — every model is ≤32B; the reasoning/NLQ engine + fine-tune target is **4B**.
- **Best MiniCPM Build** — MiniCPM-V (OCR + vision) and **MiniCPM3-4B** (reasoning/NLQ +
  fine-tuned) are the core of the experience; vision/omni variants qualify.
- **Best Agent** — multi-step, self-healing agentic browser automation + an IDP state graph.
- **Ships as a Gradio Space** in the Build Small org; runs offline (deterministic ERP engine +
  sidecar OCR) and upgrades to hosted MiniCPM when keys are set.

## ▶️ Use it (tabs)
- **Process a document** — pick an OCR backend (`auto`, `minicpm`, `tesseract`) + a sample
  (try `extreme_receipt_photo` or `complex_invoice_messy`) or upload your own → multi-layer fields + KPIs.
- **ERP DocIQ (chat)** — ask *"Why did spend rise in Q2 2026?"*, *"Top vendors by spend"*,
  *"late-payment rate"* → grounded answer + SQL + the fine-tuning panel.
- **Search (RAG)** — semantic vendor-master retrieval. **Web Automation** — multi-step browser flow.

## 📦 Published results (`results/`)
- `ocr_quality_report.json` — OCR CER/WER + field accuracy per backend.
- `erp_finetune_report.json` + `erp_sft.jsonl` — fine-tune metrics + the instruction dataset.

## ⚙️ Configure (Space → Settings → Variables and secrets)
- Variables: `MINICPM_BASE_URL=https://api.modelbest.cn/v1`, `MINICPM_MODEL=MiniCPM-V-4.6-Instruct`
- Secret: `MINICPM_API_KEY=…` (OpenBMB/ModelBest). Tesseract ships via `packages.txt`.
- **Runs fully offline without a key** — ERP DocIQ uses its deterministic SQL engine and OCR
  falls back to the sidecar, so every tab works.

## 🎥 Demo & social
- **Demo video:** https://youtu.be/mWs7eRVH_GM
- **Social post:** https://www.linkedin.com/posts/kaniskamandal_huggingface-buildsmall-buildsmall-share-7472163579094401024-fjp9/