README.md

metadata

title: OCR Notification & Data Extractor
emoji: 🧾
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false
license: mit

OCR Notification & Data Extractor

Upload scanned PDFs or images and turn them into structured, exportable data. Built for teams that need to pull actionable "notification" events and key fields out of paper-heavy workflows:

• Construction / SAP data support — change orders, delay notices, punch list items, RFIs, PO numbers, SAP document numbers, quantities.
• Compliance — non-conformances, violations, expirations, corrective actions, certification codes (ISO, OSHA, ASTM, ANSI).
• Business finance — past-due invoices, payment rejections, credit holds, invoice numbers, dollar amounts, due dates.

Notifications vs. fields

The tool treats two kinds of requests differently, rather than running one blanket extraction pass over every document:

• Notifications — keyword/phrase hits that flag an event (a boolean "this happened on this page"), e.g. "past due", "non-compliant", "change order". Each category (construction/compliance/finance/general) has its own built-in phrase list; you can add your own custom notification keywords too.
• Fields — labeled values extracted as data, not just flagged. Standard fields are scoped to the categories you select (see CATEGORY_FIELD_MAP in extractors.py) so a compliance document doesn't generate empty invoice-number columns: construction gets PO/SAP numbers, compliance gets certification codes, finance gets invoice numbers/percentages, and dates/amounts apply where relevant. On top of that, you can name your own custom fields — any label that appears in the document followed by a value (e.g. "reference number", "gross weight", "delivery date") — and the app finds the label and captures what follows it.

How it works

1. Tailor extraction (chat tab) — describe in plain language what you want flagged vs. pulled as data (e.g. "flag overdue invoices over $10k and non-compliance notices, and pull the reference number and gross weight"). A Hugging Face-hosted chat model turns that into extraction settings (categories, notification keywords, custom fields, export filter), which populate the Upload & Run tab automatically. You can also skip the chat and set everything by hand there.

2. Upload one or more PDFs or images.

3. Each page's text is pulled directly from the PDF when it has a text layer; scanned/image-only pages are rendered and run through Tesseract OCR.

4. The text is scanned for notification keywords and both standard and custom fields, per the scoping above.

5. Results are shown as two tidy (one-fact-per-row) tables — extracted fields and notification hits — and exported as three CSVs:

   • ocr_fields_*.csv — file, page, field_type (standard/custom), field_name, field_value, OCR source/confidence.
   • ocr_notifications_*.csv — file, page, category, keyword, context, OCR source/confidence.
   • ocr_run_metadata_*.csv — an audit record of exactly what produced the run: categories, notification keywords, custom fields, the match filter, the chat model used, and the full chat transcript. This is the lineage trail for anyone auditing why a given row was (or wasn't) extracted.

   Long/tidy tables were chosen over one wide fixed-schema table so that irrelevant fields never show up as blank columns, and so the output loads cleanly into a database, pivot table, or BI tool without reshaping — consistent with standard data-quality practice (e.g. DAMA DMBOK's completeness, validity, and lineage dimensions).

Chat assistant setup

The chat tab calls the Hugging Face Inference API, so it needs a token:

• On a Space: add a repo secret named HF_TOKEN (Settings -> Variables and secrets) with a token that has Inference API access.
• Locally / as a fallback: paste a token into the "Hugging Face token" field in the chat tab; it's only used for that session.

The model defaults to HuggingFaceH4/zephyr-7b-beta (ungated). Override it with the HF_CHAT_MODEL environment variable/secret if you'd rather use a different instruct model your token can access.

If the chat call fails or no token is available, the Upload & Run tab still works fully via its manual checkboxes/textboxes — the chat is a convenience layer on top, not a dependency.

Local development

pip install -r requirements.txt
# Tesseract must also be installed on the system, e.g.:
#   apt-get install tesseract-ocr
export HF_TOKEN=hf_...   # optional, enables the chat tab
python app.py

Files

• app.py — Gradio UI and orchestration.
• ocr_engine.py — PDF/image loading and OCR (PyMuPDF + Tesseract).
• extractors.py — keyword notification scanning, category-scoped standard field extraction, and generic label->value custom field extraction.
• chat_config.py — chat assistant that turns natural-language instructions into extraction settings via a Hugging Face chat model.