Hugging Face's logo

factlogic
/
phoenix-arabic-manuscript-htr

Image-to-Text
Core ML
Arabic
kraken
htr
ocr
arabic
arabic-manuscripts
historical-documents
ctc
human-in-the-loop
Eval Results (legacy)
Model card Files
xet
 Community
1
phoenix-arabic-manuscript-htr / PROJECT_FACTS_DOSSIER.md
factlogic's picture
factlogic
Update publication links for Phoenix model name (part 2)
7211659 verified
preview code
|
Raw
History Blame
12.3 kB

📋 PROJECT FACTS DOSSIER: Rakīm AI (رَقِيم)

This dossier provides a verified, factual inventory of the Arabic manuscript HTR/OCR project based on a deep inspection of codebase files, test logs, config files, and the presentation dashboard.

1. Project Naming Inventory & Recommendation

  • Names Currently in Use:

    1. Phoenix AI: Main name defined in config.py (PROJECT_NAME = "Phoenix AI"), docker-compose.yml, and environmental configurations. Developed by the "Phoenix Team" (فريق فنكس).
    2. Muharaf (مُحرف): Used in the core README.md, the older project_review_report.md ("Muharaf OCR Project Review & Execution Guide"), and the SQLite cache database filename (muharaf_cache.db).
    3. Rakīm / رَقيم (Rakīm AI): Used in the unified presentation document المقارنة_الموحدة.md and the HTR handoff notes HTR_handoff_v2.md.
    4. Athar: Mentioned in historical requirements but not active in core codebase configurations.

  • Unified Name Recommendation: We strongly recommend unifying the project under the name Rakīm AI (رَقيم) for the final report and user interface.

    • Rationale: "رَقِيم" is a classical Arabic term meaning "inscribed tablet" or "writing," which perfectly aligns with the cultural preservation of manuscripts. It sounds highly professional, unique, and premium for a national AI competition, whereas "Phoenix AI" represents the team name and "Muharaf" is a generic tool name.

2. One-Paragraph Project Description

Rakīm AI is an intelligent, end-to-end pipeline system designed to transcribe, segment, analyze, and catalog historical handwritten Arabic manuscripts (HTR/OCR), with a primary focus on Maghrebi calligraphic hands. The system takes raw manuscript page images, processes them through a layout routing engine to select optimal neural segmentation and recognition models, classifies lines into main text (matn) or margins (hashiya), corrects transcriptions using a local character n-gram language model, and provides a suite of AI-assisted tools (automatic summarization, title suggestion, entity extraction, context dictionary definitions, fuzzy Levenshtein search, and manuscript catalog matching against OpenITI and Fihrist databases).

3. Implemented Architecture

The system implements a structured processing pipeline:

graph TD
    A[Raw Page Image] --> B[Image Deskew & Preprocessing]
    B --> C[Kraken nlbin Binarization]
    C --> D[Layout Router]
    D -->|Framed Layout| E1[Logic Segmentation Model: logic_philosophy_v2_seg]
    D -->|General Layout| E2[Default Segmentation Model: seg_best]
    E1 & E2 --> F[Line Polygons & Coordinates]
    F --> G[Adaptive Line Merging & Noise Filtering]
    G --> H[Region Classifier: Matn vs. Hashiya]
    H --> I[Recognition Model Routing]
    I -->|Framed Layout & Logic| J1[Logic Rec Model: logic_philosophy_v2_rec]
    I -->|General/Dense Layout| J2[Default Rec Model: exp6_best]
    J1 & J2 --> K[CTC Beam Search decoding / Optional LM fusion: char8_trainval.lm]
    K --> L[Logical Arabic Text Output]
    L --> M[Post-OCR AI Analysis / Fuzzy Search / Catalog Matching]
  • Optional Components: Language Model (LM) decoding (activated by setting USE_LM=true in .env) and AI Analysis Features (rely on external OpenRouter/Gemini API keys).

4. Model Inventory

Model File Type Role Used in Profile Status / Notes
seg_best.mlmodel Segmentation (BLLA) Default line segmenter default profile, and fallback in auto Implemented and active. Tends to over-segment pages with margins/marginalia.
exp6_best.mlmodel Recognition (TorchSeq) Default recognition model for Maghrebi handwriting default profile, and fallback in auto Implemented and active. Highly accurate on Maghrebi manuscript hands.
logic_philosophy_v2_seg.mlmodel Segmentation (BLLA) Specialized segmenter for framed/dense manuscripts logic profile, and selected in auto when frame detected Implemented and active. Renamed from logic_shamsiyya_seg.mlmodel.
logic_philosophy_v2_rec.mlmodel Recognition (TorchSeq) Specialized recognition for philosophy manuscripts logic profile, and selected in auto when framed page detected Implemented and active. Renamed from logic_shamsiyya.mlmodel or reg.mlmodel.
char8_trainval.lm Language Model (n-gram) Character-level language model for error correction Enabled via USE_LM=true Implemented and verified. Improves CER on test sets.
reg(96.5).mlmodel Recognition (Legacy) Legacy standard recognition model None Obsolete. Must NOT be used by default (causes gibberish outputs on Maghrebi hands).
logic_shamsiyya.mlmodel Recognition (Alternative) Alternative name / backup of logic recognition model None Present in models directory.
logic_shamsiyya_seg.mlmodel Segmentation (Alternative) Alternative name / backup of logic segmentation model None Present in models directory.

5. Model Routing System

The application features an independent routing system that selects segmentation and recognition models dynamically or lets users force them:

  • Segmentation Profiles: auto | default | logic
  • Recognition Profiles: auto | default | logic

Routing Decisions under auto:

  1. Framed Page Layout: The router scans the page for a framed structure (straight horizontal and vertical borders enclosing a text block). If detected, it sets the route to segmentation=logic and recognition=logic (reason: framed_text_block).
  2. Dense Handwritten Layout (No Frame): The router measures ink density, horizontal line groupings, and the presence of red ink. If it looks like a dense philosophy page, it uses segmentation=logic to capture crowded text, but falls back to recognition=default (exp6_best.mlmodel) because the general recognition model is more accurate for dense handwriting. (reason: dense_red_layout:...;recognition_default_for_dense_layout).
  3. General Layout: If layout cues are weak, it falls back to segmentation=default (seg_best.mlmodel) and recognition=default (exp6_best.mlmodel). (reason: forced_default).

Verification Reports (output/logic_router_tests/):

  • report_default_default.md: Confirms that when profiles are forced to default, both models run seg_best.mlmodel and exp6_best.mlmodel on all samples.
  • report_logic_default.md: Confirms that when profiles are forced to logic/default, the system pairs logic_philosophy_v2_seg.mlmodel with exp6_best.mlmodel.
  • report_logic_logic.md: Confirms that when profiles are forced to logic/logic, the system pairs logic_philosophy_v2_seg.mlmodel with logic_philosophy_v2_rec.mlmodel.
  • report.md (auto/auto): Verifies that sample_01_framed_margin.jpg correctly routes to logic/logic (framed), while sample_02, 03, 04 correctly route to logic/default (dense handwriting).

6. Current .env Configuration (Safe Values)

Safe parameters extracted from backend/.env (and backend/.env.example templates):

  • USE_VISION_OCR=false (forces local Kraken engine instead of cloud vision APIs).
  • REC_MODEL_FILE=exp6_best.mlmodel (default recognition model).
  • SEG_MODEL_FILE=seg_best.mlmodel (default segmentation model).
  • DEVICE=auto (resolves to cuda if torch detects GPU, else cpu).
  • USE_FP16=true (enables half-precision for GPU speedups).
  • USE_LM=false (disabled by default in the environment for stable demonstration, but fully functional when set to true).
  • CANDIDATES_ENABLED=false (disabled by default; when enabled, allows extraction of visual/text alternatives).
  • AUTO_CANDIDATES=false (disabled by default; when enabled, automatically pre-computes candidates for low-confidence lines).
  • FAST_SEGMENTATION=true (reduces neural BLLA segmentation time to 1-3 seconds instead of 15-30 seconds).
  • MAX_IMAGE_SIZE=4000 (limits image resolution to prevent memory overflow).

Note: Confidential API keys (OPENROUTER_API_KEY and GOOGLE_GEMINI_KEY) are loaded but masked for safety.

7. GPU/Docker Runtime Environment

  • Docker Services (from docker-compose.yml):
    1. nginx: Runs Alpine Nginx load balancer on port 80, mapping to /etc/nginx/nginx.conf.
    2. ocr-worker: Runs the backend on port 8001 with 2 or 3 replicas (CPUs limited to 2.0, Memory 4GB in production).
  • Local Access URLs:
    • Through load balancer: http://localhost/app
    • Direct API: http://localhost:8001/app / API Docs: http://localhost:8001/docs
  • GPU Status in Docker:
    • Docker-compose files force DEVICE=cpu and USE_FP16=false in the environment variables for workers. Therefore, Docker runs on CPU only.
    • Local execution (running python run.py natively) supports GPU and mixed precision (DEVICE=auto will resolve to cuda on local setups with CUDA-compatible cards like the RTX 4050).
  • Verification Command: To check actual logs inside Docker, use: docker logs phoenix_e2e_lmtrue --tail 150 (or the active worker container name). To inspect system resources and hardware: GET http://localhost:8001/api/system/info

8. Language Model (LM) Decoding Status

  • Status: Fully implemented and integrated.
  • Model File: نماذج/char8_trainval.lm (character n-gram language model, ~34.5 MB).
  • Operation: Shallow fusion decoding. It replaces standard greedy decoding in Kraken with a Beam Search CTC decoder (LMDecoder in backend/app/services/ocr/lm_decoder.py) that scores character transitions based on language probability.
  • Measured Improvements:
    • RASAM Dataset (Maghrebi): Character Error Rate (CER) improved from 7.48% → 6.79% (absolute reduction of 0.69%).
    • TariMa Dataset: Character Error Rate (CER) improved from 10.27% → 8.81% (absolute reduction of 1.46%).
  • Technical Note: This is a local statistical n-gram model executing via C++ binding (Kraken/PyTorch), not an external generative API (like ChatGPT).

9. Catalog and Index Data

  • Metadata Databases: Combined dataset utilizing OpenITI (13,364 works) and FIHRIST (15,543 manuscripts from UK libraries).
  • Catalog Statistics (catalog_stats.json):
    • Total Works: 35,632
    • Total Manuscript Copies: 49,980
    • Works with Multiple Copies: 6,003 (16.85%)
    • Average Copies per Work: 1.4
    • Top Institutions: Wellcome Collection, London (11,192 copies); Bodleian Library, Oxford (7,167 copies); University Library, Cambridge (5,233 copies).
    • Top Sources: Fihrist (36,616 copies); OpenITI (13,364 copies).
  • App Implementation: Fully implemented as a dedicated tab ("فهرس المخطوطات") in the frontend workspace. It supports search queries (Arabic/Latin) and matches OCR-extracted text fingerprints against catalog works (/api/catalog/match-content).

10. UI Enhancements

Verified frontend improvements:

  1. Organized Toolbar Layout: Restructured into a clean two-row toolbar:
    • Row 1: Main operations (File upload, Segmentation/Recognition selectors, region filters, bounding box toggle, direct mode, cache clear, and export).
    • Row 2: Labeled "🤖 التحليل الذكي (AI)" with analysis buttons (terminology explanation, page explanation, entity extraction, summary/title generation, uncertain highlighting, candidates, and manuscript search).
  2. Scroll Lock Fix: Replaced layout locks on mobile/tablet viewports (max-width: 900px) with natural vertical scrolling (overflow-y: auto !important on body and panels), ensuring the text workspace is fully accessible on all screen sizes.
  3. Dedicated Dropdowns: Added separate dropdown lists for Segmentation (القص) and Recognition (التفريغ) with choices auto, default, and logic to support independent routing control.
  4. Arabic Localization: Buttons, titles, and explanations are localized into clear Arabic.