PROTOCOL.md

Pre-registered Research Protocol

1. Administrative record

• Project: TrOCR Sinhala Printed Lines
• Protocol version: 1.0
• Protocol date: 2026-07-28
• Researcher: Kalana Hewapathirana
• Planned repository: kaan84/trocr-sinhala-printed-lines
• Phase: Phase 0, before training

Commit this file before training. Any later change must be appended to the deviation log with its date, rationale, and expected impact.

2. Research question

Does synthetic-data fine-tuning of Ransaka/TrOCR-Sinhala improve printed, line-level Sinhala OCR on real documents?

3. Hypotheses

Primary hypothesis H1

Fine-tuning the starting checkpoint on Ransaka/sinhala_synthetic_ocr-large will reduce CER on a real printed Sinhala line benchmark relative to the unchanged checkpoint and Tesseract 5.5.

Secondary hypothesis H2

The fine-tuned checkpoint will reduce WER relative to the unchanged checkpoint and Tesseract 5.5.

Secondary hypothesis H3

Performance will be lower on older or visibly degraded documents than on newer or cleaner documents.

4. Experimental unit and scope

The evaluation unit is one cropped text-line image paired with one reference transcription. Page-level and line-level metrics must not be combined in the same comparison table without an explicit unit label.

Included:

• Printed Sinhala
• Real and synthetic text lines
• CER and WER
• Starting checkpoint and Tesseract comparisons
• Optional Surya comparison

Excluded:

• Handwriting
• End-to-end page layout analysis
• Production-readiness claims
• Hyperparameter optimization outside the declared contingency rules

5. Assets and fixed revisions

Before training, replace TO_RECORD with immutable Hub commit hashes:

Asset	Role	Revision
Ransaka/TrOCR-Sinhala	Starting model	TO_RECORD
Ransaka/sinhala_synthetic_ocr-large	Train/validation	TO_RECORD
avishadilhara/sinhala-ocr-lk-acts-1010	Real evaluation source	TO_RECORD

The experiment manifest must also record the Transformers, Datasets, PyTorch, CUDA, jiwer, Pillow, NumPy, and Tesseract versions.

6. Data splits

Synthetic dataset

Use the published dataset's single train split.

• Split seed: 42
• Train share: 90%
• Validation share: 10%
• Split method: deterministic shuffled index split
• No sample may appear in both partitions
• Save the exact indices to version-controlled JSON files
• Record SHA-256 checksums of the index files

Real-document benchmark

Use only the published test partition for the final real-document analysis. Do not use its labels for training or hyperparameter selection.

Because the source is page-level, create line crops and transcriptions using one versioned procedure. Record:

• Source page identifier
• Line identifier
• Crop coordinates
• Reference transcription
• Segmentation method and version
• Any exclusion reason
• Document year or period, if available

Freeze the line benchmark before final evaluation. Manual corrections made after viewing model predictions invalidate the final test and must trigger a new held-out test set.

7. Preprocessing

Images

• Convert to RGB
• Use the processor associated with the starting checkpoint
• Do not apply test-time enhancement unless declared as a separate experimental condition
• Preserve raw evaluation crops

Text normalization

Report both raw and normalized metrics.

The fixed normalized condition:

1. Unicode normalize with NFC.
2. Convert line breaks and tabs to spaces.
3. Collapse consecutive Unicode whitespace to one ASCII space.
4. Strip leading and trailing whitespace.
5. Preserve Sinhala characters, punctuation, numerals, and letter case.

Do not remove punctuation or diacritics in the primary analysis. Any alternative normalization is exploratory and must be labeled accordingly.

8. Training configuration

Parameter	Fixed value
Model	Ransaka/TrOCR-Sinhala
Optimizer	AdamW
Learning rate	4e-5
Epochs	10
Effective batch size	8
Precision	FP16 if supported
Seeds	42, 123, 2026
Generation beams	4
Save strategy	Every epoch
Selection rule	Lowest validation CER

Gradient accumulation may change only to preserve effective batch size 8. Physical batch size may be lowered for memory limits.

9. Contingency rules

Permitted without redefining hypotheses:

• Reduce physical batch size and increase gradient accumulation.
• Resume from the latest complete checkpoint after infrastructure interruption.
• Use gradient checkpointing if the GPU cannot hold the declared effective batch.
• Disable FP16 only if the hardware or software fails to support it reliably.

Not permitted in the confirmatory experiment:

• Selecting a new learning rate after inspecting final-test results
• Increasing epochs because final-test performance is weak
• Training on real-test labels
• Removing difficult samples based on prediction errors
• Reporting only the best seed

Any other adjustment becomes exploratory and must be separated from confirmatory results.

10. Baselines

Required:

1. Unchanged Ransaka/TrOCR-Sinhala
2. Tesseract 5.5
3. Fine-tuned TrOCR for all three seeds

Optional:

4. Surya OCR, only after recording the exact version and applicable model terms

Every system must receive the same frozen line images. Use system-appropriate decoding without model-specific alteration of references.

11. Metrics

Primary: Character Error Rate.

Secondary: Word Error Rate.

Use jiwer with transformations that exactly implement Section 7. Compute:

• Per-sample edit counts
• Corpus-level raw CER and WER
• Corpus-level normalized CER and WER
• Mean and standard deviation across training seeds
• Paired bootstrap 95% confidence interval for differences, resampling line IDs with 10,000 replicates and bootstrap seed 42

12. Decision rules

H1 is supported only when:

1. The fine-tuned model's mean normalized CER is lower than both the starting checkpoint and Tesseract 5.5.
2. Improvement over each comparator has a paired-bootstrap 95% confidence interval that excludes zero in the favorable direction.
3. All three fine-tuning seeds improve over the starting checkpoint.

H2 uses the same rule with normalized WER.

H3 is descriptive unless each period contains at least 30 evaluation lines. Report counts and confidence intervals and avoid strong generalization when slices are small.

Publish results regardless of outcome. A result above published page-level baselines is not automatically a failure because the evaluation units differ.

13. Error analysis

Before viewing predictions, define these error groups:

• Character substitution
• Character deletion
• Character insertion
• Whitespace or word-boundary error
• Punctuation or numeral error
• Degraded print
• Skew or rotation
• Segmentation failure
• Mixed-language or unsupported symbol

Sample up to 50 errors using a deterministic random seed. Do not select only visually compelling cases.

14. Reporting requirements

The final report must include:

• Every seed
• Mean and standard deviation
• Confidence intervals
• Dataset and code revisions
• Hardware and runtime
• Raw and normalized metrics
• Exact evaluation-unit counts
• Exclusion counts and reasons
• Period-stratified results when valid
• Known limitations
• Deviations from this protocol

15. Licensing gate

The training dataset license is confirmed MIT. The intermediate checkpoint currently has no declared license. Training experiments may be prepared, but public redistribution of derivative weights is blocked until the checkpoint licensing position is clarified or independently reviewed.

16. Deviation log

Date	Change	Rationale	Expected impact	Confirmatory or exploratory
—	None	—	—	—