YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
Korean Classical Chinese Punctuation Prediction Model
한국 고전한문 구두점 예측 모델
License: CC BY-NC-SA 4.0 · GitHub · DOI
Deep learning model for automatic punctuation prediction in Korean classical Chinese texts. Achieving F1 score of 0.9110 (v2.5) on 7 punctuation marks.
딥러닝 기반 한국 고전 한문 텍스트 자동 구두점 예측 모델입니다. 7종의 구두점을 F1 Score 0.9110 (v2.5)의 정확도로 예측합니다.
Model Versions
| Version | Base Model | F1 Score | Note |
|---|---|---|---|
| v2.5 | SikuRoBERTa (SIKU-BERT/sikuroberta) | 0.9110 | Latest Recommended (Korean Cultural Studies 111) |
| v2.0 | SikuRoBERTa (SIKU-BERT/sikuroberta) | 0.9110 | Reference Model (Korean Cultural Studies 111) |
| v1 | Chinese-RoBERTa (hfl/chinese-roberta-wwm-ext) | 0.9050 | Korean Journal of History 100 |
🔄 v2.0 → v2.5 Update
Version 2.5 is an incremental update of the v2.0 release, focusing on training stability and reproducibility. The model architecture, task definition, and data scheme remain identical to v2.0.
Name: Korean Classical Chinese Punctuation Prediction Model v2.5 Base Model: SIKU-BERT/sikuroberta Task: Multi-label Token Classification Labels: 7 punctuation marks (, 。 · ? ! 《 》)
v2.5 Updates:
- Removal of PatternAwareLoss
- Adoption of standard BCEWithLogitsLoss for multi-label classification
- Masked-mean loss computation based on token attention masks
- Simplified training policy prioritizing stability and reproducibility
v2.0 (PatternAwareLoss) and v2.5 (standard BCE) showed equivalent performance, with no significant difference observed apart from a slight recall advantage of v2.0 on the closing bracket (》). Given sufficient training data and domain-specific pretraining, the structural pattern of consecutive punctuation marks converges probabilistically during training even without an explicit loss-function rule. Accordingly, v2.5 — the simpler training structure without additional assumptions — was adopted as the final model. (Korean Cultural Studies, vol. 111)
Validation - F1: 0.9110 (v2.5) Per punctuation:
- ?: F1=0.9485, P=0.9435, R=0.9535
- ,: F1=0.9182, P=0.9181, R=0.9183
- 。: F1=0.8880, P=0.9103, R=0.8668
- ·: F1=0.8866, P=0.9133, R=0.8615
- 《: F1=0.7805, P=0.8178, R=0.7465
- 》: F1=0.7735, P=0.8108, R=0.7395
- !: F1=0.7015, P=0.8437, R=0.6003
Model Description
Architecture: Custom Multi-label Token Classification
- BERT Encoder
- Dropout Layer (rate: 0.1)
- Linear Classification Head (768 → 7)
- 7 independent binary classifiers (one per punctuation type)
- Sigmoid activation with threshold 0.5
Important Note: This model uses a custom architecture, not the standard AutoModelForTokenClassification. For full inference capabilities, use the custom PunctuationPredictor class from the GitHub repository.
Task: Multi-label Token Classification
- Each character token can have 0 or more punctuation marks following it
- 7 punctuation types: , (comma), 。 (period), · (middle dot), ? (question mark), ! (exclamation mark), 《 (left title bracket), 》 (right title bracket)
Training Data: ~3.4 million training samples from Korean classical Chinese historical texts
- Source documents: ~420 million characters
- 8 document genres: chronicles, registers, diaries, literary collections, laws, geography, biographies, etc.
Performance
Overall Performance
| Version | F1 Score | Precision | Recall |
|---|---|---|---|
| v2.5 | 0.9110 | - | - |
| v2.0 | 0.9110 | 0.9117 | 0.9103 |
| v1 | 0.9050 | 0.9057 | 0.9043 |
Per-Punctuation Performance
| Punctuation | v1 F1 | v2.0 F1 | v2.5 F1 | v1 Precision | v2.0 Precision | v2.5 Precision | v1 Recall | v2.0 Recall | v2.5 Recall |
|---|---|---|---|---|---|---|---|---|---|
| ? | 0.9436 | 0.9486 | 0.9485 | 0.9419 | 0.9437 | 0.9435 | 0.9454 | 0.9536 | 0.9535 |
| , | 0.9127 | 0.9182 | 0.9182 | 0.9130 | 0.9175 | 0.9181 | 0.9124 | 0.9190 | 0.9183 |
| 。 | 0.8818 | 0.8879 | 0.8880 | 0.9054 | 0.9109 | 0.9103 | 0.8594 | 0.8661 | 0.8668 |
| · | 0.8759 | 0.8864 | 0.8866 | 0.9157 | 0.9134 | 0.9133 | 0.8394 | 0.8609 | 0.8615 |
| 《 | 0.7367 | 0.7817 | 0.7805 | 0.8155 | 0.8204 | 0.8178 | 0.6717 | 0.7465 | 0.7465 |
| 》 | 0.7311 | 0.7764 | 0.7735 | 0.8024 | 0.8126 | 0.8108 | 0.6713 | 0.7432 | 0.7395 |
| ! | 0.6369 | 0.7007 | 0.7015 | 0.8114 | 0.8399 | 0.8437 | 0.5241 | 0.6011 | 0.6003 |
Domain-specific Performance (v1.0 → v2.5)
| Domain | v1.0 F1 | v2.5 F1 | Δ F1 | Data Size (chars) |
|---|---|---|---|---|
| Geography (지리지) | 0.9116 | 0.9385 | +0.0269 | 501,942 |
| Chronicles (연대기) | 0.9162 | 0.9199 | +0.0037 | 30,682,976 |
| Registers (등록) | 0.9114 | 0.9257 | +0.0143 | 1,896,232 |
| Biographies (전기) | 0.8606 | 0.8953 | +0.0347 | 591,983 |
| Laws (법령) | 0.8485 | 0.8799 | +0.0314 | 907,893 |
| Literary Collections (문집) | 0.8354 | 0.8693 | +0.0339 | 1,885,268 |
| Diaries (일기) | 0.8229 | 0.8515 | +0.0286 | 544,768 |
Chronicles, already above 0.91 in v1.0, show only a marginal gain, whereas the relatively lower-performing types (Diaries, Literary Collections, Laws, Biographies) show larger gains. See Korean Cultural Studies, vol. 111, for detailed analysis.
External Validation (Unseen Data, v1)
| Dataset | F1 Score | Data Size (chars) | Source |
|---|---|---|---|
| 한국문집총간 (Korean Literary Collections) | 0.8784 | 166,763,095 | ITKC Database |
| 일성록 (Ilseongrok) | 0.9065 | 6,743,710 | Kyujanggak Institute for Korean Studies |
How to Use
⚠️ Important: Custom Model Architecture
This model uses a custom architecture and cannot be loaded with standard Hugging Face methods. You must use the specialized inference code from the GitHub repository.
Installation
# Clone repository
git clone https://github.com/yachagye/korean-classical-chinese-punctuation
cd korean-classical-chinese-punctuation
# Install dependencies
pip install torch transformers pytorch-lightning
Quick Start (Requires GitHub Repository Code)
from 구두점7_추론모델 import PunctuationPredictor
# Load model (requires .ckpt checkpoint file)
predictor = PunctuationPredictor(
checkpoint_path="path/to/checkpoint.ckpt",
device="cuda" # or "cpu"
)
# Predict punctuation
text = "太祖康獻大王姓李諱成桂字君晉古諱旦號松軒"
result = predictor.predict(text)
print(result)
# Output: 太祖康獻大王, 姓李, 諱成桂, 字君晉。古諱旦, 號松軒。
Processing Files
TXT files:
python 구두점7_지정_txt.py --checkpoint checkpoint.ckpt --input your_file.txt
CSV files:
python 구두점7_지정_csv.py --checkpoint checkpoint.ckpt --input your_file.csv
GUI Application (Windows)
Standalone Windows executables are available on Google Drive - no Python installation required.
- v2.5 (Recommended): 한문구두점추론_v2.5.zip
- v2.0: 한문구두점추론_v2.0.zip
- v1.0: 한문구두점추론_v1.0.zip
Training Details
Training Hyperparameters
| Parameter | Value |
|---|---|
| GPU | L40S 48GB |
| Batch Size (per GPU) | 80 |
| Gradient Accumulation Steps | 2 |
| Effective Batch Size | 160 |
| Learning Rate | 2e-5 |
| Learning Rate Schedule | Linear warmup + decay |
| Warmup Ratio | 0.1 |
| Training Epochs | 3 |
| Mixed Precision | bf16-mixed |
| Optimizer | AdamW |
| Weight Decay | 0.01 |
| Gradient Clipping | 1.0 |
| Dropout Rate | 0.1 |
| Max Sequence Length | 512 tokens |
| Framework | PyTorch Lightning |
Training Data Processing
Sliding Window Approach:
- Window size: 512 tokens
- Overlap: 50 tokens
Data Split:
- Training: ~90% (approx. 3.4M samples)
- Validation: ~10%
- Random seed: 42
Data Format (JSONL):
{
"c": "太祖康獻大王姓李諱成桂字君晉",
"l": [[],[],[],[],[1],[],[],[],[],[],[0,1],...],
"n": 15
}
Training Data Sources
Primary Sources:
- National Institute of Korean History Database (국사편찬위원회)
- Korean Classics Database (한국고전종합DB)
- Jangseogak Digital Archive (디지털장서각)
Preprocessing Pipeline (6 stages):
- Remove non-Chinese text (Korean Hangul, Japanese Kana, numbers)
- Standardize punctuation marks (26 types → unified format)
- Keep only Chinese characters and punctuation
- Reduce to 7 primary punctuation types
- Remove lines without punctuation
- Remove duplicate consecutive punctuation
Limitations
- Paired Brackets: Book title markers (《》) show lower performance (F1 ~0.78)
- Rare Punctuation: Exclamation mark (!) shows lowest recall (0.6003) due to scarcity
- Context Window: 512-token limit (longer texts processed via sliding window)
- Domain Bias: Trained primarily on official records
Intended Uses
✅ Recommended Uses
- Digital humanities research
- Historical document digitization
- Academic study of classical Chinese texts
- Educational tools
- Public digital archives
- Cultural heritage preservation
❌ Not Recommended
- Commercial products without separate licensing
- Critical scholarly editions without expert review
- Real-time applications requiring 100% accuracy
Disclaimer: Predictions are not perfect. For important scholarly publications, always verify with domain experts.
Citation
If you use this model in your research, please cite:
APA Style:
Yang, J. (2025). Development and Application of a Deep Learning–Based Model
for Automated Punctuation Inference in Korean Classical Chinese.
The Korean Journal of History (Yoksahak Yongu), 100, 267-297.
https://doi.org/10.37924/JSSW.100.9
Yang, J. (2026). Conventions of Interpretation: Improving a Punctuation
Prediction Model for Korean Classical Chinese. Korean Cultural Studies,
111, 7-29. https://doi.org/10.17948/kcs.2026..111.7
BibTeX:
@article{yang2025punctuation,
title={Development and Application of a Deep Learning--Based Model for Automated Punctuation Inference in Korean Classical Chinese},
author={Yang, Junghyun},
journal={The Korean Journal of History (Yoksahak Yongu)},
volume={100},
pages={267--297},
year={2025},
publisher={Honam Historical Society},
doi={10.37924/JSSW.100.9}
}
@article{yang2026interpretation,
title={Conventions of Interpretation: Improving a Punctuation Prediction Model for Korean Classical Chinese},
author={Yang, Junghyun},
journal={Korean Cultural Studies},
volume={111},
pages={7--29},
year={2026},
publisher={Research Institute of Korean Studies, Korea University},
doi={10.17948/kcs.2026..111.7}
}
Paper Information:
v1.0 (Prior Study)
- Journal: 역사학연구 (The Korean Journal of History)
- Volume: 100 / Pages: 267-297
- Publication Date: November 30, 2025
- Publisher: 호남사학회 (Honam Historical Society)
- DOI: 10.37924/JSSW.100.9
v2.0–v2.5 (Improvement Study)
- Journal: 민족문화연구 (Korean Cultural Studies)
- Volume: 111 / Pages: 7-29
- Publication Year: 2026
- Publisher: 고려대학교 민족문화연구원 (Research Institute of Korean Studies, Korea University)
- DOI: 10.17948/kcs.2026..111.7
License
CC BY-NC-SA 4.0 (Creative Commons Attribution-NonCommercial-ShareAlike 4.0)
✅ Permitted:
- Academic research and publication
- Educational use
- Non-commercial projects
- Modification and redistribution (with same license)
❌ Restricted:
- Commercial use without separate agreement
- Proprietary derivatives
📧 Commercial Licensing: yachagye@naver.com
Full license: https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode
Links
- GitHub Repository: https://github.com/yachagye/korean-classical-chinese-punctuation
- Model & Data Downloads: Google Drive
- Paper (DOI, v1.0): 10.37924/JSSW.100.9
- Paper (DOI, v2.0–v2.5): 10.17948/kcs.2026..111.7
- Issues: https://github.com/yachagye/korean-classical-chinese-punctuation/issues
Contact
- Developer: Junghyun Yang (양정현)
- Email: yachagye@naver.com
- Commercial Inquiries: yachagye@naver.com