| --- |
| language: |
| - fr |
| license: apache-2.0 |
| tags: |
| - medical |
| - healthcare |
| - french |
| - checkpoints |
| - composer |
| - doctobert |
| --- |
| |
| # DoctoBERT-fr — Training Checkpoints |
|
|
| <center> |
| <img src="assets/logo_doctobert.png" width="100%" alt="DoctoBERT"> |
| </center> |
|
|
| <p align="center"> |
| <a href="https://huggingface.co/blog/bofenghuang/doctobert-fr-release">🤗 Blog</a> | |
| <a href="https://arxiv.org/abs/2606.22079">📄 Paper</a> | |
| <a href="https://github.com/doctolib-lab/doctobert">💻 Code</a> | |
| <a href="https://huggingface.co/collections/doctolib-lab/finemed-fr">🌐 FineMed</a> | |
| <a href="https://huggingface.co/collections/doctolib-lab/doctobert-fr">🩺 DoctoBERT</a> |
| </p> |
|
|
| ## 📚 Introduction |
|
|
| Per-stage **[MosaicML Composer](https://github.com/mosaicml/composer)** training-state checkpoints for the DoctoBERT-fr encoder family. |
|
|
| Each `.pt` is the full state (model, optimizer, scheduler, RNG), for resuming or analysis. DoctoBERT-fr keeps the final checkpoint of each stage; DoctoModernBERT-fr also keeps intermediate checkpoints every ~5,000 batches. |
|
|
| For ready-to-use inference models, use [DoctoBERT-fr-base](https://huggingface.co/doctolib-lab/doctobert-fr-base) and [DoctoModernBERT-fr-base](https://huggingface.co/doctolib-lab/doctomodernbert-fr-base). |
|
|
| <!-- |
| ## 📖 Citation |
|
|
| ```bibtex |
| @misc{doctobert2026, |
| title = {Where Does the Signal Live? A Web Data Recipe for Medical Encoder Pretraining}, |
| author = {Huang, Bofeng and Sun, Jacques and Bouchacourt, Diane and Barascud, Nicolas and Fogel, Fajwel}, |
| year = {2026}, |
| eprint = {2606.XXXXX}, |
| archivePrefix = {arXiv}, |
| primaryClass = {cs.CL} |
| } |
| ``` |
| --> |
|
|
| ## ⚖️ License |
|
|
| Released under Apache-2.0. Trained on FineMed-fr and FineMed-rephrased-fr, which derive from FineWeb-2 / FinePDFs (ODC-BY 1.0) and FineWiki (CC BY-SA 4.0); please attribute those upstream sources. |
|
|
| ## 🏛️ Acknowledgments |
|
|
| This work was granted access to the HPC resources of IDRIS (Jean Zay) under the allocations 2025-AD011016291 and 2026-A0200617487 made by GENCI. |
|
|
