File size: 2,051 Bytes
bdbe072 f4b7e6b bdbe072 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | ---
language:
- fr
license: apache-2.0
tags:
- medical
- healthcare
- french
- checkpoints
- composer
- doctobert
---
# DoctoBERT-fr — Training Checkpoints
<center>
<img src="assets/logo_doctobert.png" width="100%" alt="DoctoBERT">
</center>
<p align="center">
<a href="https://huggingface.co/blog/bofenghuang/doctobert-fr-release">🤗 Blog</a> |
<a href="https://arxiv.org/abs/2606.22079">📄 Paper</a> |
<a href="https://github.com/doctolib-lab/doctobert">💻 Code</a> |
<a href="https://huggingface.co/collections/doctolib-lab/finemed-fr">🌐 FineMed</a> |
<a href="https://huggingface.co/collections/doctolib-lab/doctobert-fr">🩺 DoctoBERT</a>
</p>
## 📚 Introduction
Per-stage **[MosaicML Composer](https://github.com/mosaicml/composer)** training-state checkpoints for the DoctoBERT-fr encoder family.
Each `.pt` is the full state (model, optimizer, scheduler, RNG), for resuming or analysis. DoctoBERT-fr keeps the final checkpoint of each stage; DoctoModernBERT-fr also keeps intermediate checkpoints every ~5,000 batches.
For ready-to-use inference models, use [DoctoBERT-fr-base](https://huggingface.co/doctolib-lab/doctobert-fr-base) and [DoctoModernBERT-fr-base](https://huggingface.co/doctolib-lab/doctomodernbert-fr-base).
<!--
## 📖 Citation
```bibtex
@misc{doctobert2026,
title = {Where Does the Signal Live? A Web Data Recipe for Medical Encoder Pretraining},
author = {Huang, Bofeng and Sun, Jacques and Bouchacourt, Diane and Barascud, Nicolas and Fogel, Fajwel},
year = {2026},
eprint = {2606.XXXXX},
archivePrefix = {arXiv},
primaryClass = {cs.CL}
}
```
-->
## ⚖️ License
Released under Apache-2.0. Trained on FineMed-fr and FineMed-rephrased-fr, which derive from FineWeb-2 / FinePDFs (ODC-BY 1.0) and FineWiki (CC BY-SA 4.0); please attribute those upstream sources.
## 🏛️ Acknowledgments
This work was granted access to the HPC resources of IDRIS (Jean Zay) under the allocations 2025-AD011016291 and 2026-A0200617487 made by GENCI.
|