Yale-BIDS-Chen/medpmc-multi-fig-separation-yolov10

MedPMC Multi-Panel Figure Separation Model

This repository provides the multi-panel figure separation model used in the MedPMC data curation pipeline.

The model is a YOLOv10-based object detection model trained to detect individual panels within multi-panel biomedical figures. It is intended for processing figures from biomedical literature, especially figures from PubMed Central (PMC) articles.

Model

The released checkpoint is:

model.pt

The model predicts individual panel bounding boxes and can save the corresponding panel crops.

Installation

The provided requirements.txt contains:

ultralytics
torch
opencv-python
pillow

Install PyTorch following the official instructions for the CUDA version available in your environment. Then install the remaining dependencies:

pip install -r requirements.txt

The released checkpoint was validated with the official YOLOv10 implementation from THU-MIG. To ensure compatibility with the YOLOv10 interface, install it after the requirements:

pip install --no-deps \
  git+https://github.com/THU-MIG/yolov10.git

The --no-deps option prevents this installation from replacing an already configured PyTorch and CUDA environment.

For optional programmatic checkpoint download from the Hugging Face Hub:

pip install huggingface-hub

Usage

Load a locally downloaded checkpoint

Download model.pt from this repository and place it in the working directory.

import os

from ultralytics import YOLOv10


def load_trusted_checkpoint(checkpoint_path: str) -> YOLOv10:
    """Load the trusted MedPMC full-object YOLOv10 checkpoint."""
    variable = "TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD"
    previous_value = os.environ.get(variable)

    os.environ[variable] = "1"

    try:
        model = YOLOv10(checkpoint_path)
    finally:
        if previous_value is None:
            os.environ.pop(variable, None)
        else:
            os.environ[variable] = previous_value

    return model


model = load_trusted_checkpoint("model.pt")

results = model.predict(
    source="path/to/images",
    save=True,
    save_crop=True,
    save_txt=True,
    batch=1,
    conf=0.5,
)

The input source can be a single image, a directory of images, or another input type supported by Ultralytics.

Download the checkpoint programmatically

import os

from huggingface_hub import hf_hub_download
from ultralytics import YOLOv10


def load_trusted_checkpoint(checkpoint_path: str) -> YOLOv10:
    """Load the trusted MedPMC full-object YOLOv10 checkpoint."""
    variable = "TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD"
    previous_value = os.environ.get(variable)

    os.environ[variable] = "1"

    try:
        model = YOLOv10(checkpoint_path)
    finally:
        if previous_value is None:
            os.environ.pop(variable, None)
        else:
            os.environ[variable] = previous_value

    return model


checkpoint_path = hf_hub_download(
    repo_id="Yale-BIDS-Chen/medpmc-multi-fig-separation-yolov10",
    filename="model.pt",
)

model = load_trusted_checkpoint(checkpoint_path)

results = model.predict(
    source="path/to/images",
    save=True,
    save_crop=True,
    save_txt=True,
    batch=1,
    conf=0.5,
)

PyTorch 2.6 or Later

The released model.pt checkpoint contains a serialized Ultralytics model object rather than only a tensor state dictionary.

Beginning with PyTorch 2.6, torch.load() uses weights_only=True by default. Loading this trusted full-object checkpoint therefore requires explicitly enabling the previous full-object loading behavior.

The examples above temporarily set:

TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1

only while the model is being loaded and restore the previous environment setting immediately afterward.

A warning similar to the following may be displayed during loading:

Environment variable TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD detected,
forcing weights_only=False.

This warning is expected for this checkpoint.

Only enable full-object loading for checkpoints obtained from this trusted repository. Pickle-based full-object checkpoints from untrusted sources may execute arbitrary code during loading.

Model Outputs

The model returns panel bounding boxes, confidence scores, class IDs, and optional panel crops.

results = model.predict(
    source="path/to/images",
    conf=0.5,
    batch=1,
    save=True,
    save_crop=True,
    save_txt=True,
    save_conf=True,
)

for result in results:
    print(result.boxes.data.cpu().numpy())

Each detection follows:

[x1, y1, x2, y2, confidence, class_id]

Example:

[508.76, 546.65, 722.00, 797.48, 0.9066, 0]

Saved label files use normalized YOLO coordinates:

class_id x_center y_center width height confidence

MedPMC Pipeline Post-processing

The YOLOv10 model predicts bounding boxes but does not assign final MedPMC subfigure identifiers.

In the original MedPMC pipeline, boxes were reordered using their normalized top-left coordinates. Boxes were sorted from top to bottom, adjacent boxes with a y difference no greater than 0.05 were grouped into the same row, and boxes within each row were sorted from left to right.

Citation

When using this model or the MedPMC curation pipeline, please cite:

@article{kim2026medpmc,
  title={MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models},
  author={Hyunjae Kim and Dain Kim and Pan Xiao and Serina S. Applebaum and Younjoon Chung and Xuguang Ai and Yu Yin and Roy Jiang and Yuexi Du and Yawen Wei and Yiming Kong and Tuo Guo and Zhiyuan Cao and Mengmeng Du and Yuelei Fu and Yan Hu and Rui Shi and Gui Yang and Kevin W. Jin and Yuntian Liu and Yuxuan Tian and Jonathan Marquez and Zhen Chen and Sheng Zhang and Hoifung Poon and Hua Xu and Jaewoo Kang and Qingyu Chen},
  journal={arXiv preprint arXiv:2607.07673},
  year={2026}
}

The underlying detector architecture is based on YOLOv10:

@article{wang2024yolov10,
  title={Yolov10: Real-time end-to-end object detection},
  author={Wang, Ao and Chen, Hui and Liu, Lihao and Chen, Kai and Lin, Zijia and Han, Jungong and Ding, Guiguang},
  journal={Advances in neural information processing systems},
  volume={37},
  pages={107984--108011},
  year={2024}
}