---
library_name: transformers
license: cc-by-sa-4.0
language:
- en
- ja
base_model:
- Qwen/Qwen2.5-7B
tags:
- pharmacy
- biology
- chemistry
- medical
---

# JPharmatron-7B-base-v3

<!-- Provide a quick summary of what the model is/does. -->

JPharmatron is a 7B large language model designed for pharmaceutical applications and researches.

The training data consists of 10B tokens in total.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

The JPharmatron-7B-base-v3 is a continually pretrained model based on Qwen2.5-7B using 2B tokens from Japanese datasets & 8B tokens from English medical papers (without deduplication).

- **Developed by:** EQUES Inc.
- **Funded by [optional]:** [GENIAC Project](https://www.meti.go.jp/policy/mono_info_service/geniac/index.html)
- **Model type:** Causal decoder-only
- **Language(s) (NLP):** Japanese, English
- **License:** CC-BY-SA-4.0

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** https://github.com/EQUES-Inc/pharma-LLM-eval
- **Paper [optional]:** [A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP](https://arxiv.org/abs/2505.16661)

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

This model has not undergone any post-training including instruction fine-tuning. Therefore, direct use of this model for downstream tasks is not recommended. Also, it is not validated for medical use or any other risk-sensitive use.


## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**This paper has been accepted to IJCNLP-AACL 2025.**

**BibTeX:**

```
@inproceedings{ono-etal-2025-japanese,
    title = "A {J}apanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical {NLP}",
    author = "Ono, Shinnosuke  and
      Sukeda, Issey  and
      Fujii, Takuro  and
      Buma, Kosei  and
      Sasaki, Shunsuke",
    editor = "Inui, Kentaro  and
      Sakti, Sakriani  and
      Wang, Haofen  and
      Wong, Derek F.  and
      Bhattacharyya, Pushpak  and
      Banerjee, Biplab  and
      Ekbal, Asif  and
      Chakraborty, Tanmoy  and
      Singh, Dhirendra Pratap",
    booktitle = "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics",
    month = dec,
    year = "2025",
    address = "Mumbai, India",
    publisher = "The Asian Federation of Natural Language Processing and The Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.ijcnlp-long.72/",
    pages = "1316--1332",
    ISBN = "979-8-89176-298-5",
    abstract = "We present **JPharmatron**, a Japanese domain-specific large language model (LLM) for the pharmaceutical field, developed through continual pre-training on two billion Japanese pharmaceutical tokens and eight billion English biomedical tokens. For rigorous evaluation, we introduce **JPharmaBench**, a benchmark suite consisting of three new benchmarks: YakugakuQA, based on national pharmacist licensing exams; NayoseQA, which tests cross-lingual synonym and terminology normalization; and SogoCheck, a novel task involving cross-document consistency checking.We evaluate our model against open-source medical LLMs and commercial models, including GPT-4o. Experimental results show that **JPharmatron** outperforms existing open models and achieves competitive performance with commercial ones.Interestingly, even GPT-4o performs poorly on SogoCheck, suggesting that cross-sentence consistency reasoning remains an open challenge.**JPharmatron** enables secure and local model deployment for pharmaceutical tasks, where privacy and legal constraints limit the use of closed models. Besides, **JPharmaBench** offers a reproducible framework for evaluating Japanese pharmaceutical natural language processing. Together, they demonstrate the feasibility of practical and cost-efficient language models for Japanese healthcare and pharmaceutical sectors.Our model, codes, and datasets are available on HuggingFace: https://huggingface.co/collections/EQUES/jpharmatron and https://huggingface.co/collections/EQUES/jpharmabench."
}
```

## More Information [optional]

See our conference paper: [A Japanese Language Model and Three New Evaluation Benchmarks for Pharmaceutical NLP](https://aclanthology.org/2025.ijcnlp-long.72.pdf).

## Model Card Authors [optional]

[@shinnosukeono](https://shinnosukeono.github.io/)