| ---
|
| license: llama3
|
| datasets:
|
| - TsinghuaC3I/UltraMedical
|
| - TsinghuaC3I/UltraMedical-Preference
|
| language:
|
| - en
|
| base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
| ---
|
| <div align="center"> |
| <h1> |
| UltraMedical: Building Specialized Generalists in Biomedicine. |
| </h1> |
| </div> |
|
|
| <p align="center"> |
| <a href="https://huggingface.co/datasets/TsinghuaC3I/UltraMedical">SFT Dataset</a> • |
| <a href="https://huggingface.co/datasets/TsinghuaC3I/UltraMedical-Preference">Pref Dataset</a> • |
| <a href="https://huggingface.co/collections/TsinghuaC3I/ultramedical-66d4076bad293ffc4bc41327">Collection</a> • |
| <a href="https://github.com/TsinghuaC3I/UltraMedical">GitHub Code</a> • |
| <a href="https://arxiv.org/abs/2406.03949">Paper</a> |
| </p> |
|
|
| Llama-3.1-8B-UltraMedical is an open-access large language model (LLM) specialized in biomedicine. Developed by the Tsinghua C3I Lab, this model aims to enhance medical examination access, literature comprehension, and clinical knowledge. |
|
|
| Building on the foundation of Meta's Llama-3.1-8B, Llama-3.1-8B-UltraMedical is trained on our [UltraMedical](https://github.com/TsinghuaC3I/UltraMedical) collection with supervised fine-tuning (SFT), iterative preference learning (like DPO and KTO). The UltraMedical collection is a large-scale, high-quality dataset of biomedical instructions, comprising 410,000 synthetic and manually curated samples, along with more than 100,000 preference data. |
|
|
|
|
| ### Evaluations |
|
|
| | Model | MultiMedQA | MMLU | GPQA | Alpaca Eval 2 (LC/WR) | |
| | ----- | ---------- | ---- | ---- | ------------- | |
| | Llama-3.1-8B-Instruct | 71.38 | 73.00 | 30.40 | 20.90/21.80 | |
| | Llama-3.1-8B-UltraMedical | 76.82 | 72.96 | 34.82 | 41.22/48.20 | |
|
|
| ### Inference with vLLM |
|
|
| ```python |
| from transformers import AutoTokenizer |
| from vllm import LLM, SamplingParams |
| |
| llm = LLM(model="TsinghuaC3I/Llama-3.1-8B-UltraMedical", trust_remote_code=True) |
| tokenizer = AutoTokenizer.from_pretrained("TsinghuaC3I/Llama-3.1-8B-UltraMedical") |
| sampling_params = SamplingParams(temperature=0.7, top_p=0.9, max_tokens=1024, stop=["<|eot_id|>"]) |
| |
| messages = [ |
| {"role": "user", "content": """The question format used in the above input examples。"""}, |
| ] |
| prompts = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| print(prompts[0]) |
| """ |
| <|begin_of_text|><|start_header_id|>user<|end_header_id|> |
| |
| {question}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
| |
| """ |
| |
| outputs = llm.generate(prompts=prompts, sampling_params=sampling_params) |
| print(outputs[0].outputs[0].text) |
| ``` |
|
|
|
|
| ### Citation |
|
|
| ``` |
| @misc{zhang2024ultramedical, |
| title={UltraMedical: Building Specialized Generalists in Biomedicine}, |
| author={Kaiyan Zhang and Sihang Zeng and Ermo Hua and Ning Ding and Zhang-Ren Chen and Zhiyuan Ma and Haoxin Li and Ganqu Cui and Biqing Qi and Xuekai Zhu and Xingtai Lv and Hu Jinfang and Zhiyuan Liu and Bowen Zhou}, |
| year={2024}, |
| eprint={2406.03949}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CL} |
| } |
| ``` |
