FlowerTune Lora Collection
Collection
12 items • Updated
How to use zjudai/flowertune-medical-lora-llama-3.1-8b-instruct with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
model = PeftModel.from_pretrained(base_model, "zjudai/flowertune-medical-lora-llama-3.1-8b-instruct")This is a LoRA adapter for meta-llama/Llama-3.1-8B-Instruct fine-tuned with Flower federated learning framework on medical NLP datasets.
This model is a LoRA adapter fine-tuned on meta-llama/Llama-3.1-8B-Instruct using the Flower federated learning framework. It was trained on medical NLP datasets through distributed learning. The model achieved scores of PubMedQA: 0.5994, MedMCQA: 0.6940, MedQA: 0.5994, CareQA: 0.5774, with an average score of 0.6175.
Base model
meta-llama/Llama-3.1-8B