gamzadole/llama3_Alpaca_Finetune_test

Model Card for Model ID

Base Model : "beomi/Llama-3-Open-Ko-8B"

Dataset : "Bingsu/ko_alpaca_data"

LORA를 사용해 파인튜닝한 모델입니다. 하드웨어 메모리가 부족하여 alpaca dataset의 상위 10개의 데이터만 가지고 학습을 진행하였습니다.

Model Details

해당 모델을 사용해보기 위해선 basemodel의 tokenizer와 특정 instruction template을 지켜야 제대로 된 결과가 출력됩니다.

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

BASEMODEL = "beomi/Llama-3-Open-Ko-8B"
config = PeftConfig.from_pretrained("gamzadole/llama3_Alpaca_Finetune")
base_model = AutoModelForCausalLM.from_pretrained("beomi/Llama-3-Open-Ko-8B", load_in_4bit=True, device_map="auto")
model = PeftModel.from_pretrained(base_model, "gamzadole/llama3_Alpaca_Finetune")
model = model.cuda()

tokenizer = AutoTokenizer.from_pretrained(BASEMODEL)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

prompt_input_template = """아래는 작업을 설명하는 지시사항과 추가 정보를 제공하는 입력이 짝으로 구성됩니다. 이에 대한 적절한 응답을 작성해주세요.

### 지시사항:
{instruction}

### 입력:
{input}

### 응답:"""


prompt_no_input_template = """아래는 작업을 설명하는 지시사항입니다. 이에 대한 적절한 응답을 작성해주세요.

### 지시사항:
{instruction}

### 응답:"""

def generate_response(prompt, model):
    encoded_input = tokenizer(prompt,  return_tensors="pt", add_special_tokens=True)
    model_inputs = encoded_input.to('cuda')

    generated_ids = model.generate(**model_inputs, max_new_tokens=512, do_sample=True, pad_token_id=tokenizer.eos_token_id)

    decoded_output = tokenizer.batch_decode(generated_ids)

    return decoded_output[0].replace(prompt, "")

instruction = "건강을 유지하기 위한 세 가지 팁을 알려주세요."
prompt = prompt_no_input_template.format(instruction=instruction)

generate_response(prompt, model)

Model Description

• Developed by: [More Information Needed]
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Model type: [More Information Needed]
• Language(s) (NLP): [More Information Needed]
• License: [More Information Needed]
• Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Direct Use

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Training Procedure

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: [More Information Needed]

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Framework versions

• PEFT 0.11.1