| --- |
| library_name: transformers |
| tags: |
| - Japanese |
| - Fine |
| - tuning |
| license: apache-2.0 |
| --- |
| |
| # Model Card for Model ID |
| Taccha/llm-jp-3-13b-finetune |
|
|
| ## Model Details |
|
|
| llm-jp-3 1.8B, 3.7B, 13BのsnapshotをBaseに |
| 関根聡ら. ichikara-instruction: LLMのための日本語インストラクションデータの構築. |
| を用いてSFTを行ったモデルです |
|
|
| ### Model Description |
|
|
|
|
| This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
|
|
| - **Developed by:** Taccha |
| - **Model type:** [More Information Needed] |
| - **Language(s) (NLP):** En |
|
|
| ## Uses |
|
|
| <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
|
|
| ### How to Get Started with the Model |
|
|
| Hugging Faceにアップロードしたモデルを用いてELYZA-tasks-100-TVの出力を得るためのコードです。 |
| こちらはLoRA_template このコードで生成されたjsonlファイルは課題の成果として提出可能なフォーマットになっております。 |
| |
| !pip install -U bitsandbytes |
| !pip install -U transformers |
| !pip install -U accelerate |
| !pip install -U datasets |
| !pip install -U peft |
| |
| # notebookでインタラクティブな表示を可能とする（ただし、うまく動かない場合あり） |
| !pip install ipywidgets --upgrade |
| |
| |
| from transformers import ( |
| AutoModelForCausalLM, |
| AutoTokenizer, |
| BitsAndBytesConfig, |
| ) |
| from peft import PeftModel |
| import torch |
| from tqdm import tqdm |
| import json |
| |
| # Hugging Faceで取得したTokenをこちらに貼る。 |
| HF_TOKEN = "Your－Token” |
|
|
| # ベースとなるモデルと学習したLoRAのアダプタ。 |
|
|
| # base_ |
| model_id = "llm-jp/llm-jp-3-13b" |
| adapter_id = "Taccha/llm-jp-3-13b-finetune" # こちらにアップロードしたHugging FaceのIDを指定してください。 |
|
|
| # QLoRA config |
| bnb_config = BitsAndBytesConfig( |
| load_in_4bit=True, |
| bnb_4bit_quant_type="nf4", |
| bnb_4bit_compute_dtype=torch.bfloat16, |
| ) |
| |
| # Load model |
| model = AutoModelForCausalLM.from_pretrained( |
| model_id, |
| quantization_config=bnb_config, |
| device_map="auto", |
| token = HF_TOKEN |
| ) |
| |
| # Load tokenizer |
| tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True, token = HF_TOKEN) |
| |
| |
| # 元のモデルにLoRAのアダプタを統合。 |
| model = PeftModel.from_pretrained(model, adapter_id, token = HF_TOKEN) |
|
|
| # データセットの読み込み。 |
| # omnicampusの開発環境では、左にタスクのjsonlをドラッグアンドドロップしてから実行。 |
| datasets = [] |
| with open("./elyza-tasks-100-TV_0.jsonl", "r") as f: |
| item = "" |
| for line in f: |
| line = line.strip() |
| item += line |
| if item.endswith("}"): |
| datasets.append(json.loads(item)) |
| item = "" |
| |
| # llmjp |
| results = [] |
| for data in tqdm(datasets): |
| |
| input = data["input"] |
| |
| prompt = f"""### 指示 |
| {input} |
| ### 回答 |
| """ |
| |
| tokenized_input = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt").to(model.device) |
| attention_mask = torch.ones_like(tokenized_input) |
| with torch.no_grad(): |
| outputs = model.generate( |
| tokenized_input, |
| attention_mask=attention_mask, |
| max_new_tokens=100, |
| do_sample=False, |
| repetition_penalty=1.2, |
| pad_token_id=tokenizer.eos_token_id |
| )[0] |
| output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True) |
| |
| results.append({"task_id": data["task_id"], "input": input, "output": output}) |
|
|
| # jsolの生成 |
| import re |
| jsonl_id = re.sub(".*/", "", adapter_id) |
| with open(f"./{jsonl_id}-outputs.jsonl", 'w', encoding='utf-8') as f: |
| for result in results: |
| json.dump(result, f, ensure_ascii=False) # ensure_ascii=False for handling non-ASCII characters |
| f.write('\n') |
