YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
Model Summary
Llama3-8B-COIG-CQIA is an instruction-tuned language model for Chinese & English users with various abilities such as roleplaying & tool-using built upon the Meta-Llama-3-8B-Instruct model.
Developed by: Wenfeng Qiu
- License: Llama-3 License
- Base Model: Meta-Llama-3-8B-Instruct
- Model Size: 8.03B
- Context length: 8K
1. Introduction
Training framework: unsloth.
Training details:
- epochs: 1
- learning rate: 2e-4
- learning rate scheduler type: linear
- warmup steps: 5
- cutoff len (i.e. context length): 2048
- global batch size: 2
- fine-tuning type: full parameters
- optimizer: adamw_8bit
2. Usage
Inference, use to llama.cpp or a UI based system like GPT4All. You can install GPT4All by going here.
Here is the example in llama.cpp.
from llama_cpp import Llama
model = Llama(
"/Your/Path/To/Llama3-8B-COIG-CQIA.Q8_0.gguf",
verbose=False,
n_gpu_layers=-1,
)
system_prompt = "You are a helpful assistant."
def generate_reponse(_model, _messages, _max_tokens=8192):
_output = _model.create_chat_completion(
_messages,
stop=["<|eot_id|>", "<|end_of_text|>"],
max_tokens=_max_tokens,
)["choices"][0]["message"]["content"]
return _output
# The following are some examples
messages = [
{
"role": "system",
"content": system_prompt,
},
{"role": "user", "content": "你是谁？"},
]
print(generate_reponse(_model=model, _messages=messages), end="\n\n\n")
- Downloads last month
- 3
8-bit