Instructions to use haijian06/Yi-1.5-6B-Chat-Agent_sft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use haijian06/Yi-1.5-6B-Chat-Agent_sft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="haijian06/Yi-1.5-6B-Chat-Agent_sft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("haijian06/Yi-1.5-6B-Chat-Agent_sft")
model = AutoModelForMultimodalLM.from_pretrained("haijian06/Yi-1.5-6B-Chat-Agent_sft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use haijian06/Yi-1.5-6B-Chat-Agent_sft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "haijian06/Yi-1.5-6B-Chat-Agent_sft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "haijian06/Yi-1.5-6B-Chat-Agent_sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/haijian06/Yi-1.5-6B-Chat-Agent_sft

SGLang

How to use haijian06/Yi-1.5-6B-Chat-Agent_sft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "haijian06/Yi-1.5-6B-Chat-Agent_sft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "haijian06/Yi-1.5-6B-Chat-Agent_sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "haijian06/Yi-1.5-6B-Chat-Agent_sft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "haijian06/Yi-1.5-6B-Chat-Agent_sft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use haijian06/Yi-1.5-6B-Chat-Agent_sft with Docker Model Runner:
```
docker model run hf.co/haijian06/Yi-1.5-6B-Chat-Agent_sft
```

haijian06/Yi-1.5-6B-Chat-Agent_sft

Overview

The haijian06/Yi-1.5-6B-Chat-Agent_sft model is an advanced conversational agent built upon the Yi-1.5-6B-Chat model. This model has been fine-tuned to enhance its capabilities in handling agent tasks and function calls, making it a versatile tool for a variety of applications.

Features

Improved Conversational Abilities: Enhanced dialogue management and natural language understanding.
Function Call Capability: Supports complex function call operations, making it suitable for automation and task handling.
High Performance: Optimized for speed and accuracy in responses.

Installation

To use this model, you need to have Python and the necessary libraries installed. You can install the required dependencies using the following commands:

pip install torch transformers

Usage

Here is a basic example of how to use the haijian06/Yi-1.5-6B-Chat-Agent_sft model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_name = "haijian06/Yi-1.5-6B-Chat-Agent_sft"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Generate a response
input_text = "Hello, how can I assist you today?"
input_ids = tokenizer.encode(input_text, return_tensors='pt')

with torch.no_grad():
    output = model.generate(input_ids, max_length=50)

response = tokenizer.decode(output[0], skip_special_tokens=True)
print(response)

Fine-Tuning

To fine-tune this model on your own dataset, follow these steps:

Prepare your dataset in a suitable format.
Use the Trainer class from the transformers library for training.

Example training script:

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir='./results',         
    num_train_epochs=3,             
    per_device_train_batch_size=4,  
    per_device_eval_batch_size=4,   
    warmup_steps=500,               
    weight_decay=0.01,              
    logging_dir='./logs',           
)

trainer = Trainer(
    model=model,                        
    args=training_args,                 
    train_dataset=train_dataset,         
    eval_dataset=eval_dataset            
)

trainer.train()

Contributing

Contributions are welcome! Please fork this repository and submit a pull request with your improvements.

License

This work is a derivative of Yi-1.5-6B by 01.AI, used under the Apache 2.0 License.

Acknowledgements

This model is built upon the Yi-1.5-6B-Chat model. Special thanks to the developers and contributors of the original model.

For more information, please visit our GitHub repository.

Downloads last month: 2

Safetensors

Model size

6B params

Tensor type

BF16