Instructions to use openaccess-ai-collective/hippogriff-30b-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openaccess-ai-collective/hippogriff-30b-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openaccess-ai-collective/hippogriff-30b-chat")

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("openaccess-ai-collective/hippogriff-30b-chat")
model = AutoModelForMultimodalLM.from_pretrained("openaccess-ai-collective/hippogriff-30b-chat")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use openaccess-ai-collective/hippogriff-30b-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openaccess-ai-collective/hippogriff-30b-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openaccess-ai-collective/hippogriff-30b-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/openaccess-ai-collective/hippogriff-30b-chat

SGLang

How to use openaccess-ai-collective/hippogriff-30b-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openaccess-ai-collective/hippogriff-30b-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openaccess-ai-collective/hippogriff-30b-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openaccess-ai-collective/hippogriff-30b-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openaccess-ai-collective/hippogriff-30b-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use openaccess-ai-collective/hippogriff-30b-chat with Docker Model Runner:
```
docker model run hf.co/openaccess-ai-collective/hippogriff-30b-chat
```

Will you consider releasing a public dataset?

by Jackdiy - opened May 31, 2023

Discussion

Jackdiy

May 31, 2023

Here's the thing, I've noticed that from Mega to Manticore, and now to Hippogriff, it seems like you all have been using the Pygmalion dataset. The open-source community has probably also realized that in order to achieve better and more open-ended role-playing effects, it's not necessarily required to align with datasets like Alpaca and Vicuna that resemble GPT more. Instead, we should lean towards Pygmalion.

If you consider releasing datasets like Pygmalion and hellaswag (updated with 30K+ rows), it should encourage the open-source community to use Falcon, Guanaco, RedPajama, BLOOM, and other tools to train better models based on Pygmalion.

winglian

Open Access AI Collective org May 31, 2023

Unfortunately I'm bound by oath not to release the pygmalion dataset. The hellaswag dataset I'm using is here: https://huggingface.co/datasets/winglian/evals/blob/main/hellaswag/hellaswag.jsonl

HDiffusion

May 31, 2023

•

edited May 31, 2023

releasing datasets like Pygmalion

From what I've heard from one of the people involved with the project, the reason they don't release it is because it contains a lot of data that might be upsetting to some people. If you actually intend to use it for training and have trained models in the past you can probably reach out to one of the members for a copy.

Jackdiy

Jun 1, 2023

Thank you both for your patient explanation and sharing. I will try to contact the Pygmalion team.

winglian changed discussion status to closed Jun 12, 2023

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment