Instructions to use humarin/chatgpt_paraphraser_on_T5_base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use humarin/chatgpt_paraphraser_on_T5_base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="humarin/chatgpt_paraphraser_on_T5_base")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("humarin/chatgpt_paraphraser_on_T5_base")
model = AutoModelForSeq2SeqLM.from_pretrained("humarin/chatgpt_paraphraser_on_T5_base")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use humarin/chatgpt_paraphraser_on_T5_base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "humarin/chatgpt_paraphraser_on_T5_base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "humarin/chatgpt_paraphraser_on_T5_base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/humarin/chatgpt_paraphraser_on_T5_base

SGLang

How to use humarin/chatgpt_paraphraser_on_T5_base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "humarin/chatgpt_paraphraser_on_T5_base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "humarin/chatgpt_paraphraser_on_T5_base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "humarin/chatgpt_paraphraser_on_T5_base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "humarin/chatgpt_paraphraser_on_T5_base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use humarin/chatgpt_paraphraser_on_T5_base with Docker Model Runner:
```
docker model run hf.co/humarin/chatgpt_paraphraser_on_T5_base
```

Error using model

#16

by Amitontheweb - opened Dec 1, 2023

Discussion

Amitontheweb

Dec 1, 2023

Hello!

Facing this error today while using "chatgpt_paraphraser_on_T5_base". Please help with a resolution. Thanks.

Runtime error

Space failed. Exit code: 1. Reason: alidators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1349, in hf_hub_download
raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/user/app/app.py", line 12, in
tokenizer = AutoTokenizer.from_pretrained("humarin/chatgpt_paraphraser_on_T5_base")
File "/home/user/.local/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 716, in from_pretrained
config = AutoConfig.from_pretrained(
File "/home/user/.local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1034, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 620, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 675, in _get_config_dict
resolved_config_file = cached_file(
File "/home/user/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 469, in cached_file
raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like humarin/chatgpt_paraphraser_on_T5_base is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment