Instructions to use eci-io/climategpt-13b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eci-io/climategpt-13b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="eci-io/climategpt-13b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("eci-io/climategpt-13b")
model = AutoModelForCausalLM.from_pretrained("eci-io/climategpt-13b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use eci-io/climategpt-13b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eci-io/climategpt-13b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eci-io/climategpt-13b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eci-io/climategpt-13b

SGLang

How to use eci-io/climategpt-13b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "eci-io/climategpt-13b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eci-io/climategpt-13b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "eci-io/climategpt-13b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eci-io/climategpt-13b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use eci-io/climategpt-13b with Docker Model Runner:
```
docker model run hf.co/eci-io/climategpt-13b
```

dthulke commited on Mar 22, 2024

Commit

32a05f1

1 Parent(s): d1fc2d9

adds chat template and update README

Browse files

Files changed (2) hide show

README.md +8 -2
tokenizer_config.json +2 -1

README.md CHANGED Viewed

@@ -12,6 +12,9 @@ co2_eq_emissions:
   training_type: "pre-training"
   geographical_location: "Washington, USA"
   hardware_used: "8x NVIDIA H100 HBM"
 ---
 # ClimateGPT-13B
@@ -32,7 +35,7 @@ The model is designed to be used together with retrieval augmentation to extend
 - **Context length:** 4K tokens
 - **Input:** Text-only data
 - **Output:** Model generates text only
-- **Paper:** [Download](https://shareddatastgacct.blob.core.windows.net/shared-data/climategpt-v1-publication.pdf)
 - **Website:** [eci.io](https://eci.io)
 ## Uses
@@ -71,7 +74,7 @@ The model was trained using ChatML so the following format should be followed wh
 ## Evaluation
-Detailed evaluation results are presented in our [paper](https://shareddatastgacct.blob.core.windows.net/shared-data/climategpt-v1-publication.pdf) on our model card website: [eci.io/model-card](https://eci.io/model-card)
 ## Environmental Impact
 - **Hardware Type:** 8x NVIDIA H100 HBM
@@ -91,5 +94,8 @@ If you find ClimateGPT is useful in your work, please cite it with:
       title={ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change},
       author={David Thulke and Yingbo Gao and Petrus Pelser and Rein Brune and Rricha Jalota and Floris Fok and Michael Ramos and Ian van Wyk and Abdallah Nasir and Hayden Goldstein and Taylor Tragemann and Katie Nguyen and Ariana Fowler and Andrew Stanco and Jon Gabriel and Jordan Taylor and Dean Moro and Evgenii Tsymbalov and Juliette de Waal and Evgeny Matusov and Mudar Yaghi and Mohammad Shihadah and Hermann Ney and Christian Dugast and Jonathan Dotan and Daniel Erasmus},
       year={2024},
 }
 ```

   training_type: "pre-training"
   geographical_location: "Washington, USA"
   hardware_used: "8x NVIDIA H100 HBM"
+license: other
+license_name: climategpt-community-license
+license_link: LICENSE.txt
 ---
 # ClimateGPT-13B
 - **Context length:** 4K tokens
 - **Input:** Text-only data
 - **Output:** Model generates text only
+- **Paper:** [arXiv:2401.09646](https://arxiv.org/abs/2401.09646)
 - **Website:** [eci.io](https://eci.io)
 ## Uses
 ## Evaluation
+Detailed evaluation results are presented in our [paper](https://arxiv.org/abs/2401.09646) on our model card website: [eci.io/model-card](https://eci.io/model-card)
 ## Environmental Impact
 - **Hardware Type:** 8x NVIDIA H100 HBM
       title={ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change},
       author={David Thulke and Yingbo Gao and Petrus Pelser and Rein Brune and Rricha Jalota and Floris Fok and Michael Ramos and Ian van Wyk and Abdallah Nasir and Hayden Goldstein and Taylor Tragemann and Katie Nguyen and Ariana Fowler and Andrew Stanco and Jon Gabriel and Jordan Taylor and Dean Moro and Evgenii Tsymbalov and Juliette de Waal and Evgeny Matusov and Mudar Yaghi and Mohammad Shihadah and Hermann Ney and Christian Dugast and Jonathan Dotan and Daniel Erasmus},
       year={2024},
+      eprint={2401.09646},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG}
 }
 ```

tokenizer_config.json CHANGED Viewed

@@ -145,5 +145,6 @@
   "spaces_between_special_tokens": false,
   "tokenizer_class": "LlamaTokenizer",
   "unk_token": "<unk>",
-  "use_default_system_prompt": false
 }

   "spaces_between_special_tokens": false,
   "tokenizer_class": "LlamaTokenizer",
   "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"
 }