Instructions to use abacusai/Smaug-72B-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use abacusai/Smaug-72B-v0.1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="abacusai/Smaug-72B-v0.1")# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("abacusai/Smaug-72B-v0.1") model = AutoModelForMultimodalLM.from_pretrained("abacusai/Smaug-72B-v0.1") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use abacusai/Smaug-72B-v0.1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "abacusai/Smaug-72B-v0.1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abacusai/Smaug-72B-v0.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/abacusai/Smaug-72B-v0.1
- SGLang
How to use abacusai/Smaug-72B-v0.1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "abacusai/Smaug-72B-v0.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abacusai/Smaug-72B-v0.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "abacusai/Smaug-72B-v0.1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "abacusai/Smaug-72B-v0.1", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use abacusai/Smaug-72B-v0.1 with Docker Model Runner:
docker model run hf.co/abacusai/Smaug-72B-v0.1
Adding the Open Portuguese LLM Leaderboard Evaluation Results
#30 opened over 1 year ago
by
leaderboard-pt-pr-bot
Adding Evaluation Results
#29 opened almost 2 years ago
by
leaderboard-pr-bot
Request: DOIOkan
2
#28 opened about 2 years ago
by
Okanx52
LMSYS Leaderboard? I want human evaluations:)
π 1
#27 opened about 2 years ago
by
LordTwave
Request: DOI
1
#24 opened about 2 years ago
by
Seregon501
Deploy on Azure 4 x A100 (80 GB) got hang
#23 opened about 2 years ago
by
hugging-face-infrax
2x GPU but only one is being used
1
#22 opened over 2 years ago
by
ecaglar
What is GPU requirements to load and run the model?
#21 opened over 2 years ago
by
ecaglar
Adding Evaluation Results
#20 opened over 2 years ago
by
leaderboard-pr-bot
bad output of model
4
#19 opened over 2 years ago
by
coldanimal
Questions about architecture (+ LoRA)
2
#16 opened over 2 years ago
by
alex0dd
Fine-tune for Qwen1.5
π 8
2
#14 opened over 2 years ago
by
TNTOutburst
how to get tokenizer.model
#13 opened over 2 years ago
by
waleking
Smaug - Japanese Language Support
2
#11 opened over 2 years ago
by
msmmpts
LLAMA Architecture
#10 opened over 2 years ago
by
arhanovich
AlpacaEval or LMSYS eval
π€π 3
1
#9 opened over 2 years ago
by
jhartman
Great name!
β€οΈ 5
#8 opened over 2 years ago
by
bkieser
we need Smaug-14B-v0.1-q4_k_m.gguf (14B) (i love u)
π 8
2
#7 opened over 2 years ago
by
windkkk
which chat template should we use?
4
#6 opened over 2 years ago
by
wyxwangmed
Nice
π€ 6
3
#5 opened over 2 years ago
by
Kquant03
exl2 version please
1
#4 opened over 2 years ago
by
MarxistLeninist
GGUF version
β€οΈπ 11
8
#2 opened over 2 years ago
by
johnnnna
Congratulations!
π 5
10
#1 opened over 2 years ago
by
TomGrc