Instructions to use ibivibiv/aegolius-acadicus-v1-30b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ibivibiv/aegolius-acadicus-v1-30b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ibivibiv/aegolius-acadicus-v1-30b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ibivibiv/aegolius-acadicus-v1-30b")
model = AutoModelForCausalLM.from_pretrained("ibivibiv/aegolius-acadicus-v1-30b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ibivibiv/aegolius-acadicus-v1-30b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ibivibiv/aegolius-acadicus-v1-30b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibivibiv/aegolius-acadicus-v1-30b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ibivibiv/aegolius-acadicus-v1-30b

SGLang

How to use ibivibiv/aegolius-acadicus-v1-30b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ibivibiv/aegolius-acadicus-v1-30b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibivibiv/aegolius-acadicus-v1-30b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ibivibiv/aegolius-acadicus-v1-30b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ibivibiv/aegolius-acadicus-v1-30b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ibivibiv/aegolius-acadicus-v1-30b with Docker Model Runner:
```
docker model run hf.co/ibivibiv/aegolius-acadicus-v1-30b
```

ibivibiv commited on Jan 25, 2024

Commit

a6f8c47

verified ·

1 Parent(s): 5c2c39a

Update README.md

Browse files

Files changed (1) hide show

README.md +66 -1

README.md CHANGED Viewed

@@ -48,7 +48,72 @@ print(text)
 # Benchmark Scores
-pending
 ## Citations

 # Benchmark Scores
+| Test Name                                            | Accuracy             |
+|------------------------------------------------------|----------------------|
+| all                                                  | 0.6566791267920726   |
+|arc:challenge                          | 0.7005119453924915   |
+|hellaswag                               | 0.7103166699860586   |
+|hendrycksTest-abstract_algebra           | 0.34                 |
+|hendrycksTest-anatomy                    | 0.6666666666666666   |
+|hendrycksTest-astronomy                  | 0.6907894736842105   |
+|hendrycksTest-business_ethics            | 0.65                 |
+|hendrycksTest-clinical_knowledge         | 0.7132075471698113   |
+|hendrycksTest-college_biology            | 0.7708333333333334   |
+|hendrycksTest-college_chemistry          | 0.48                 |
+|hendrycksTest-college_computer_science   | 0.53                 |
+|hendrycksTest-college_mathematics        | 0.33                 |
+|hendrycksTest-college_medicine           | 0.6705202312138728   |
+|hendrycksTest-college_physics            | 0.4019607843137255   |
+|hendrycksTest-computer_security          | 0.77                 |
+|hendrycksTest-conceptual_physics         | 0.5787234042553191   |
+|hendrycksTest-econometrics               | 0.5                  |
+|hendrycksTest-electrical_engineering     | 0.5517241379310345   |
+|hendrycksTest-elementary_mathematics     | 0.42592592592592593  |
+|hendrycksTest-formal_logic               | 0.48412698412698413  |
+|hendrycksTest-global_facts               | 0.37                 |
+|hendrycksTest-high_school_biology        | 0.7806451612903226   |
+|hendrycksTest-high_school_chemistry      | 0.4975369458128079   |
+|hendrycksTest-high_school_computer_science | 0.69              |
+|hendrycksTest-high_school_european_history | 0.7757575757575758 |
+|hendrycksTest-high_school_geography      | 0.803030303030303    |
+|hendrycksTest-high_school_government_and_politics | 0.8963730569948186 |
+|hendrycksTest-high_school_macroeconomics | 0.6641025641025641   |
+|hendrycksTest-high_school_mathematics    | 0.36666666666666664  |
+|hendrycksTest-high_school_microeconomics | 0.6890756302521008   |
+|hendrycksTest-high_school_physics        | 0.37748344370860926  |
+|hendrycksTest-high_school_psychology     | 0.8403669724770643   |
+|hendrycksTest-high_school_statistics     | 0.5                  |
+|hendrycksTest-high_school_us_history     | 0.8480392156862745   |
+|hendrycksTest-high_school_world_history  | 0.8059071729957806   |
+|hendrycksTest-human_aging                | 0.6995515695067265   |
+|hendrycksTest-human_sexuality            | 0.7938931297709924   |
+|hendrycksTest-international_law          | 0.8099173553719008   |
+|hendrycksTest-jurisprudence              | 0.7870370370370371   |
+|hendrycksTest-logical_fallacies          | 0.7484662576687117   |
+|hendrycksTest-machine_learning           | 0.4375               |
+|hendrycksTest-management                 | 0.7766990291262136   |
+|hendrycksTest-marketing                  | 0.8888888888888888   |
+|hendrycksTest-medical_genetics           | 0.72                 |
+|hendrycksTest-miscellaneous              | 0.8314176245210728   |
+|hendrycksTest-moral_disputes             | 0.7398843930635838   |
+|hendrycksTest-moral_scenarios            | 0.4324022346368715   |
+|hendrycksTest-nutrition                  | 0.7189542483660131   |
+|hendrycksTest-philosophy                 | 0.7041800643086816   |
+|hendrycksTest-prehistory                 | 0.7469135802469136   |
+|hendrycksTest-professional_accounting    | 0.5035460992907801   |
+|hendrycksTest-professional_law           | 0.4758800521512386   |
+|hendrycksTest-professional_medicine      | 0.6727941176470589   |
+|hendrycksTest-professional_psychology    | 0.6666666666666666   |
+|hendrycksTest-public_relations           | 0.6727272727272727   |
+|hendrycksTest-security_studies           | 0.7183673469387755   |
+|hendrycksTest-sociology                  | 0.8407960199004975   |
+|hendrycksTest-us_foreign_policy          | 0.85                 |
+|hendrycksTest-virology                   | 0.5542168674698795   |
+|hendrycksTest-world_religions            | 0.8421052631578947   |
+|truthfulqa:mc                            | 0.6707176642401714   |
+|winogrande                               | 0.8492501973164956   |
+|gsm8k                                    | 0.7050796057619408   |
 ## Citations