Instructions to use skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv")
model = AutoModelForCausalLM.from_pretrained("skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv

SGLang

How to use skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv with Docker Model Runner:
```
docker model run hf.co/skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv
```

Model Details

Finetune of gpt-sw3-6.7b-v2 using LoRA with 4-bit quantization. Adapters merged with base model, however with bfloat16 precision tensors.

Usage:

This is a finetune experiment. How to use will be provided later.

Evaluation

ScandEval benchmarks:

Dataset	Performance (Metric 1 / Metric 2)
swerec	74.95 ± 1.17 / 61.38 ± 1.37
suc3	30.75 ± 4.11 / 25.69 ± 4.83
scala-sv	8.96 ± 2.09 / 51.50 ± 2.94
scandiqa-sv	50.71 ± 0.99 / 56.76 ± 0.89
swedn	64.37 ± 0.72 / 18.25 ± 0.29
mmlu-sv	5.45 ± 0.91 / 28.14 ± 0.82
hellaswag-sv	27.95 ± 0.73 / 4.19 ± 0.94
speed	5322.20 ± 1132.75 / 1280.06 ± 408.08

Downloads last month: 15

Safetensors

Model size

7B params

Tensor type

BF16

Model tree for skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv

Base model

AI-Sweden-Models/gpt-sw3-6.7b-v2

Finetuned

(6)

this model

skvarre
/

gpt-sw3-6.7b-v2-instruct-slimorcasv

Model Details

Usage:

Evaluation

Model tree for skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv

Dataset used to train skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv