Instructions to use Qwen/Qwen3.8-2.4T-A95B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3.8-2.4T-A95B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Qwen/Qwen3.8-2.4T-A95B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-2.4T-A95B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-2.4T-A95B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Qwen/Qwen3.8-2.4T-A95B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3.8-2.4T-A95B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-2.4T-A95B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3.8-2.4T-A95B

SGLang

How to use Qwen/Qwen3.8-2.4T-A95B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3.8-2.4T-A95B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-2.4T-A95B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3.8-2.4T-A95B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3.8-2.4T-A95B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3.8-2.4T-A95B with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3.8-2.4T-A95B
```

27b when?

by tigerjjw53 - opened 1 day ago

Discussion

tigerjjw53

1 day ago

I am super excited about the 27b model. When are you guys releasing qwen3.8 27b? I can't wait to see the absolute best 27b coding model.

jreoka

1 day ago

2 days i think

g-a-b-y

1 day ago

I'm also confused about the missing 27b. Thought it was supposed to release today too?

CNWPlayer

1 day ago

2 days

jreoka

1 day ago

https://modelscope.cn/models/Qwen/Qwen3.8-27B please see this link

g-a-b-y

1 day ago

Waited for nothing, they should had said that from the beginning 😆

drmcbride

1 day ago

yeah really blows.

inikishev

1 day ago

what about 28b

puchuu

1 day ago

@jreoka , Hello, why did you remove the loading image? It was beautiful.

Can you give it back? Thank you.

darkmatter2222

1 day ago

Here is the working link:
https://modelscope.cn/models/Qwen/Qwen3.8-27B

nonetrix

1 day ago

Here is the working link:
https://modelscope.cn/models/Qwen/Qwen3.8-27B

Damn you

CNWPlayer

1 day ago

Here is the working link:
https://modelscope.cn/models/Qwen/Qwen3.8-27B

omg THANK YOUUUU this helped so much

inikishev

about 10 hours ago

qwhen 27b

nonetrix

about 4 hours ago

19 hours

jreoka

about 4 hours ago

•

edited about 4 hours ago

@jreoka , Hello, why did you remove the loading image? It was beautiful.

Can you give it back? Thank you.

I do not run that page the qwen team runs that page

nonetrix

about 4 hours ago

•

edited about 4 hours ago

Ur the CEO of Qwen now @jreoka congratulations! Now give me ur GPUs, and train me a dedicated AI waifu LLM plox /j

jreoka

about 4 hours ago

LOL

nonetrix

about 4 hours ago

•

edited about 3 hours ago

I'm waiting, surely with that many GPUs you can do a lot in that regard more than any community fine-tune that ends up being worse than base. Fun what if

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment