Instructions to use XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1")
model = AutoModelForMultimodalLM.from_pretrained("XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1

SGLang

How to use XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 with Docker Model Runner:
```
docker model run hf.co/XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1
```

Qwopus3.6-27B-v2-GPTQ-Pro-v1 / README.md

Commit History

Document upstream vLLM reproducibility

9cca2de
verified

XReyRobert commited on Jun 30

Remove unpublished custom image target from card

66caaae
verified

XReyRobert commited on Jun 29

Clarify custom vLLM image scope

8fb3cbd
verified

XReyRobert commited on Jun 29

Document vLLM deployment image for reproducibility

15b501e
verified

XReyRobert commited on Jun 29

Align support buttons in model card

46a8c29
verified

XReyRobert commited on Jun 29

Add support button to model card

f9c945f
verified

XReyRobert commited on Jun 28

Restore X follow button

e8845bf
verified

XReyRobert commited on Jun 21

Remove misplaced 35B runtime image from 27B card

4e8de53
verified

XReyRobert commited on Jun 16

Add Qwopus3.6-35B-A3B runtime image to model card

a03e47e
verified

XReyRobert commited on Jun 16

Clarify Terminal-Bench protocol settings

6e0077f

Xavier Rey-Robert commited on Jun 16

Update Qwen Terminal-Bench comparison table

3b1639f

Xavier Rey-Robert commited on Jun 16

Fix Qwen Terminal-Bench 2.0 comparison

2c59376

Xavier Rey-Robert commited on Jun 16

Add Qwen Terminal-Bench comparison

82f2c89

Xavier Rey-Robert commited on Jun 16

Add Terminal-Bench 2.0 evaluation report

898a3f9

Xavier Rey-Robert commited on Jun 16

Fix X follow button handle

f57be03
verified

XReyRobert commited on Jun 7

Add X follow button to model card

147a354
verified

XReyRobert commited on Jun 7

docs: align runtime notes with metrics card layout

64026c0
verified

XReyRobert commited on Jun 1

docs: add vLLM 0.21 stable runtime note

64cb9f7
verified

XReyRobert commited on Jun 1

docs: note reproducible vLLM runtime for 131K serving

c31263b
verified

XReyRobert commited on Jun 1

Update MMLU-Pro one-shot benchmark

41deefe
verified

XReyRobert commited on May 25

Refine validation scope wording

b59acaf
verified

XReyRobert commited on May 24

Clarify MMLU-Pro evaluation protocol

387a561
verified

XReyRobert commited on May 24

Use Jackrong-style benchmark HTML section

096765c
verified

XReyRobert commited on May 24

Update model card with 24 May MMLU-Pro results

45c4cbd
verified

XReyRobert commited on May 24

Clean model card for public release

8040574
verified

XReyRobert commited on May 23

Add personal project notice

ca73e3d
verified

XReyRobert commited on May 23

Use simple benchmark visual in model card

30edb4a
verified

XReyRobert commited on May 23

Embed RTX 3090 benchmark visual

6144553
verified

XReyRobert commited on May 23

Rename public repo references to short v1 name

c48c3cd
verified

XReyRobert commited on May 23

Remove internal build lineage from model card

8ec1162
verified

XReyRobert commited on May 23

Prepare public v1 model card

df61a28
verified

XReyRobert commited on May 23

Update v1 model card

570a950
verified

XReyRobert commited on May 23

Clarify ns256 artifact is non-MTP as published

541c8cf
verified

XReyRobert commited on May 23

Format RTX 3090 metrics as table

73e9bcd
verified

XReyRobert commited on May 23

Update RTX 3090 Hermes metrics

022c832
verified

XReyRobert commited on May 23

Update model card intro and reasoning notes

14ee4f3
verified

XReyRobert commited on May 23

Update model card with RTX 3090 validation plan

22925f4
verified

XReyRobert commited on May 23

Upload GPTQ-Pro FOEM 4-bit g128 ns256 v2

de5bd59
verified

XReyRobert commited on May 23

Commit History

Document upstream vLLM reproducibility 9cca2de verified

Remove unpublished custom image target from card 66caaae verified

Clarify custom vLLM image scope 8fb3cbd verified

Document vLLM deployment image for reproducibility 15b501e verified

Align support buttons in model card 46a8c29 verified

Add support button to model card f9c945f verified

Restore X follow button e8845bf verified

Remove misplaced 35B runtime image from 27B card 4e8de53 verified

Add Qwopus3.6-35B-A3B runtime image to model card a03e47e verified

Clarify Terminal-Bench protocol settings 6e0077f

Update Qwen Terminal-Bench comparison table 3b1639f

Fix Qwen Terminal-Bench 2.0 comparison 2c59376

Add Qwen Terminal-Bench comparison 82f2c89

Add Terminal-Bench 2.0 evaluation report 898a3f9

Fix X follow button handle f57be03 verified

Add X follow button to model card 147a354 verified

docs: align runtime notes with metrics card layout 64026c0 verified

docs: add vLLM 0.21 stable runtime note 64cb9f7 verified

docs: note reproducible vLLM runtime for 131K serving c31263b verified

Update MMLU-Pro one-shot benchmark 41deefe verified

Refine validation scope wording b59acaf verified

Clarify MMLU-Pro evaluation protocol 387a561 verified

Use Jackrong-style benchmark HTML section 096765c verified

Update model card with 24 May MMLU-Pro results 45c4cbd verified

Clean model card for public release 8040574 verified

Add personal project notice ca73e3d verified

Use simple benchmark visual in model card 30edb4a verified

Embed RTX 3090 benchmark visual 6144553 verified

Rename public repo references to short v1 name c48c3cd verified

Remove internal build lineage from model card 8ec1162 verified

Prepare public v1 model card df61a28 verified

Update v1 model card 570a950 verified

Clarify ns256 artifact is non-MTP as published 541c8cf verified

Format RTX 3090 metrics as table 73e9bcd verified

Update RTX 3090 Hermes metrics 022c832 verified

Update model card intro and reasoning notes 14ee4f3 verified

Update model card with RTX 3090 validation plan 22925f4 verified

Upload GPTQ-Pro FOEM 4-bit g128 ns256 v2 de5bd59 verified

Document upstream vLLM reproducibility

9cca2de
verified

Remove unpublished custom image target from card

66caaae
verified

Clarify custom vLLM image scope

8fb3cbd
verified

Document vLLM deployment image for reproducibility

15b501e
verified

Align support buttons in model card

46a8c29
verified

Add support button to model card

f9c945f
verified

Restore X follow button

e8845bf
verified

Remove misplaced 35B runtime image from 27B card

4e8de53
verified

Add Qwopus3.6-35B-A3B runtime image to model card

a03e47e
verified

Clarify Terminal-Bench protocol settings

6e0077f

Update Qwen Terminal-Bench comparison table

3b1639f

Fix Qwen Terminal-Bench 2.0 comparison

2c59376

Add Qwen Terminal-Bench comparison

82f2c89

Add Terminal-Bench 2.0 evaluation report

898a3f9

Fix X follow button handle

f57be03
verified

Add X follow button to model card

147a354
verified

docs: align runtime notes with metrics card layout

64026c0
verified

docs: add vLLM 0.21 stable runtime note

64cb9f7
verified

docs: note reproducible vLLM runtime for 131K serving

c31263b
verified

Update MMLU-Pro one-shot benchmark

41deefe
verified

Refine validation scope wording

b59acaf
verified

Clarify MMLU-Pro evaluation protocol

387a561
verified

Use Jackrong-style benchmark HTML section

096765c
verified

Update model card with 24 May MMLU-Pro results

45c4cbd
verified

Clean model card for public release

8040574
verified

Add personal project notice

ca73e3d
verified

Use simple benchmark visual in model card

30edb4a
verified

Embed RTX 3090 benchmark visual

6144553
verified

Rename public repo references to short v1 name

c48c3cd
verified

Remove internal build lineage from model card

8ec1162
verified

Prepare public v1 model card

df61a28
verified

Update v1 model card

570a950
verified

Clarify ns256 artifact is non-MTP as published

541c8cf
verified

Format RTX 3090 metrics as table

73e9bcd
verified

Update RTX 3090 Hermes metrics

022c832
verified

Update model card intro and reasoning notes

14ee4f3
verified

Update model card with RTX 3090 validation plan

22925f4
verified

Upload GPTQ-Pro FOEM 4-bit g128 ns256 v2

de5bd59
verified