Instructions to use WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static")
model = AutoModelForMultimodalLM.from_pretrained("WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static

SGLang

How to use WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static with Docker Model Runner:
```
docker model run hf.co/WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static
```

Qwen3-VL-8B-Instruct SDNQ INT8 static

SDNQ static weight-only int8 quantization of Qwen/Qwen3-VL-8B-Instruct (snapshot 0c351dd01ed87e9c1b53cbc748cba10e6187ff3b).

Built as the instruction encoder (mllm) for the Boogu Image 0.1 pipelines: the mllm/ shipped in WaveCut/Boogu-Image-0.1-Turbo-SDNQ-uint4-static (and the Edit variant) is the stock Qwen3-VL-8B-Instruct byte for byte, so this repo is a drop-in replacement that cuts the encoder's host RAM from ~17.5 GiB (bf16) to ~9.4 GiB on disk / in RAM. It is a complete Qwen3-VL-8B-Instruct checkpoint (processor + tokenizer included) and loads with plain transformers after import sdnq.

Recipe

SDNQ 0.2.5 static quantization, weights_dtype=int8, group_size=0 (auto), dequantize_fp32=True.
Kept in bf16: embeddings, lm_head, multi_modal_projector, vision patch_embed and every module below SDNQ's size floor.
Quantized layers: {"int8": 368}; quantized params: {"int8": 7517270016}; saved size 9.37 GiB (bf16 source 16.33 GiB).
Save/load round trip verified through sdnq.loader.load_sdnq_model (all parameters bit-identical); encoder last_hidden_state vs bf16 on a mixed RU/EN/ZH prompt: {"cos_mean": 0.9716747403144836, "cos_min": 0.7031210064888, "max_abs": 30.71875, "rel_l2": 0.24023021757602692}.
Runtime used for all numbers below: torch 2.10.0+cu130, transformers 4.57.1, diffusers 0.39.0.dev0, sdnq 0.2.5, NVIDIA GeForce RTX 4090 (RunPod, disposable pod), Python 3.12.3.

Boogu Image 0.1 Turbo A/B (same seed, bf16 encoder vs this encoder)

Pipeline BooguImageTurboPipeline from WaveCut/Boogu-Image-0.1-Turbo-SDNQ-uint4-static @ 422ac37ad305169703fd8f847ef76418cab8363f (SDNQ uint4 transformer), 1024x1024, 4 DMD steps, text_guidance_scale=1.0, image_guidance_scale=1.0, empty_instruction_guidance_scale=0.0, dmd_conditioning_sigma=0.001, max_sequence_length=1280, sequential CPU offload, SDNQ quantized matmul on for the transformer. 12 prompts (EN/RU/ZH, typography, diagrams, long prompt); image metrics are against the bf16-encoder image of the same prompt and seed, embedding metrics compare the encoder output that is fed to the transformer. Encoder rows ending in -wo run weight-only (quantized weights dequantized per layer on the GPU, use_quantized_matmul=False); rows without the suffix also quantize the activations to int8 for the matmul, which is measurably worse for this encoder (Qwen3-VL activation outliers) and is not the recommended mode.

Encoder	Gen mean s	Encode mean s	Peak VRAM MiB	Host RSS after load GiB	RSS anon GiB	PSNR	SSIM	LPIPS	Embed cos	Embed rel-L2
bf16 (reference)	20.01	1.27	2446	1.33	0.83	-	-	-	-	-
int8-wo (this repo)	18.86	1.65	2446	1.34	0.83	20.40	0.7507	0.1837	0.99901	0.03786
int8	19.48	3.52	2446	1.34	0.83	14.66	0.6070	0.3689	0.96518	0.23915
uint4-svd32	18.85	3.23	2446	1.58	0.92	13.72	0.5794	0.4101	0.91081	0.36603
uint4-svd32-wo	18.80	1.87	2446	1.57	0.91	13.90	0.5969	0.3834	0.92782	0.32357

Host RSS is the whole pipeline process (transformer + VAE + encoder) after load with sequential offload, measured from /proc/self/status.

Per prompt (int8-wo)

Prompt	PSNR	SSIM	LPIPS	Embed cos	Embed rel-L2
00_vase	34.89	0.9766	0.0138	0.99899	0.03695
01_portrait	18.32	0.6910	0.2393	0.99904	0.03652
02_abstract	19.80	0.6543	0.2766	0.99887	0.03938
03_typography_en	17.92	0.7703	0.1275	0.99906	0.03706
04_text_ru	18.21	0.7628	0.1471	0.99918	0.03715
05_diagram	17.15	0.8007	0.2020	0.99893	0.03835
06_product	21.88	0.8377	0.2001	0.99894	0.03795
07_anime	14.41	0.5586	0.2380	0.99891	0.03882
08_landscape	24.23	0.8055	0.2121	0.99893	0.03843
09_multiobject	19.96	0.7162	0.1882	0.99903	0.03771
10_long	20.10	0.6254	0.2218	0.99933	0.03286
11_zh	17.93	0.8093	0.1378	0.99894	0.04319

Pairwise images (bf16 | quantized) are in comparison/.

Usage

import torch
import sdnq  # registers the SDNQ quantizer with transformers
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

repo = "WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static"
processor = AutoProcessor.from_pretrained(repo)
model = Qwen3VLForConditionalGeneration.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

Loader used by the aifarm draw service (weights stay on CPU, sequential offload moves layers per forward):

from sdnq.loader import load_sdnq_model
from huggingface_hub import snapshot_download

# model_cls=None: sdnq 0.2.5 crashes on an explicit transformers class (fixed in 0.2.6);
# use_quantized_matmul=False: weight-only, keeps the encoder output at cos ~0.999 vs bf16.
mllm = load_sdnq_model(snapshot_download(repo), model_cls=None,
                       dtype=torch.bfloat16, device="cpu", use_quantized_matmul=False)

Files

model-*.safetensors, config.json (with quantization_config), quantization_config.json - SDNQ checkpoint.
quantization_summary.json - recipe, layer counts, storage, save/load checks.
benchmark/summary.json, benchmark/*.metrics.json - raw A/B metrics; benchmark/prompts.json - the prompt set.
comparison/ - pairwise images and the contact sheet.
SHA256SUMS - checksums of the checkpoint files.

Downloads last month: -

Safetensors

Model size

9B params

Tensor type

BF16

F32

Model tree for WaveCut/Qwen3-VL-8B-Instruct-SDNQ-int8-static

Base model

Qwen/Qwen3-VL-8B-Instruct

Quantized

(110)

this model