Instructions to use desva0/Qwen3.8-27B-Uncensored-W4A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use desva0/Qwen3.8-27B-Uncensored-W4A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="desva0/Qwen3.8-27B-Uncensored-W4A16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("desva0/Qwen3.8-27B-Uncensored-W4A16")
model = AutoModelForMultimodalLM.from_pretrained("desva0/Qwen3.8-27B-Uncensored-W4A16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use desva0/Qwen3.8-27B-Uncensored-W4A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "desva0/Qwen3.8-27B-Uncensored-W4A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "desva0/Qwen3.8-27B-Uncensored-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/desva0/Qwen3.8-27B-Uncensored-W4A16

SGLang

How to use desva0/Qwen3.8-27B-Uncensored-W4A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "desva0/Qwen3.8-27B-Uncensored-W4A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "desva0/Qwen3.8-27B-Uncensored-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "desva0/Qwen3.8-27B-Uncensored-W4A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "desva0/Qwen3.8-27B-Uncensored-W4A16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use desva0/Qwen3.8-27B-Uncensored-W4A16 with Docker Model Runner:
```
docker model run hf.co/desva0/Qwen3.8-27B-Uncensored-W4A16
```

Qwen3.8-27B-Uncensored — W4A16 (RTX 3090 build)

A 4-bit weight-only (W4A16) quantization of the uncensored / abliterated Qwen3.8-27B vision-language model, sized to run on a single 24 GB GPU (e.g. RTX 3090).

This is the exact quant used to serve an agentic assistant ("Sage") live on an RTX 3090 in day-to-day use, so it's known to load and run under vLLM on consumer 24 GB hardware — not just a theoretical export.

What it is

Base model: Qwen3.8-27B (VL, function-calling, reasoning), architecture Qwen3_5ForConditionalGeneration.
Uncensoring: the refusal-removed ("abliterated") build published by OrcaRouter under Apache-2.0. This repo is a quantization of that build.
Quantization: 4-bit weight-only, produced with AutoRound and exported in the compressed-tensors pack-quantized format:
- num_bits: 4, type: int, symmetric: true, group_size: 128, strategy: group
- Vision-tower linears are kept at higher precision (listed under ignore), so image understanding is preserved.
Footprint: ~15 GB of weights → fits comfortably in 24 GB with room for KV cache at a useful context length.

Serving with vLLM

vllm serve <path-or-repo>/Qwen3.8-27B-Uncensored-W4A16 \
  --host 0.0.0.0 --port 8000 \
  --served-model-name qwen3.8-27b \
  --gpu-memory-utilization 0.90 \
  --max-model-len 32768 \
  --enable-auto-tool-choice --tool-call-parser hermes

Then hit the OpenAI-compatible endpoint at http://localhost:8000/v1. Adjust --max-model-len to trade context length against KV-cache memory on a 24 GB card. A recent vLLM with compressed-tensors support is required.

Provenance & attribution

Qwen — the Qwen3.8-27B base model (© the Qwen team).
OrcaRouter — the Apache-2.0 abliterated build this repo quantizes.
This repo — 4-bit W4A16 (compressed-tensors) quantization for 24 GB GPUs.

License

Apache-2.0, inherited from the upstream abliterated build. See the bundled LICENSE. You must retain the license and attribution when redistributing.

Responsible use

This is an uncensored / abliterated model: its built-in refusal behaviour has been removed, so it will attempt requests an aligned model would decline. You are responsible for how you deploy it, for any guardrails you add, and for compliance with the laws and platform rules that apply to you.

Downloads last month: 46

Safetensors

Model size

6B params

Tensor type

I32

BF16

Model tree for desva0/Qwen3.8-27B-Uncensored-W4A16

Base model

Qwen/Qwen3.8-27B

Quantized

(849)

this model