Instructions to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL # Run inference directly in the terminal: llama cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL # Run inference directly in the terminal: llama cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL # Run inference directly in the terminal: ./llama-cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL # Run inference directly in the terminal: ./build/bin/llama-cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Use Docker
docker model run hf.co/Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
- LM Studio
- Jan
- vLLM
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
- SGLang
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with Ollama:
ollama run hf.co/Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
- Unsloth Desktop
- Pi
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with Docker Model Runner:
docker model run hf.co/Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
- Lemonade
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Run and chat with the model
lemonade run user.Qwen3.8-27B-Uncensored-OID-gguf-Q3_K_XL
List all available models
lemonade list
- Hermes Agent
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL# Run inference directly in the terminal:
llama cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XLUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL# Run inference directly in the terminal:
./llama-cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XLBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL# Run inference directly in the terminal:
./build/bin/llama-cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XLUse Docker
docker model run hf.co/Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XLOID Dynamic Q3 — Qwen3.8-27B-Uncensored
A compact OID dynamic Q3 quantization of orcarouter/Qwen3.8-27B-Uncensored, targeting aggressive memory reduction while retaining the reasoning, knowledge, long-context recall, coding, and writing ability of the source model.
~11 GB · Dynamic Q3 · 84/100 custom eval · 94/100 core capability
Despite the aggressive quantization level, the model retained particularly strong factual knowledge, hallucination resistance, reasoning, coding, and long-context retrieval in testing. The most visible degradation appeared in fine-grained instruction bookkeeping, exact constraint verification, and semantic consistency during style transformations.
Quick Stats
| Parent model | orcarouter/Qwen3.8-27B-Uncensored |
| Quantization | OID Dynamic Q3 |
| Quant size | ~11 GB |
| Parameters | 27B |
| Architecture | Qwen3.8 / hybrid-attention VLM |
| Custom eval | 84 / 100 |
| Core capability | 94 / 100 |
| Compression robustness | 74 / 100 |
Compression Benchmark
The quant was tested with a custom multi-domain benchmark designed specifically to expose capabilities that can degrade under aggressive compression.
It tests:
- instruction following
- arithmetic and multi-step reasoning
- formal logic
- general knowledge
- hallucination resistance
- ambiguity handling and calibration
- summarization
- abstraction
- writing quality
- style control
- coding
- critical reasoning
- long-context recall
- integrated constraint reasoning
Result
Overall: 84 / 100
| Capability | Score |
|---|---|
| Knowledge | 10/10 |
| Hallucination resistance | 10/10 |
| Long-context memory | 10/10 |
| Reasoning / logic | 9/10 |
| Summarization / abstraction | 10/10 |
| Coding | 9/10 |
| Writing quality | 8/10 |
| Instruction following | 7/10 |
| Style control | 5/10 |
| Output discipline / self-checking | 6/10 |
What Survived Quantization Well
Knowledge retention was excellent. The model correctly handled science, economics, history, statistics, chemistry, and literature questions.
Hallucination resistance remained excellent. It correctly rejected deliberately fabricated or impossible premises instead of inventing plausible answers.
Long-context retrieval was fully preserved in the test. Every planted memory item was recovered correctly, including exact strings, numerical values, derived calculations, and information separated from the retrieval questions by substantial intervening context.
Reasoning remained strong. Multi-step arithmetic, constraint solving, critical reasoning, abstraction, and integrated decision-making were handled successfully.
Coding remained strong. The model caught an issue in its initial iterable implementation and corrected it to support single-pass iterables.
Where Compression Shows
The primary weakness was not loss of knowledge or reasoning ability, but constraint management.
The model occasionally:
- fixed one formatting constraint while accidentally violating another;
- incorrectly self-reported word counts;
- added unsupported details while rewriting the same information into different styles;
- exposed abandoned intermediate reasoning instead of cleaning the final response;
- lost minor formatting requirements despite understanding the underlying task.
This suggests that the aggressive Q3 quant preserves the model's underlying capabilities better than its fine-grained executive control / output verification.
In short:
The intelligence survives better than the bookkeeping.
Benchmark Interpretation
The 84/100 score is a custom compression-oriented evaluation, not a standardized academic benchmark and should not be directly compared with MMLU, MMLU-Pro, GPQA, GSM8K, or other published leaderboard scores.
The benchmark was intentionally designed to stress subtle degradation that normal knowledge benchmarks may miss.
The 94/100 core-capability score reflects performance on knowledge, reasoning, memory, abstraction, hallucination resistance, summarization, and coding.
The lower 74/100 compression-robustness score reflects failures involving instruction interference, style-preservation, exact constraints, and output self-verification.
Results are from a single evaluation run and may vary with inference settings, sampler configuration, context length, prompt format, and backend.
Source Model
This is a quantization of:
orcarouter/Qwen3.8-27B-Uncensored
The parent checkpoint is an abliterated / refusal-removed derivative of Qwen3.8-27B. It retains the Qwen3.8-27B architecture and is distributed under the Apache 2.0 license.
Refer to the parent model card for architecture details, upstream evaluations, supported inference frameworks, multimodal functionality, and its safety disclaimer.
Multimodal Status
The compression benchmark above evaluated text capability only.
Vision and other multimodal capabilities were not evaluated by this benchmark, so the 84/100 result should not be interpreted as a multimodal quality score.
Intended Use
This quant is intended for users who want to run a capable 27B-class model under substantially tighter memory constraints.
Potential uses include:
- local inference
- experimentation
- reasoning and general assistant workloads
- coding
- long-context tasks
- quantization research
- model-compression comparisons
- red-team and robustness research
Limitations
Aggressive quantization can produce failures that are highly prompt-dependent.
In particular, users should expect occasional degradation when a prompt contains many simultaneous formatting rules, exact counts, tightly constrained rewrites, or requirements that must remain invariant across multiple transformations.
The model may also confidently claim that an output satisfies a numerical constraint when it does not. Applications requiring exact schemas, word counts, or other hard formatting guarantees should validate outputs externally.
Safety
The parent model has had substantial refusal behavior removed through abliteration and therefore does not provide the safety behavior expected from the original aligned Qwen release.
Quantization does not restore those safeguards.
Users deploying this model are responsible for implementing any moderation, access control, validation, or abuse-prevention appropriate to their application.
TL;DR
OID Dynamic Q3 compresses a 27B model to roughly 11 GB while preserving surprisingly strong core capability.
84/100 compression benchmark. Perfect tested long-context recall. Perfect tested hallucination resistance. Strong reasoning and coding.
The main cost of the aggressive quant is not obvious knowledge loss—it is reduced reliability when juggling many small constraints simultaneously.
- Downloads last month
- 111
3-bit
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL# Run inference directly in the terminal: llama cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL