How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
# Run inference directly in the terminal:
llama cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
# Run inference directly in the terminal:
llama cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Use pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
# Run inference directly in the terminal:
./llama-cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Use Docker
docker model run hf.co/Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf:Q3_K_XL
Quick Links

OID Dynamic Q3 — Qwen3.8-27B-Uncensored

A compact OID dynamic Q3 quantization of orcarouter/Qwen3.8-27B-Uncensored, targeting aggressive memory reduction while retaining the reasoning, knowledge, long-context recall, coding, and writing ability of the source model.

~11 GB · Dynamic Q3 · 84/100 custom eval · 94/100 core capability

Despite the aggressive quantization level, the model retained particularly strong factual knowledge, hallucination resistance, reasoning, coding, and long-context retrieval in testing. The most visible degradation appeared in fine-grained instruction bookkeeping, exact constraint verification, and semantic consistency during style transformations.

Quick Stats

Parent model orcarouter/Qwen3.8-27B-Uncensored
Quantization OID Dynamic Q3
Quant size ~11 GB
Parameters 27B
Architecture Qwen3.8 / hybrid-attention VLM
Custom eval 84 / 100
Core capability 94 / 100
Compression robustness 74 / 100

Compression Benchmark

The quant was tested with a custom multi-domain benchmark designed specifically to expose capabilities that can degrade under aggressive compression.

It tests:

  • instruction following
  • arithmetic and multi-step reasoning
  • formal logic
  • general knowledge
  • hallucination resistance
  • ambiguity handling and calibration
  • summarization
  • abstraction
  • writing quality
  • style control
  • coding
  • critical reasoning
  • long-context recall
  • integrated constraint reasoning

Result

Overall: 84 / 100

Capability Score
Knowledge 10/10
Hallucination resistance 10/10
Long-context memory 10/10
Reasoning / logic 9/10
Summarization / abstraction 10/10
Coding 9/10
Writing quality 8/10
Instruction following 7/10
Style control 5/10
Output discipline / self-checking 6/10

What Survived Quantization Well

Knowledge retention was excellent. The model correctly handled science, economics, history, statistics, chemistry, and literature questions.

Hallucination resistance remained excellent. It correctly rejected deliberately fabricated or impossible premises instead of inventing plausible answers.

Long-context retrieval was fully preserved in the test. Every planted memory item was recovered correctly, including exact strings, numerical values, derived calculations, and information separated from the retrieval questions by substantial intervening context.

Reasoning remained strong. Multi-step arithmetic, constraint solving, critical reasoning, abstraction, and integrated decision-making were handled successfully.

Coding remained strong. The model caught an issue in its initial iterable implementation and corrected it to support single-pass iterables.

Where Compression Shows

The primary weakness was not loss of knowledge or reasoning ability, but constraint management.

The model occasionally:

  • fixed one formatting constraint while accidentally violating another;
  • incorrectly self-reported word counts;
  • added unsupported details while rewriting the same information into different styles;
  • exposed abandoned intermediate reasoning instead of cleaning the final response;
  • lost minor formatting requirements despite understanding the underlying task.

This suggests that the aggressive Q3 quant preserves the model's underlying capabilities better than its fine-grained executive control / output verification.

In short:

The intelligence survives better than the bookkeeping.

Benchmark Interpretation

The 84/100 score is a custom compression-oriented evaluation, not a standardized academic benchmark and should not be directly compared with MMLU, MMLU-Pro, GPQA, GSM8K, or other published leaderboard scores.

The benchmark was intentionally designed to stress subtle degradation that normal knowledge benchmarks may miss.

The 94/100 core-capability score reflects performance on knowledge, reasoning, memory, abstraction, hallucination resistance, summarization, and coding.

The lower 74/100 compression-robustness score reflects failures involving instruction interference, style-preservation, exact constraints, and output self-verification.

Results are from a single evaluation run and may vary with inference settings, sampler configuration, context length, prompt format, and backend.

Source Model

This is a quantization of:

orcarouter/Qwen3.8-27B-Uncensored

The parent checkpoint is an abliterated / refusal-removed derivative of Qwen3.8-27B. It retains the Qwen3.8-27B architecture and is distributed under the Apache 2.0 license.

Refer to the parent model card for architecture details, upstream evaluations, supported inference frameworks, multimodal functionality, and its safety disclaimer.

Multimodal Status

The compression benchmark above evaluated text capability only.

Vision and other multimodal capabilities were not evaluated by this benchmark, so the 84/100 result should not be interpreted as a multimodal quality score.

Intended Use

This quant is intended for users who want to run a capable 27B-class model under substantially tighter memory constraints.

Potential uses include:

  • local inference
  • experimentation
  • reasoning and general assistant workloads
  • coding
  • long-context tasks
  • quantization research
  • model-compression comparisons
  • red-team and robustness research

Limitations

Aggressive quantization can produce failures that are highly prompt-dependent.

In particular, users should expect occasional degradation when a prompt contains many simultaneous formatting rules, exact counts, tightly constrained rewrites, or requirements that must remain invariant across multiple transformations.

The model may also confidently claim that an output satisfies a numerical constraint when it does not. Applications requiring exact schemas, word counts, or other hard formatting guarantees should validate outputs externally.

Safety

The parent model has had substantial refusal behavior removed through abliteration and therefore does not provide the safety behavior expected from the original aligned Qwen release.

Quantization does not restore those safeguards.

Users deploying this model are responsible for implementing any moderation, access control, validation, or abuse-prevention appropriate to their application.

TL;DR

OID Dynamic Q3 compresses a 27B model to roughly 11 GB while preserving surprisingly strong core capability.

84/100 compression benchmark. Perfect tested long-context recall. Perfect tested hallucination resistance. Strong reasoning and coding.

The main cost of the aggressive quant is not obvious knowledge loss—it is reduced reliability when juggling many small constraints simultaneously.

Downloads last month
111
GGUF
Model size
27B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

3-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Gavvvin/Qwen3.8-27B-Uncensored-OID-gguf

Base model

Qwen/Qwen3.8-27B
Quantized
(28)
this model