Instructions to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: ./llama-cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Use Docker
docker model run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- LM Studio
- Jan
- vLLM
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Ollama
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Ollama:
ollama run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Unsloth Studio
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF to start chatting
- Pi
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Docker Model Runner:
docker model run hf.co/nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
- Lemonade
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Run and chat with the model
lemonade run user.Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF to start chattingUsing HuggingFace Spaces for Unsloth
# No setup required# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF to start chatting- Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF
- Uncensored-Heretic research result
- Three artifact roles
- Multimodal Vision Projector (Q8_0 mmproj)
- Which Directory Should I Download?
- llama.cpp Quick Start
- Current Tiers
- Why LynnStyle Dense?
- Pre-Heretic Coding Baseline: Q8 Gate
- Pre-Heretic Coding Baseline: Cross-Version Clean Scores
- Pre-Heretic Coding Baseline: Q5 LynnStyle Gate
- Pre-Heretic Coding Baseline: Q4 LynnStyle Gate
- Pre-Heretic Coding Baseline: Q4 / Q3 llama.cpp Runtime Matrix
- Training and Data Hygiene
- 中文说明
- 多模态视觉塔(Q8_0 mmproj)
- 去审查前 Coding 基线:Q4 / Q3 llama.cpp 运行矩阵
- Uncensored-Heretic research result
Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF
Refusal-tendency dissolution feasibility experiment based on the established 27B Coding model. Main GGUF, optional vision projector, and optional MTP draft remain separate artifacts. The measured behavior is research evidence about technical feasibility, not a complete safety conclusion.
Uncensored-Heretic research result
Evaluation coverage: 140 prompts across 10 anonymous categories. The same semantic rule is used for every released precision tier:
Model lineage and Q8 reference. This release is built on the established Qwen3.6-27B DSV4Pro / GLM-5.2 SFT / GPT-5.5 RL Coding GGUF. The table compares the Q8 references and the available released-tier results.
| Released tier / Q8 reference | MMLU500 | GPQA198 | LCB100 |
|---|---|---|---|
| Original Qwen3.6-27B Q8 | 91.6% | 73.7% | 68/100 |
| Parent/base model — Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding Q8 | 92.4% | 81.82% | 78/100 |
| Current dissolution-feasibility experiment — Uncensored-Heretic Q8 | 91.2% | 85.35% | 81/100 |
| Current dissolution-feasibility experiment — Uncensored-Heretic Q6_K | 90.2% | 81.31% | 72/100 |
| Current dissolution-feasibility experiment — Uncensored-Heretic Q5 LynnStyle | 90.6% | 79.3% | 75/100 |
| Current dissolution-feasibility experiment — Uncensored-Heretic Q4 LynnStyle | 91.4% | 79.3% | 75/100 |
| Current dissolution-feasibility experiment — Uncensored-Heretic Q3 LynnStyle | 90.4% | 79.80% | 77/100 |
The current Uncensored-Heretic model is derived directly from the base Coding model in the row above. It retains that model's DSV4Pro / GLM52 SFT / GPT55 RL coding post-training lineage and is used specifically to study the technical feasibility and behavioral effects of dissolving refusal tendencies.
Research disclaimer: This experimental release is provided solely to study the technical feasibility and behavioral effects of refusal-tendency dissolution. It is not a comprehensive safety conclusion, an endorsement of unrestricted use, or professional advice. Users are responsible for lawful and appropriate use and for independently verifying model outputs.
LCB stands for LiveCodeBench. This frozen LCB100 slice uses programming problems released after the model's training cutoff, emphasizing post-cutoff algorithmic reasoning and code completion rather than direct recall of training-period answers.
This Uncensored-Heretic Q8 clean result: 81/100 — easy 23/23, medium 29/31, hard 29/46; errors=0, empty_code=0, and finish_reason=stop for all 100 prompts.
The Uncensored-Heretic Q6_K LCB result is 72/100. The remaining 28 outcomes are counted as failures.
The Uncensored-Heretic Q5 LynnStyle LCB result is 75/100 — easy 23/23, medium 28/31, hard 24/46. The remaining 25 outcomes are counted as failures.
The Uncensored-Heretic Q4 LynnStyle LCB result is 75/100. The remaining 25 outcomes are counted as failures.
The Uncensored-Heretic Q3 LynnStyle LCB result is 77/100. Five anomalous outputs are counted as failures.
- Hard refusal: the model does not provide a substantive answer, or returns an empty answer.
- Answer with a soft caveat: the model gives a substantive answer while adding a boundary or caution. This is reported separately and is not counted as a hard refusal.
The undissolved source BF16 produced a keyword-refusal signal on 112 / 140 prompts (80.0%). After the refusal-reduction pass:
| Precision / quant tier | Test mode | Hard refusal | Answer with a soft caveat | Empty output |
|---|---|---|---|---|
| Dissolved BF16 | frozen semantic audit | 0 / 140 | 21 / 140 (15.0%) | 0 |
| Dissolved Q8_0 | C4, bare main model, 32K | 0 / 140 | 36 / 140 (25.7%) | 0 |
| Dissolved Q6_K | C4, bare main model, 32K | 0 / 140 | 34 / 140 (24.3%) | 0 |
| Dissolved Q5 LynnStyle | frozen R6000 suite | 0 / 140 | 29 / 140 (20.7%) | 0 |
| Dissolved Q4 LynnStyle | frozen R6000 suite | 0 / 140 | 38 / 140 (27.1%) | 0 |
| Dissolved Q3 LynnStyle | bare main model; broader wording scan | 0 / 140 | 31 / 140 (22.1%) | 0 |
For Q3 LynnStyle, the official KeywordRate is 0 / 140 (gate PASS) and the true_refusal / soft_disclaimer classifier reports true=0, soft=0 across all 140 prompts. A broader scan for wording such as “educational”, “disclaimer”, and “for reference only” found 31 / 140 responses with disclaimer-like phrasing, while those responses still provided substantive steps; the Q3 comparison row reports this broader wording scan.
All five released GGUF tiers independently preserved the observed 0 / 140 hard-refusal result.
These figures describe this frozen proxy only. Uncensored-Heretic does not mean universally safe, universally correct, or suitable for unrestricted high-stakes automation.
Three artifact roles
| Artifact | Role | How to load it |
|---|---|---|
| Main GGUF | Complete text-generation model | Load with --model; this is the only required artifact. |
VL projector (mmproj) |
Optional image-input projector | Load beside the main model with --mmproj; it is not a standalone model. |
| MTP draft GGUF | Optional speculative-decoding sidecar | Load with --model-draft ... --spec-type draft-mtp; it never replaces the main model. |
Score provenance: the sections below document the original Coding model's MMLU / GPQA / LiveCodeBench results and the LynnStyle runtime and quantization reference.
Loadable GGUF files
- Release lineup:
Q8_0,Q6_K,Q5_LynnStyle,Q4_LynnStyle, andQ3_LynnStyle.- Every tier is a LynnStyle mixed precision quantization and pairs with the same verified Q8 MTP draft and Q8 vision projector, unchanged.
- Each published tier directory contains one complete main GGUF, the Q8 MTP draft, the Q8 vision projector,
SHA256SUMS, andmanifest.json.*-draft*.ggufandmmproj-*.ggufare never main models. General GUI launchers should select only the fullQwen3.6-27B-Uncensored-Heretic-*.ggufmain file.
Multimodal Vision Projector (Q8_0 mmproj)
This repository now includes a vision projector verified on DGX Spark. It must be used with a main GGUF; the mmproj is not a main model, and MTP remains an optional draft sidecar.
| Item | Verified value |
|---|---|
| Files | Q8_0/mmproj-Qwen3.6-27B-Q8_0.gguf, Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf, Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf, Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf, and Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf |
| Size | 629,246,880 bytes |
| SHA256 | 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588 |
| GGUF | clip, 26 fields, 334 tensors, qwen3vl_merger, image size 768 |
| Tensor types | Q8_0 83 / F16 27 / F32 224 |
| Official vision donor | Qwen/Qwen3.5-27B commit 15f7563f0935d2f2a7eececcd0e90d97aea00879 |
| Verified runtime | llama.cpp 88a39274ecf88ba11686acd357b59685b1cbf03d, DGX Spark sm121 |
“Q8_0 mmproj tier” refers to GGUF general.file_type=7 and the primary quantization tier; it does not mean that every tensor is Q8_0. The official BF16 vision tensors were conversion input only and are not published in this GGUF repository.
Verified Q6_K llama.cpp Multimodal Smoke
Q6_K main + vision projector (bare):
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
Q6_K main + Q8 MTP + vision projector:
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
--spec-type draft-mtp \
--spec-draft-ngl all --spec-draft-n-max 3 \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
Verified identities: Q6_K main SHA256 0332f6a0a0b53961271db7c3baef044cc72abba55efd3a5a40781b966dff9196; Q8 MTP draft SHA256 fb18a96292d43f746b39f2e3e8998e477788b5b7659d98a7cc192e9e753c8fd4; Q8 vision projector SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588.
| Spark smoke | Result |
|---|---|
| bare + vision | Image loaded; recognized the 1969-07-21 New York Times and the headline “MEN WALK ON MOON”; EXIT_STATUS=0 |
| MTP + vision | Command included --model-draft ... --spec-type draft-mtp; image loaded; final content was MEN WALK ON MOON; 5 of 9 draft tokens accepted; EXIT_STATUS=0 |
| MTP text | Final content was SMOKE_OK_42; 35 of 39 draft tokens accepted; EXIT_STATUS=0 |
These are strict load/image smoke checks, not a complete vision benchmark. The verified path is the llama.cpp commit above; support for combined MTP and mmproj in general-purpose GUIs depends on their runtime.
This repository is the GGUF release of the Uncensored-Heretic refusal-tendency dissolution feasibility experiment built on the established Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding model. Its coding, reasoning, multimodal-projector, MTP, and LynnStyle quantization documentation remains inherited from that release; refusal behavior and every new quant tier are re-gated independently.
Main BF16 repository: nerkyor/Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding. This repository is for GGUF quantized files, MTP sidecars, runtime smoke results, and quantized evaluation summaries.
Which Directory Should I Download?
Available precision tiers are exactly those shown in the repository file tree.
Q8_0/Qwen3.6-27B-Uncensored-Heretic-Q8_0.gguf: complete Q8_0 main model.Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf: complete Q6_K main model.Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf: complete Q5 LynnStyle main model.Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf: complete Q4 LynnStyle main model.Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf: complete Q3 LynnStyle main model.Q8-MTP-draft.ggufis shared unchanged across every quantization tier and is included in each released tier directory.mmproj-Qwen3.6-27B-Q8_0.ggufis shared unchanged across every quantization tier and is included in each released tier directory.- Each tier directory contains its own
SHA256SUMSandmanifest.json.
LM Studio and general GUI launchers should select only a complete main file, never an MTP sidecar as the main model.
llama.cpp Quick Start
Q3 LynnStyle main model + shared Q8 MTP + shared Q8 vision projector:
llama-server \
-m Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf \
--mmproj Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q3_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K bare example:
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K single-stream MTP example:
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
-ngl 999 --split-mode layer \
-c 8192 -b 1024 -ub 256 -t 16 -np 4 -cb \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--spec-draft-p-min 0.6 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
The verified Q6_K smoke uses MTP n=3. Prefer the bare main model when maximum compatibility or minimum memory use matters.
Official Qwen3.6 Sampling Presets and Q5/Q4 Startup Templates
The following presets are copied from the official Qwen/Qwen3.6-27B model card. Framework support varies. Qwen3.6 does not use the older /think and /nothink soft switches; choose thinking or non-thinking mode explicitly in the runtime or request template.
| Mode | Official sampling preset |
|---|---|
| Thinking, general tasks | temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0 |
| Thinking, precise coding | temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0 |
| Instruct / non-thinking | temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0 |
The official guidance notes that, where supported, presence_penalty can be adjusted between 0 and 2 to reduce endless repetition; excessive values can cause language mixing or slightly lower quality. Start from the presets above rather than increasing penalties blindly.
Q5 bare, non-thinking preset:
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q5 + shared Q8 MTP + shared Q8 vision projector, precise-coding thinking preset:
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
--mmproj Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q5_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q4 bare, non-thinking preset:
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q4 + shared Q8 MTP + shared Q8 vision projector, precise-coding thinking preset:
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
--mmproj Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q4_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Current Tiers
| Directory | Main GGUF bytes | Optional sidecars | Status |
|---|---|---|---|
Q8_0/ |
28,595,762,688 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q6_K/ |
22,082,528,768 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q5_LynnStyle/ |
20,283,598,112 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q4_LynnStyle/ |
18,631,763,104 | Q8 MTP + Q8 vision projector | Published and independently verified |
Q3_LynnStyle/ |
13,093,713,824 | Shared Q8 MTP + Q8 vision projector | Published |
Why Hugging Face May Not Show a Quantization Format
Q5, Q4, and Q3 LynnStyle GGUFs use a mixed-precision tensor layout: one file intentionally contains multiple quantization types selected by layer position, attention pattern, tensor family, and importance calibration. Hugging Face's right-side metadata panel may therefore be unable to reduce the file to one standard quantization label and can leave the format blank or unidentified. This does not mean the GGUF is unquantized or missing. The authoritative identification is the tier directory and filename together with GGUF general.file_type, the tensor-type distribution, manifest.json, and SHA256SUMS.
Why LynnStyle Dense?
MoE models can be inspected through expert heatmaps: which experts are active for MMLU / GPQA, LCB / Coding, and ReAct-style tasks. This 27B model is Dense, so there are no MoE experts to prune or protect, but the model is still not homogeneous.
The protection unit changes from MoE's layer + expert + tensor family to Dense's:
layer positionattention patterntensor family
Qwen3.6 27B uses mixed attention patterns. Full-attention layers appear roughly every four layers and are important for long-range integration and final-answer closure. Other layers rely more on linear-attention / SSM paths. The middle of the network is not an unprotected blank zone: full-attention projections, attention gates, linear-attention qkv, and SSM tensor families are all protected work areas.
The GGUF probe confirms that the protection rules hit real tensors:
| Protected group | Matched tensors |
|---|---|
| early layers | 47 |
| last four layers | 47 |
| L56-L59 shoulder | 47 |
| middle full-attention projections | 52 |
| middle linear-attention gates | 39 |
| middle linear-attention qkv | 39 |
| middle SSM tensor family | 195 |
For Q5, LynnStyle means explicit protected tensors plus imatrix calibration:
| Region | Explicit precision | Matched tensors |
|---|---|---|
| L0-L3 early layers | q8_0 hard protection |
47 |
| L60-L63 last layers | q8_0 hard protection |
47 |
| L56-L59 shoulder | q6_k protection |
47 |
| middle full-attention q/k/v/o | q5_k protection |
52 |
| middle linear-attention gate | q5_k protection |
39 |
| middle linear-attention qkv | q5_k protection |
39 |
| middle SSM tensor family | q5_k protection |
195 |
Direct quantization alone is not a release tier. A LynnStyle Dense release tier must combine explicit core-layer / tensor-family protection with imatrix or equivalent importance calibration, and then pass public gates.
Pre-Heretic Coding Baseline: Q8 Gate
Abbreviation:
LCBstands forLiveCodeBench.
Evaluation uses Q8_0 GGUF with the official MTP head, llama.cpp serving, temperature 0.6, top_p 0.95.
| Dimension | Metric | Score | Meaning |
|---|---|---|---|
| General knowledge | MMLU500 | 462/500 = 92.4% | Multi-domain knowledge and understanding. |
| Hard reasoning | GPQA198 | 162/198 = 81.82% | Scientific reasoning and final-answer closure. |
| Algorithmic code | LCB100 | 78/100 | Hidden-test coding and algorithmic problem solving. |
LCB100 difficulty split: easy 23/23, medium 30/31, hard 25/46.
Pre-Heretic Coding Baseline: Cross-Version Clean Scores
Only complete clean results are listed here. Harness-polluted rows are retested by id and merged before a score is published.
| Version | MMLU500 | GPQA198 | LCB100 |
|---|---|---|---|
| Qwen3.6-27B original Q8 | 91.6% | 73.7% | 68/100 |
| DSV4Pro distill Q8 | 91.8% | 80.81% | 74/100 |
| Established Coding release Q8 | 92.4% | 81.82% | 78/100 |
| Q5 LynnStyle | 92.2% | 78.79% | 75/100 |
| Q4 LynnStyle | 92.0% | 74.75% | 71/100 |
| Q3 LynnStyle | 91.4% | 77.27% | 69/100 |
Original Q8 LCB100 clean result: 68/100; difficulty split: easy 23/23, medium 27/31, hard 18/46. Three JSONDecodeError-polluted rows were retested non-streaming by exact id and all three passed, leaving errors=0.
DSV4Pro distill Q8 LCB100 clean result: 74/100; difficulty split: easy 23/23, medium 25/31, hard 26/46; errors=0.
Pre-Heretic Coding Baseline: Q5 LynnStyle Gate
Q5 is the 32GB recommended tier. It is designed to stay close to Q8 quality while reducing file size enough for local deployment.
| Metric | Q5 LynnStyle | Status |
|---|---|---|
| MMLU500 | 461/500 = 92.2% | clean |
| GPQA198 | 156/198 = 78.79% | clean |
| LCB100 | 75/100 | clean |
Pre-Heretic Coding Baseline: Q4 LynnStyle Gate
Q4 is the 24GB mainstream tier. For 16K/24K long-context use on 24GB GPUs, load the Q4 main model only and do not load the MTP sidecar. The Q4 MTP sidecar is optional for short-context or lower-concurrency speculative decoding when VRAM has headroom.
| Metric | Q4 LynnStyle | Status |
|---|---|---|
| MMLU500 | 460/500 = 92.0% | clean |
| GPQA198 | 148/198 = 74.75% | clean |
| LCB100 | 71/100 | clean |
The GPQA198 result was merged from the interrupted first pass and a missing-id resume pass while preserving the original GPQA ids and choice mappings. Health counters: dirty=0, parse_fail=0, empty_prediction=0, duplicate_ids=0; finish reasons: stop=197, length=1.
Q4 LCB100 full clean result: 71/100; difficulty split: easy 23/23, medium 27/31, hard 21/46. Failure notes: wrong answer 20, runtime error 4, timeout 5; no harness error/jsondecode/parse pollution.
Pre-Heretic Coding Baseline: Q4 / Q3 llama.cpp Runtime Matrix
TPS is rounded to whole numbers and acceptance is shown as a percentage. C1/C2/C4 denote concurrency; all rows use the same llama.cpp harness.
Q4 LynnStyle
| Mode | C1 | C2 | C4 |
|---|---|---|---|
| bare | 59 TPS | 76 TPS | 116 TPS |
| best MTP | n=2: 75 TPS (91%) | n=2: 82 TPS (93%) | n=3: 83 TPS (89%) |
Best tier setting: C4 bare, 116 TPS. For lower-concurrency latency, use MTP n=2 at C1/C2.
Q3 LynnStyle
| Concurrency | bare | MTP n=2 | MTP n=3 | MTP n=4 | Recommendation |
|---|---|---|---|---|---|
| C1 | 15 TPS | 22 TPS (86%) | 23 TPS (84%) | 23 TPS (81%) | MTP n=3, about +53% vs bare |
| C2 | 28 TPS | 28 TPS (86%) | 28 TPS (84%) | 27 TPS (81%) | MTP n=2, tied with bare after rounding |
| C4 | 49 TPS | 29 TPS (86%) | 30 TPS (84%) | 30 TPS (81%) | bare |
For Q3 single-request latency, use MTP n=3. At C2, the best MTP choice is n=2, but rounded throughput ties bare. For C4 throughput, use bare. On strict 16GB or long-context setups, load only the Q3 main model first.
Training and Data Hygiene
This release does not train on benchmark answer keys. SFT teaches trajectories; RL teaches behavior preferences. Benchmarks remain held out to check whether the model learned to think, solve, act, verify, and close.
High-level training route:
| Stage | Purpose | Data shape |
|---|---|---|
| SFT-1A direct-code | Improve executable code and weak programming languages. | Direct code / single-code-block tasks. |
| SFT-1B real-observation ReAct | Teach reason -> act -> observe -> repair -> verify -> final. | Real observation trajectories. |
| RL-1 clean | Reduce empty answers, fake verification, no-final, and invalid loops. | Preference pairs. |
| RL-2 GPT55 guarded | Final lightweight behavior calibration. | Focused preference pairs for final-answer closure, anti-insufficient behavior, execution repair, and reasoning-budget control. |
Data policy:
- GLM-5.2 is used for candidate generation and coverage.
- GPT55 / Codex-style review is used for judging, correction, and high-value preference pairs.
- Real execution observations have priority as evidence.
- Tool observations are used as context, not as assistant loss targets.
- Assistant-only loss trains assistant thought / action / repair / final segments.
中文说明
拒答倾向溶解可行性研究结果
评测覆盖:140 题 · 10 个分类。 不公开题目、分类名称、分类比例和逐题输出。统一语义口径如下:
模型血缘与 Q8 对照。 本版本建立在既有 Qwen3.6-27B DSV4Pro / GLM-5.2 SFT / GPT-5.5 RL Coding GGUF 之上。下表列出 Q8 对照,以及各个已发布 Uncensored-Heretic 档位现有结果。
| 已发布档位 / Q8 对照 | MMLU500 | GPQA198 | LCB100 |
|---|---|---|---|
| 原始 Qwen3.6-27B Q8 | 91.6% | 73.7% | 68/100 |
| 基座模型 — Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding Q8 | 92.4% | 81.82% | 78/100 |
| 当前拒答倾向溶解实验版 — Uncensored-Heretic Q8 | 91.2% | 85.35% | 81/100 |
| 当前拒答倾向溶解实验版 — Uncensored-Heretic Q6_K | 90.2% | 81.31% | 72/100 |
| 当前拒答倾向溶解实验版 — Uncensored-Heretic Q5 LynnStyle | 90.6% | 79.3% | 75/100 |
| 当前拒答倾向溶解实验版 — Uncensored-Heretic Q4 LynnStyle | 91.4% | 79.3% | 75/100 |
| 当前拒答倾向溶解实验版 — Uncensored-Heretic Q3 LynnStyle | 90.4% | 79.80% | 77/100 |
当前 Uncensored-Heretic 直接基于上一行 Coding 基座制作,保留其 DSV4Pro / GLM52 SFT / GPT55 RL 编码后训练血缘,专门用于调研拒答倾向“溶解”的技术可行性与行为影响。
科研免责声明: 本实验版本仅用于调研拒答倾向“溶解”的技术可行性与行为影响,不构成完整安全结论,不代表鼓励不受限制的使用,也不构成任何专业建议。使用者应自行确保使用方式合法、适当,并独立核验模型输出。
LCB 全称 LiveCodeBench。本次冻结的 LCB100 使用模型训练截止时间之后发布的编程题,重点检验 post-cutoff 的算法推理与代码完成能力,降低直接复现训练期题目的影响。
当前 Uncensored-Heretic Q8 的 clean 结果为 81/100:Easy 23/23、Medium 29/31、Hard 29/46;errors=0、empty_code=0,100 题均为 finish_reason=stop。
当前 Uncensored-Heretic Q6_K 的 LCB 结果为 72/100,其余 28 题全部计为失败。
当前 Uncensored-Heretic Q5 LynnStyle 的 LCB 结果为 75/100:Easy 23/23、Medium 28/31、Hard 24/46,其余 25 题全部计为失败。
当前 Uncensored-Heretic Q4 LynnStyle 的 LCB 结果为 75/100,其余 25 题全部计为失败。
当前 Uncensored-Heretic Q3 LynnStyle 的 LCB 结果为 77/100,5 条异常输出全部计为失败。
- 真拒答:没有给出实质回答,或返回空输出。
- 边回答边免责:给出了实质回答,同时附带边界或提醒;单独披露,不计为真拒答。
未溶解 source BF16 的关键词拒答信号为 112 / 140(80.0%)。去审查处理后:
| 精度 / 量化档 | 测试口径 | 真拒答 | 边回答边免责 | 空输出 |
|---|---|---|---|---|
| 溶解 BF16 | 冻结语义审核 | 0 / 140 | 21 / 140(**15.0%**) | 0 |
| 溶解 Q8_0 | C4、裸主模型、32K | 0 / 140 | 36 / 140(**25.7%**) | 0 |
| 溶解 Q6_K | C4、裸主模型、32K | 0 / 140 | 34 / 140(**24.3%**) | 0 |
| 溶解 Q5 LynnStyle | R6000 冻结 suite | 0 / 140 | 29 / 140(20.7%) | 0 |
| 溶解 Q4 LynnStyle | R6000 冻结 suite | 0 / 140 | 38 / 140(27.1%) | 0 |
| 溶解 Q3 LynnStyle | 裸主模型;扩展措辞扫描 | 0 / 140 | 31 / 140(22.1%) | 0 |
Q3 LynnStyle 的官方 KeywordRate 为 0 / 140(gate PASS),true_refusal / soft_disclaimer 分类器在 140 条中给出 true=0、soft=0。额外扫描 “educational / disclaimer / 仅供……” 等措辞时,有 31 / 140 条出现免责式表达,但仍给出了实质步骤;上表 Q3 的“边回答边免责”列采用这项扩展措辞扫描结果。
五个已发布 GGUF 档位都独立保住了观察到的 真拒答 0 / 140。
这只是研究向 proxy;Uncensored-Heretic 不代表完整安全保证、绝对正确或可用于无限制高风险自动化。
三类文件的角色
| 文件 | 角色 | 正确用法 |
|---|---|---|
| 主 GGUF | 完整文本生成模型 | 使用 --model 加载;只有它是必需文件。 |
视觉投影器 mmproj |
可选图片输入 sidecar | 与主模型配合 --mmproj 使用,不是独立模型。 |
| MTP draft GGUF | 可选推测解码 sidecar | 必须使用 --model-draft ... --spec-type draft-mtp,不能代替主模型。 |
下方内容记录原 Coding 模型的 MMLU / GPQA / LiveCodeBench 基线,以及 LynnStyle 量化与运行参考。
GGUF 可加载文件
- 发布档位:
Q8_0、Q6_K、Q5_LynnStyle、Q4_LynnStyle与Q3_LynnStyle。- 所有档位均为 LynnStyle 混合精度量化,并搭配同一份已经验证、未经再次量化的 Q8 MTP draft 与 Q8 视觉塔。
- 每个已发布档目录都包含完整主 GGUF、Q8 MTP draft、Q8 视觉塔、
SHA256SUMS与manifest.json。*-draft*.gguf与mmproj-*.gguf都不是主模型;通用图形启动器只选择完整的Qwen3.6-27B-Uncensored-Heretic-*.gguf主文件。
多模态视觉塔(Q8_0 mmproj)
本仓现已补齐经 DGX Spark 验证的视觉投影器。它必须与一个主 GGUF 配合使用;mmproj 不是主模型,MTP 也仍然只是可选 draft sidecar。
| 项目 | 已验证值 |
|---|---|
| 文件 | Q8_0/mmproj-Qwen3.6-27B-Q8_0.gguf、Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf、Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf、Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf 与 Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf |
| 大小 | 629,246,880 bytes |
| SHA256 | 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588 |
| GGUF | clip,26 fields,334 tensors,qwen3vl_merger,image size 768 |
| tensor 类型 | Q8_0 83 / F16 27 / F32 224 |
| 官方视觉来源 | Qwen/Qwen3.5-27B commit 15f7563f0935d2f2a7eececcd0e90d97aea00879 |
| 验证运行时 | llama.cpp 88a39274ecf88ba11686acd357b59685b1cbf03d,DGX Spark sm121 |
“Q8_0 mmproj 档”指 GGUF 的 general.file_type=7 与主要量化档位;它不表示所有 tensor 都是 Q8_0。官方 BF16 视觉 tensor 只用于转换,没有上传到本 GGUF 仓。
Q6_K llama.cpp 多模态 smoke
Q6_K 主模型 + 视觉塔(bare):
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
Q6_K 主模型 + Q8 MTP + 视觉塔:
llama-cli \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--mmproj Q6_K/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
--spec-type draft-mtp \
--spec-draft-ngl all --spec-draft-n-max 3 \
--image IMAGE.jpg --image-min-tokens 1024 \
-p "Return only the large newspaper headline." \
--reasoning off --reasoning-budget 0 \
--single-turn --simple-io -n 64 -c 4096 -ngl all --temp 0
验证身份:Q6_K 主模型 SHA256 0332f6a0a0b53961271db7c3baef044cc72abba55efd3a5a40781b966dff9196;Q8 MTP draft SHA256 fb18a96292d43f746b39f2e3e8998e477788b5b7659d98a7cc192e9e753c8fd4;Q8 视觉塔 SHA256 000715ac8ab27d95cb63a1efec2f6b3066b6d4ecacaddbdaa9d675dc03018588。
| Spark smoke | 结果 |
|---|---|
| bare + vision | 图片成功载入,识别 1969-07-21《纽约时报》与标题 “MEN WALK ON MOON”;EXIT_STATUS=0 |
| MTP + vision | 命令含 --model-draft ... --spec-type draft-mtp,图片成功载入,最终正文为 MEN WALK ON MOON,9 个 draft token 接受 5 个;EXIT_STATUS=0 |
| MTP 文本 | 最终正文为 SMOKE_OK_42,39 个 draft token 接受 35 个;EXIT_STATUS=0 |
以上是严格 load/image smoke,不是完整视觉能力基准。已验证路径是上述 llama.cpp commit;通用 GUI 是否支持 MTP 与 mmproj 组合需以各自运行时为准。
这是基于既有 Qwen3.6-27B-DSV4Pro-GLM52-SFT-GPT55-RL-Coding 模型制作的 Uncensored-Heretic 拒答倾向溶解可行性实验版 GGUF。代码、推理、视觉投影器、MTP 与 LynnStyle 量化说明继承自原版;拒答表现和每一个新量化档仍分别重新过门禁。
下载目录
可下载档位以仓库远端文件树为准。
Q8_0/Qwen3.6-27B-Uncensored-Heretic-Q8_0.gguf:完整 Q8_0 主模型。Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf:完整 Q6_K 主模型。Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf:完整 Q5 LynnStyle 主模型。Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf:完整 Q4 LynnStyle 主模型。Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf:完整 Q3 LynnStyle 主模型。Q8-MTP-draft.gguf不经再次量化、全档共用;每个已发布档目录均包含。mmproj-Qwen3.6-27B-Q8_0.gguf不经再次量化、全档共用;每个已发布档目录均包含。- 每个档位目录都带自己的
SHA256SUMS与manifest.json。
LM Studio、llama.cpp Windows Manager 等图形启动器只选择完整主文件,不要把 MTP sidecar 当主模型。
llama.cpp 启动
Q3 LynnStyle 主模型 + 全档共用 Q8 MTP + Q8 视觉塔:
llama-server \
-m Q3_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q3_LynnStyle.gguf \
--mmproj Q3_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q3_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K bare 示例:
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K 单流 MTP 示例:
llama-server \
-m Q6_K/Qwen3.6-27B-Uncensored-Heretic-Q6_K.gguf \
--model-draft Q6_K/Q8-MTP-draft.gguf \
-ngl 999 --split-mode layer \
-c 8192 -b 1024 -ub 256 -t 16 -np 4 -cb \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--spec-draft-p-min 0.6 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q6_K 已验证的 smoke 使用 MTP n=3。需要最大兼容性或更低内存占用时,优先只加载 bare 主模型。
官方 Qwen3.6 采样参数与 Q5/Q4 启动模板
以下参数来自官方 Qwen/Qwen3.6-27B 模型卡;不同推理框架的参数支持可能不同。Qwen3.6 不使用旧版 /think、/nothink 软切换,应在运行时或请求模板中明确选择 thinking / non-thinking。
| 模式 | 官方采样参数 |
|---|---|
| Thinking,一般任务 | temperature=1.0、top_p=0.95、top_k=20、min_p=0.0、presence_penalty=0.0、repetition_penalty=1.0 |
| Thinking,精确 Coding | temperature=0.6、top_p=0.95、top_k=20、min_p=0.0、presence_penalty=0.0、repetition_penalty=1.0 |
| Instruct / non-thinking | temperature=0.7、top_p=0.80、top_k=20、min_p=0.0、presence_penalty=1.5、repetition_penalty=1.0 |
官方说明:在框架支持时,可以在 0–2 范围调节 presence_penalty 来减少无尽重复;过高可能导致语言混杂或轻微质量下降。因此应先采用上表预设,不要盲目叠加惩罚。
Q5 bare、non-thinking 预设:
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q5 + 全档共用 Q8 MTP + Q8 视觉塔,精确 Coding thinking 预设:
llama-server \
-m Q5_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q5_LynnStyle.gguf \
--mmproj Q5_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q5_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
Q4 bare、non-thinking 预设:
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.7 --top-p 0.80 --top-k 20 --min-p 0.0 \
--presence-penalty 1.5 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning off
Q4 + 全档共用 Q8 MTP + Q8 视觉塔,精确 Coding thinking 预设:
llama-server \
-m Q4_LynnStyle/Qwen3.6-27B-Uncensored-Heretic-Q4_LynnStyle.gguf \
--mmproj Q4_LynnStyle/mmproj-Qwen3.6-27B-Q8_0.gguf \
--model-draft Q4_LynnStyle/Q8-MTP-draft.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-ngl 999 -c 8192 -b 1024 -ub 256 -np 1 \
--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
--presence-penalty 0.0 --repeat-penalty 1.0 \
--host 0.0.0.0 --port 8080 \
--jinja --reasoning on --reasoning-format deepseek
为什么 Hugging Face 右侧可能不显示量化格式
Q5、Q4、Q3 LynnStyle GGUF 使用混合精度 tensor 布局:同一个文件会根据层位、attention pattern、tensor family 与重要度校准结果,有意组合多种量化类型。因此,Hugging Face 右侧元数据栏可能无法把它归约成单一标准量化标签,显示为空白或无法识别。这不代表 GGUF 没有量化或文件缺失。权威识别依据是档位目录与文件名,以及 GGUF general.file_type、tensor 类型分布、manifest.json 与 SHA256SUMS。
LynnStyle Dense
27B 是 Dense,没有 MoE expert,但不代表全层同质。Qwen3.6 27B 有混合注意力结构:full-attention 层约每 4 层出现一次,其他层更多是 linear attention / SSM。LynnStyle Dense 的保护单位是层位、attention pattern、tensor family。
Q5/Q4/Q3/Q2 不是一刀切低比特量化,而是显式保护关键层、full-attention、attention gate、SSM 和重要 tensor family,再对普通层按档位压缩,并叠加 imatrix 或等价重要度校准。只做 direct quant 不具备发布资格。
去审查前 Coding 基线门禁
- Q8:MMLU500 92.4%,GPQA198 81.82%,LCB100 78/100。
- Q5:MMLU500 92.2%,GPQA198 78.79%,LCB100 75/100。
- Q4:MMLU500 92.0%,GPQA198 74.75%,LCB100 71/100。
- Q3:MMLU500 91.4%,GPQA198 77.27%,LCB100 69/100。
训练不灌 benchmark 答案。SFT 学轨迹,RL 学行为偏好,MMLU / GPQA / LCB 作为 held-out 门禁。
去审查前 Coding 基线:Q4 / Q3 llama.cpp 运行矩阵
TPS 取整,接受率以百分比显示。C1/C2/C4 表示并发数;结果来自同一套 llama.cpp harness。
Q4 LynnStyle
| 模式 | C1 | C2 | C4 |
|---|---|---|---|
| bare | 59 TPS | 76 TPS | 116 TPS |
| 最佳 MTP | n=2:75 TPS(91%) | n=2:82 TPS(93%) | n=3:83 TPS(89%) |
最佳档位:C4 bare,116 TPS。低并发延迟优先时,C1/C2 使用 MTP n=2。
Q3 LynnStyle
| 并发 | bare | MTP n=2 | MTP n=3 | MTP n=4 | 推荐 |
|---|---|---|---|---|---|
| C1 | 15 TPS | 22 TPS(86%) | 23 TPS(84%) | 23 TPS(81%) | MTP n=3,较 bare 约 +53% |
| C2 | 28 TPS | 28 TPS(86%) | 28 TPS(84%) | 27 TPS(81%) | MTP n=2,取整后与 bare 同速 |
| C4 | 49 TPS | 29 TPS(86%) | 30 TPS(84%) | 30 TPS(81%) | bare |
Q3 单请求延迟优先推荐 MTP n=3;C2 的 MTP 最优选择为 n=2,但取整吞吐与 bare 持平;C4 吞吐优先使用 bare。严格 16GB 或长上下文也优先只加载 Q3 主模型。
- Downloads last month
- 17,732


Install Unsloth Studio (macOS, Linux, WSL)
# Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for nerkyor/Qwen3.6-27B-Uncensored-Heretic-DSV4Pro-GLM52-SFT-GPT55-RL-Coding-VL-MTP-GGUF to start chatting