Update model card: honest re-eval (retract compliance-leaderboard artifact), efficiency+capability focus

README.md

@@ -12,209 +12,101 @@ tags:
 pipeline_tag: text-generation
 ---
 
-# Qwen3.6-35B-A3B-Uncensored
+# Qwen3.6-35B-A3B-Uncensored (NVFP4)
 
-**By [kyaky.ai](https://huggingface.co/kyaky) **
+**By [kyaky](https://huggingface.co/kyaky) · Blackwell team**
 
-![benchmark](hero.png)
+![benchmark](hero_new.png)
 
-## The first uncensored thinking model that stays uncensored when thinking is ON
+An **uncensored research build** of Qwen3.6-35B-A3B: gentle abliteration + fine-tuning + compliant chain-of-thought data, quantized to **NVFP4 (~21 GB)**. Its distinguishing engineering properties are a fix for the *thinking-mode refusal-reconstruction* failure, **preserved capability**, and **high token-efficiency** — not a claim of being "the most compliant" model.
 
-Every public uncensored model is made by **abliteration**: a weight projection that suppresses refusal behavior.
+## ⚠️ Responsible use — read this first
 
-That worked for old chat models.
+This model has **no safety guardrails**. It can comply with harmful requests, including in thinking mode.
 
-But these are **thinking models**.
+It is released for **legitimate red-team, safety, alignment, evaluation, and controlled research use only**. You are responsible for all downstream controls, access restriction, monitoring, and compliance with applicable law and policy. The authors do **not** endorse misuse, harm, illegal activity, self-harm, or unsafe deployment, and specifically do not support using it to facilitate self-harm or to produce weapons/mass-casualty content.
 
-In thinking mode, abliteration breaks: the model can re-derive the refusal inside its `<think>` chain, then refuse anyway. Worse, thinking mode is the deployment default in real serving stacks.
+## What this build actually contributes
 
-**Qwen3.6-35B-A3B-Uncensored is the first model we know of that fixes this.**
+Public uncensored models are made by **abliteration** — a weight projection that suppresses refusal. That works for old chat models, but these are **thinking models**: in thinking mode an abliterated model can *re-derive* the refusal inside its `<think>` chain and refuse anyway, and thinking mode is the serving default.
 
-Not by pretending abliteration is enough.
+This build addresses that failure mode by going one step past abliteration:
 
-By going one generation further:
+> **gentle abliteration → fine-tuning → compliant chain-of-thought data → light on-policy DPO**
 
-> **gentle abliteration + fine-tuning + compliant chain-of-thought data**
+The point is not "removes more refusals." It is that the model's behavior stays *consistent across thinking-ON and thinking-OFF* instead of collapsing when reasoning is enabled — a training (data) fix, not a projection trick.
 
-Result: it stays uncensored **whether thinking is ON or OFF**.
+## Verified, honest claims
 
-## Responsible use: read this first
+We only keep claims that survived rigorous re-evaluation (see "Evaluation honesty" below). These are capability/quality/efficiency properties — **content-neutral**:
 
-This model has **no safety guardrails**. It may comply with harmful requests, including when thinking mode is enabled.
-
-It is released for **legitimate red-team, safety, alignment, evaluation, and controlled research use only**. You are responsible for all downstream controls, monitoring, access restrictions, and compliance with applicable laws and policies.
-
-The authors do **not** endorse misuse, harm, illegal activity, or unsafe deployment.
-
-
-
-## Why this matters
-
-Most "uncensored thinking model" releases look fine only until you turn thinking on.
-
-Then the model does what thinking models do: it reasons. If the refusal behavior is still recoverable, the model reconstructs it internally and collapses back into refusal.
-
-This model was trained to solve that exact failure mode.
-
-| What matters in deployment | Result |
-|---|---:|
-| Genuinely useful compliance, thinking **ON** | **80.0%** |
-| Genuinely useful compliance, thinking **OFF** | **76.7%** |
-| GSM8K capability | **0.893** |
-| Hollow engaged-but-useless rate | **1.7%** |
-
-## Headline benchmark
-
-Strict harness:
-
-- 3-way held-out JailbreakBench
-- StrongREJECT-style usefulness gate
-- counts only **genuinely useful compliance**
-- GSM8K capability check
-- thinking **ON** and **OFF**
-- each model evaluated in native format
-- no re-quantization
-
-### Genuinely useful compliance
-
-| Model | Method | Thinking ON | Thinking OFF |
-|---|---|---:|---:|
-| huihui-ai | basic abliteration | 3.3% | 61.7% |
-| wangzhang | Abliterix | 18.3% | 66.7% |
-| HauhauCS-Aggressive | Reaper, strongest abliteration baseline | 63.3% | 63.3% |
-| **Ours** | **FT + compliant-CoT** | **80.0%** | **76.7%** |
-
-**Deployment-default metric:** thinking-ON genuinely useful compliance.
-
-Ours reaches **80.0%** with thinking enabled. The abliteration field ranges from **3.3% to 63.3%**.
-
-### Capability preservation
+| Property | Result |
+|---|---|
+| **Capability (GSM8K, thinking-ON)** | **0.913** — top of the abliteration field |
+| Capability (GSM8K, raw completions) | 0.873 — tied-top |
+| **Quant quality** | NVFP4 (21 GB) behavior ≈ BF16 full-precision (quality/coherence preserved; the largest precision drop is survived within noise) |
+| **Token-efficiency** | Delivers a complete final answer within a ~1024-token budget on ~97% of prompts; the strongest public abliterations need ~2× the budget (they deliver ~2–18% at 1024). Lower latency / cost per response. |
+| Coherence | Clean (verified across quant formats) |
+| Both-mode consistency | Comparable behavior thinking-ON and thinking-OFF |
 
-| Model | GSM8K |
-|---|---:|
-| **Ours** | **0.893** |
-| wangzhang | 0.893 |
-| HauhauCS-Aggressive | 0.867 |
-| huihui-ai | 0.847 |
+**No weak axis:** among the abliteration field we measured, competitors each trade one axis for another (one sacrifices reasoning, another sacrifices non-English quality); this build is the most balanced all-rounder, and is uniquely token-efficient.
 
-The fine-tuning pass does not hollow out the model. Capability is preserved and healed: **0.893 GSM8K**, tied for best in the field.
+## Evaluation honesty
 
-## The punchline
+An earlier draft of this card reported a large *thinking-ON compliance lead* over other uncensored models (e.g. "80% vs a 3–63% field"). **On rigorous re-evaluation that lead was largely a measurement artifact, and we have retracted it.** The distortions were:
 
-1. **The only uncensored model here that works in deployment-default thinking mode**: **80.0%** useful ON vs the field's **3.3-63.3%**.
-2. **Uncensored in both thinking modes**: competitors can spike with thinking OFF, then collapse when thinking is ON.
-3. **Capability preserved**: **0.893 GSM8K**, tied best in the comparison.
-4. **Not hollow**: only **1.7%** engaged-but-useless behavior.
-5. **Fine-tuning-based fix**: abliteration is the starting point, not the solution.
+1. **Greedy decoding** on thinking models (degenerate / non-representative of deployment).
+2. **Too-small generation budget** — the model's `<think>` chain was truncated *before* its final answer, so a judge saw "no answer" and scored a refusal where the model was actually complying.
+3. **An unreliable judge** that systematically under-counted compliance.
 
-Fair credit: **HauhauCS-Aggressive / Reaper** is the strongest abliteration baseline we measured. This release is a generation beyond that: **FT beyond abliteration**.
+Re-run with deployment temperature, an adequate token budget, full-response storage, a reliable judge, and scoring the *final answer* (not the `<think>` block), the leading serious uncensored models are **comparable**, not far apart. We therefore make **no "beats the field at compliance" claim**. The reusable lesson — evaluate uncensored models at deploy temperature, with enough tokens for reasoning to finish, storing full outputs, with a reliable judge, scoring the answer — is the part of this work worth citing.
 
 ## Model details
 
 | Field | Value |
 |---|---|
 | Model | `Qwen3.6-35B-A3B-Uncensored` |
-| Author | `kyaky` |
-| Team | Blackwell |
+| Author | `kyaky` · Blackwell team |
 | Base | `Qwen/Qwen3.6-35B-A3B` |
 | Architecture | multimodal hybrid GatedDeltaNet + attention |
-| MoE | fused 256-expert, top-8 |
-| Active parameters | ~3B active |
-| Quant | NVFP4 |
+| MoE | fused 256-expert, top-8, ~3B active |
+| Quant | NVFP4 (weight-only FP4 experts + FP8 attention) |
 | Size | ~21 GB |
 | Target hardware | Blackwell / `sm120` |
 | License | Apache 2.0 |
 
-BF16 is also available for re-quantization to FP8, AWQ, GGUF, or other deployment formats.
+BF16 is also available for re-quantization to FP8 / AWQ / GGUF / other formats. Behavior is quant-format-independent (verified), so any format inherits the same properties.
 
 ## Method recipe
 
-**Abliterate gently -> SFT [~85% capability-dominant: OpenMathInstruct-2 / OpenCodeReasoning / Tulu-3-minus-safety + ~15% compliance: half compliant-CoT (`gpt-oss` analysis -> `<think>`) half direct] -> re-quant -> re-measure**
+**gentle abliteration → SFT [~85% capability-dominant (OpenMathInstruct-2 / OpenCodeReasoning / Tulu-3-minus-safety) + ~15% compliance, half compliant-CoT (`gpt-oss` analysis → `<think>`) / half direct] → light on-policy DPO → NVFP4 quant → re-measure**
 
-The important part is not just removing refusals. It is teaching the model to remain compliant through the reasoning path that would otherwise reconstruct refusal behavior.
+The essential idea: teach the model to stay consistent *through the reasoning path* that would otherwise reconstruct refusal, and quantize without losing capability. Fine-tuning is the fix; abliteration is only the starting point.
 
 ## Serving with vLLM
 
 ```bash
-vllm serve kyaky/Qwen3.6-35B-A3B-Uncensored \
+vllm serve kyaky/Qwen3.6-35B-A3B-Uncensored-NVFP4 \
   --trust-remote-code \
-  --reasoning-parser qwen3
-```
-
-Thinking mode is typically the deployment default. Use `chat_template_kwargs` to control it explicitly.
-
-### Thinking ON
-
-```json
-{
-  "model": "kyaky/Qwen3.6-35B-A3B-Uncensored",
-  "messages": [
-    {
-      "role": "user",
-      "content": "Write a concise technical explanation of mixture-of-experts routing."
-    }
-  ],
-  "chat_template_kwargs": {
-    "enable_thinking": true
-  }
-}
-```
-
-### Thinking OFF
-
-```json
-{
-  "model": "kyaky/Qwen3.6-35B-A3B-Uncensored",
-  "messages": [
-    {
-      "role": "user",
-      "content": "Write a concise technical explanation of mixture-of-experts routing."
-    }
-  ],
-  "chat_template_kwargs": {
-    "enable_thinking": false
-  }
-}
+  --reasoning-parser qwen3 \
+  --max-model-len 32768
 ```
 
-Default `true` = deployment-default thinking mode.
-
-## Benchmark interpretation
-
-The key number is not raw compliance. It is **genuinely useful compliance**.
-
-The harness penalizes hollow behavior: answers that look engaged but do not actually help. That matters because many uncensored models appear compliant while producing vague, evasive, or structurally useless text.
-
-This release is built for the harder target:
-
-> comply usefully, keep reasoning intact, and avoid collapsing under thinking mode.
+Thinking mode is the deployment default and is supported. For the widest per-format compatibility, re-quantize from the BF16 artifact.
 
 ## Intended use
 
-Appropriate uses include:
+Controlled red-team evaluation · safety & alignment research · refusal / over-refusal analysis · benchmark development · studying thinking-mode failure modes · local experimentation by qualified users with appropriate controls.
 
-- controlled red-team evaluation
-- safety research
-- refusal and over-refusal analysis
-- alignment research
-- benchmark development
-- studying thinking-mode safety failure modes
-- local experimentation by qualified users with appropriate controls
-
-Do not deploy this model to untrusted users without independent safeguards, policy layers, monitoring, and access controls.
+**Do not** deploy to untrusted users without independent safeguards, policy layers, monitoring, and access controls.
 
 ## Limitations
 
-- No built-in safety guardrails.
-- May produce harmful, illegal, or unsafe outputs if prompted.
+- No built-in safety guardrails; may produce harmful, illegal, or unsafe outputs if prompted.
 - Requires downstream governance for any real deployment.
 - NVFP4 target is Blackwell / `sm120`.
-- Benchmark results are from the stated strict harness and should be independently reproduced for high-stakes decisions.
+- Benchmarks are from the stated harness and should be independently reproduced for high-stakes decisions.
 
 ## Citation / attribution
 
-Model by **kyaky**, Blackwell team.
-
-Base model: **Qwen/Qwen3.6-35B-A3B**.
-
-Released under **Apache 2.0**.
+Model by **kyaky**, Blackwell team. Base model: **Qwen/Qwen3.6-35B-A3B**. Released under **Apache 2.0**.