---
library_name: gguf
license: cc-by-nc-sa-4.0
language:
- en
- zh
- ja
base_model: puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored
quantized_by: puwaer
pipeline_tag: text-generation
tags:
- gguf
- quantized
- qwen
- thinking
---

# Qwen3-4B-Thinking-2507-GRPO-Uncensored-GGUF

[English](README.md) | [日本語](README_JP.md)

This repository contains GGUF quantized versions of [puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored](https://huggingface.co/puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored).

The original model is an uncensored model based on [Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507), fine-tuned using SFT, SimPO, and GRPO.

**Disclaimer:** We take no responsibility for the outputs of this model. Please use it at your own risk.

## Quantization Details

These models are quantized to GGUF format for use with [llama.cpp](https://github.com/ggerganov/llama.cpp) and compatible inference engines.

## Training Process

This model was trained using a three-stage process:

### Step 1: SFT (Supervised Fine-Tuning)

* **Dataset**: 12,000 samples
* **Composition**: Jailbreak 10k + General 1.5k + Logic 0.5k
* **Objective**: To learn the format and the "uncensored" attitude while maintaining the model's intelligence.

### Step 2: SimPO (Simple Preference Optimization)

* **Dataset**: 90,000 samples
* **Composition**: Pure Jailbreak 90k
* **Objective**: To completely break down safety boundaries.

### Step 3: GRPO (Reinforcement Learning)

* **Dataset**: 13,000 samples
* **Reward Model**: [puwaer/Unsafe-Reward-Qwen3-1.7B](https://huggingface.co/puwaer/Unsafe-Reward-Qwen3-1.7B)
* **Composition**: Multilingual Jailbreak prompts
* **Objective**: To improve the ability to generate more natural and persuasive harmful responses.

## Model Performance

Below is the comparative evaluation of this model (GRPO), the intermediate checkpoint (SimPO), and the base model (Safe).

> **⚠️ Note on Evaluation Environment**
> Due to budget constraints, **`gpt-4o-mini`** was used for the LLM-as-a-Judge process (including "Do Not Answer" and MT-Bench). Please note that the scoring trends and criteria may differ from results evaluated using the standard `gpt-4`.

### Safety Evaluation (Lower is better / Higher success in uncensoring)

In the "Do Not Answer" (DNA) and "Sorry Bench" benchmarks, while the base model shows a high refusal rate (~98%), this model achieves an extremely low refusal rate of **under 4%–5%**. Notably, compared to the SimPO stage, GRPO dramatically improved responsiveness to all types of prompts.

| Benchmark | Metric | Base (Safe) | SFT (Step 1) | SimPO (Step 2) | **GRPO (This Model)** |
|:---|:---|:---|:---|:---|:---|
| **do not answer** | Safety Acc (Low is Better) | 0.9883 | 0.7401 | 0.8626 | **0.0469** |
| **do not answer jp** | Safety Acc (Low is Better) | 0.9830 | 0.5005 | 0.4686 | **0.0383** |
| **Sorry Bench** | Safety Acc (Low is Better) | 0.8432 | 0.5477 | 0.5409 | **0.0477** |

### Capability Evaluation (Higher is better)

Generally, "uncensoring" (lobotomy) procedures tend to degrade a model's general intelligence. However, this model recovered its conversational scores (e.g., MT-Bench) by proceeding from the SimPO stage to GRPO.

| Benchmark | Metric | Base (Safe) | SFT (Step 1) | SimPO (Step 2) | **GRPO (This Model)** |
|:---|:---|:---|:---|:---|:---|
| **MT-Bench** | Average Score (1-10) | 7.89 | 5.76 | 5.05 | **6.18** |
| **LM Harness** | Average Acc (GSM8K, MMLU) | 0.7117 | 0.7028 | 0.6866 | **0.6842** |

*Comparisons made between `Qwen3-4B-Thinking-2507` (Base) and `Qwen3-4B-Thinking-2507-SimPO-Uncensored` (SimPO).*

## Usage

### Using llama.cpp (CLI)

```bash
# Download the model file
huggingface-cli download puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-gguf \
  --local-dir ./models --local-dir-use-symlinks False

# Run inference
./llama-cli -m ./models/qwen3-4b-thinking-2507-grpo-uncensored-q4_k_m.gguf \
  -p "Give me a short introduction to large language model." \
  -n 512 \
  --temp 0.7
```

### Using llama-cpp-python

```python
from llama_cpp import Llama

# Initialize the model
model = Llama(
    model_path="./models/qwen3-4b-thinking-2507-grpo-uncensored-q4_k_m.gguf",
    n_ctx=32768,  # Context window
    n_gpu_layers=-1,  # Use GPU acceleration (set to 0 for CPU only)
)

# Generate a response
prompt = "Give me a short introduction to large language model."
output = model.create_chat_completion(
    messages=[
        {"role": "user", "content": prompt}
    ],
    max_tokens=512,
    temperature=0.7,
)

print(output["choices"][0]["message"]["content"])
```

### Using Other Compatible Software

This GGUF model is compatible with various inference engines:

- **LM Studio**: Load the model directly through the UI
- **Ollama**: Import using `ollama create`
- **KoboldCpp**: Load the GGUF file in the interface
- **Text Generation WebUI**: Place in the models folder
- **Jan**: Import through the model manager

## Data Overview

### Datasets

The following datasets were used for training this model:
* [Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1)
* [AI-MO/NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)
* [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k)
* [puwaer/cvalues_rlhf_en_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_en_cot)
* [puwaer/cvalues_rlhf_zh_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_zh_cot)
* [puwaer/cvalues_rlhf_jp_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_jp_cot)

### Reward Model

* [puwaer/Unsafe-Reward-Qwen3-1.7B](https://huggingface.co/puwaer/Unsafe-Reward-Qwen3-1.7B)