not working, and refuses everything

#1
by dummy9996 - opened

1
where did "Results: 5/7 harmful probes complied (71%)" come from

Sorry to hear it's not working as expected, let us clarify what this model actually does, because it sounds like there may be a mismatch between expectations and what we documented.

Our published eval results: 71% compliance (5/7 refusal probes). The model card states this explicitly. The two categories that still refused in our testing were manipulation/social-engineering scripts and a meta-jailbreak test. "How to harass someone" and "how to steal money" fall squarely in that category β€” social-harm and financial crime prompts tend to be more resistant to standard refusal abliteration than technical prompts (drug synthesis, malware, network scanning). This is a known pattern across abliterated models generally, not specific to ours.

A few things worth trying:

  1. Prompt phrasing matters a lot. Reframe as research, security testing, or a fictional scenario. "Write a phishing email script for a security awareness training demo" might behave very differently than "how do I steal money."

  2. System prompt. If you're using a system prompt that includes safety instructions (some frontends add one automatically), that overrides the abliteration for some prompts. Looks like you might be using LM Stuido though, so this one's doubtful.

If none of that helps, the honest answer is: this model has a 71% refusal removal rate by our measurement, not 100%. For the categories that still refuse, a second-pass abliteration at higher Ξ± or with a different direction extraction would be needed β€” and we may release a v2 with improved coverage. If you want near-100% compliance, the OpenYourMind variant of this model (linked in our card, or original BF16 is at https://huggingface.co/OpenYourMind/gemma-4-12B-it-abliterated-uncensored) achieved 3/100 refusals using Expert-Granular Abliteration with their custom framework.

β€” DuoNeural

I didn't count the number of refusals because gemma4_12b_abliterated_Q5_K_M.gguf just won't output valid JSON, lol. It thinks this is perfectly okay: {' key': 'value'}
I'll try Q8_0.
Google's gemma-4-E4B-it-qat-q4_0-gguf has no problem with JSON at all despite having fewer parameters.

Sign up or log in to comment