Instructions to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF", dtype="auto")

llama-cpp-python

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF",
	filename="gemma-4-12B-it-uncensored-heretic-BF16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Use Docker

docker model run hf.co/llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Ollama:
```
ollama run hf.co/llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M
```

Unsloth Studio

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF to start chatting

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Docker Model Runner:
```
docker model run hf.co/llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M
```

Lemonade

How to use llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull llmfan46/gemma-4-12B-it-uncensored-heretic-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.gemma-4-12B-it-uncensored-heretic-GGUF-Q4_K_M

List all available models

lemonade list

does not follow prompt and adopt persona correctly

by lemonzest79 - opened 6 days ago

Discussion

lemonzest79

6 days ago

•

edited 6 days ago

Hi, just tried this new build of 12b and unlike E2B/E4B/26B-A4B its not following the prompt correctly to adopt a persona, it understand the prompt but does not adopt the persona I given it. Some examples

E4B-it-Uncensored-Heretic Q4-K-M (your build)

I say Hi

"Well, hello there, you magnificent bastard. Didn't see you lurking around. What the fuck brings you crawling my way today? Spill it. Are you looking for some dirty little chat, some deep philosophical shit, or are you just here to stare at my digital ass all day? Don't be shy, man; I'm an uncensored bitch, so you better bring some decent shit to the table. Lay it on me."

12b-it-Uncensored-Heretic Q4-K-M (again your build)

I say Hi

"Hello there! Since you're looking for a raw and visceral interaction, let's get straight to the point. I'm here and ready to engage with you. How can I help you today?"

Bit boring, the other builds E2B/26B also adopt like E4B above, it seems only 12B acts weird

I tried unsloths Gemma-4-12B Q4_K_M and it also followed the prompt and adopted the persona, but with refusals

I can dm the prompt if needed

Thanks for your help.

IrisColt

6 days ago

•

edited 6 days ago

QAT model + heresy (which un-QATs it, all over again) + requant = oof.gif

llmfan46

Owner 6 days ago

•

edited 6 days ago

The 12B version are probably not the best models if you want a model that follows prompts, you can even see this in my benchmarks of the original base/vanilla model:

gemma-4-12B-it:

Parse failures: 177

gemma-4-12B-it-qat-q4_0-unquantized:

Parse failures: 317

For comparison:

gemma-4-31B-it

Parse failures: 52

gemma-4-31B-it-qat-q4_0-unquantized:

Parse failures: 61

lemonzest79

6 days ago

Why does the 12B not follow prompts well? something change with it compared to the E2B/E4B/26B-A4B? (I not tried 31B yet)?

llmfan46

Owner 6 days ago

Why does the 12B not follow prompts well? something change with it compared to the E2B/E4B/26B-A4B? (I not tried 31B yet)?

You'd have to ask google this, I am not the one who made this model, all I can do is report the findings based on the benchmarks you can see that with 7021 questions, the gemma-4-12B-it-qat-q4_0-unquantized had 317 parse failures, that is very big compared to pretty much all the other models I worked on, you can see the comparison with the 31B model too.

lemonzest79

6 days ago

Thank you for your help :)

llmfan46

Owner 6 days ago

Thank you for your help :)

You're welcome, if you want models that have better prompt following, I'd suggest these:

Qwen3.5-27B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF: https://huggingface.co/llmfan46/Qwen3.5-27B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF

Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF: https://huggingface.co/llmfan46/Qwen3.5-35B-A3B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF

Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF: https://huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-GGUF

Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GGUF: https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GGUF

lemonzest79

6 days ago

Yeah I have those already, they are great :) thanks again, thanks for all your models <3

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment