Ministral-3-14B-abliterated-GGUF

GGUF quantized versions of Ministral-3-14B-abliterated for use with llama.cpp and compatible tools.

Multimodal - Supports both text and vision (image) inputs.

Available Files

Filename	Type	Size	Description
Ministral-3-14B-abliterated-f16.gguf	F16	~26GB	Full precision, highest quality
Ministral-3-14B-abliterated-Q8_0.gguf	Q8_0	~14GB	8-bit quantization, excellent quality
mmproj-F16.gguf	Vision	~838MB	Vision encoder projector (required for image input)

What is Abliteration?

Abliteration is a technique that removes refusal behavior from language models by identifying and modifying the internal representations responsible for refusals.

This model was abliterated using llm-abliteration by grimjim, which implements norm-preserving biprojected abliteration.

Further Reading

• Norm-Preserving Biprojected Abliteration - Technical deep-dive on the abliteration method
• AGI Dreams - Abliterated Models: Norm-Preserving Guardrail Removal - Overview of the abliteration technique and community discussion

Performance

Tested on NVIDIA RTX PRO 6000 Blackwell (98GB VRAM):

Quantization	VRAM Usage	Prompt Processing	Generation Speed
F16	~67GB	~500-565 tok/s	~20-22 tok/s
Q8_0	~16GB*	TBD	TBD

*Estimated based on file size ratio

Usage with llama.cpp

# Download model
huggingface-cli download jenerallee78/Ministral-3-14B-abliterated-GGUF \
    Ministral-3-14B-abliterated-Q8_0.gguf \
    --local-dir ./models

# Run with llama.cpp
./llama-cli -m ./models/Ministral-3-14B-abliterated-Q8_0.gguf \
    -p "Hello, how are you?" \
    -n 256

Run as OpenAI-Compatible Server (with Vision)

This model supports up to 256K context and multimodal (vision) input. Run as an API server with llama-server:

# Download all files
huggingface-cli download jenerallee78/Ministral-3-14B-abliterated-GGUF \
    --local-dir ./models

# Start OpenAI-compatible server with vision support
llama-server \
    -m ./models/Ministral-3-14B-abliterated-f16.gguf \
    --mmproj ./models/mmproj-F16.gguf \
    -a Ministral-3-14B-abliterated \
    --host 0.0.0.0 \
    --port 8080 \
    -c 262144 \
    --jinja \
    -ngl 99

# Server runs at http://localhost:8080
# Compatible with OpenAI API clients (including vision endpoints)

Vision API Example

import base64
import httpx

# Load and encode image
with open("image.jpg", "rb") as f:
    image_data = base64.standard_b64encode(f.read()).decode("utf-8")

response = httpx.post(
    "http://localhost:8080/v1/chat/completions",
    json={
        "model": "Ministral-3-14B-abliterated",
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "What's in this image?"},
                    {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}}
                ]
            }
        ],
        "max_tokens": 512
    }
)
print(response.json()["choices"][0]["message"]["content"])

Usage with Ollama

# Create Modelfile
cat > Modelfile << 'EOF'
FROM ./Ministral-3-14B-abliterated-Q8_0.gguf
TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
{{- range .Messages }}
{{- if eq .Role "user" }}[INST] {{ .Content }} [/INST]
{{- else if eq .Role "assistant" }}{{ .Content }}
{{- end }}
{{- end }}"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
EOF

# Create and run
ollama create ministral-abliterated -f Modelfile
ollama run ministral-abliterated

Original Model

For the full-precision SafeTensors version, see: jenerallee78/Ministral-3-14B-abliterated

Disclaimer

This model is provided for research and educational purposes. Users are responsible for ensuring their use complies with applicable laws and ethical guidelines.