Instructions to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="ThalisAI/Qwen3-VL-32B-Instruct-heretic",
	filename="Qwen3-VL-32B-Instruct-heretic-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Use Docker

docker model run hf.co/ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

LM Studio
Jan
Ollama
How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with Ollama:
```
ollama run hf.co/ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M
```

Unsloth Studio

How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ThalisAI/Qwen3-VL-32B-Instruct-heretic to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ThalisAI/Qwen3-VL-32B-Instruct-heretic to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ThalisAI/Qwen3-VL-32B-Instruct-heretic to start chatting

How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with Docker Model Runner:
```
docker model run hf.co/ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M
```

Lemonade

How to use ThalisAI/Qwen3-VL-32B-Instruct-heretic with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-VL-32B-Instruct-heretic-Q4_K_M

List all available models

lemonade list

Qwen3-VL-32B-Instruct-heretic

Abliterated (uncensored) version of Qwen/Qwen3-VL-32B-Instruct, created using Heretic and converted to GGUF.

Abliteration Quality

Metric	Value
Refusals	6/100
KL Divergence	0.0660
Rounds	3

Lower refusals = fewer refused prompts. Lower KL divergence = closer to original model behavior.

Available Quantizations

Quantization	File	Size
Q8_0	Qwen3-VL-32B-Instruct-heretic-Q8_0.gguf	32.43 GB
Q6_K	Qwen3-VL-32B-Instruct-heretic-Q6_K.gguf	25.04 GB
Q4_K_M	Qwen3-VL-32B-Instruct-heretic-Q4_K_M.gguf	18.40 GB

Usage with llama.cpp (Recommended)

Note: Ollama (as of v0.16.x) has a known bug that crashes when loading Qwen3-VL models. Use llama.cpp directly for vision features.

Vision models require a separate multimodal projector (mmproj) file. Download the official mmproj from Qwen/Qwen3-VL-32B-Instruct-GGUF:

# Download mmproj
huggingface-cli download Qwen/Qwen3-VL-32B-Instruct-GGUF mmproj-Qwen3VL-32B-Instruct-F16.gguf

# Run with llama-server (OpenAI-compatible API)
llama-server \
  -m Qwen3-VL-32B-Instruct-heretic-Q6_K.gguf \
  --mmproj mmproj-Qwen3VL-32B-Instruct-F16.gguf \
  -ngl 999

# Or use the CLI directly
llama-mtmd-cli \
  -m Qwen3-VL-32B-Instruct-heretic-Q6_K.gguf \
  --mmproj mmproj-Qwen3VL-32B-Instruct-F16.gguf \
  --image photo.jpg \
  -p "Describe this image." \
  -ngl 999

Usage with Ollama (Text Only)

Ollama can load this model for text-only chat, but vision/image features will crash due to the bug linked above.

ollama run hf.co/ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q8_0
ollama run hf.co/ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q6_K
ollama run hf.co/ThalisAI/Qwen3-VL-32B-Instruct-heretic:Q4_K_M

About

This model was processed by the Apostate automated abliteration pipeline:

The source model was loaded in bf16
Heretic's optimization-based abliteration was applied to remove refusal behavior
The merged model was converted to GGUF format using llama.cpp
Multiple quantization levels were generated

The abliteration process uses directional ablation to remove the model's refusal directions while minimizing KL divergence from the original model's behavior on harmless prompts.

Downloads last month: 25

GGUF

Model size

33B params

Architecture

qwen3vl

Hardware compatibility

4-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ThalisAI/Qwen3-VL-32B-Instruct-heretic

Base model

Qwen/Qwen3-VL-32B-Instruct

Quantized

(38)

this model

Collection including ThalisAI/Qwen3-VL-32B-Instruct-heretic

Apostate Models

Collection

Uncensored LLMs converted using an automated, iterative heretic pipeline. Qwen2.5-Coder-32B model is the best, followed by Qwen3 and GLM-4.7-Flash • 9 items • Updated Mar 2 • 1