---
title: Gemma 4 31B Heretic FP8
emoji: 🧠
colorFrom: purple
colorTo: indigo
sdk: gradio
sdk_version: 6.20.0
python_version: "3.12.12"
app_file: app.py
pinned: false
license: apache-2.0
short_description: Chat with quantized Gemma 4 31B Heretic.
startup_duration_timeout: 1h
models:
  - 0ppxnhximxr/gemma-4-31B-it-heretic-FP8-Dynamic
tags:
  - chat
  - multimodal
  - gemma4
  - quantized
  - fp8
---

# Gemma 4 31B Heretic — FP8 Demo

Chat with a pre-quantized FP8 version of
[`coder3101/gemma-4-31B-it-heretic`](https://huggingface.co/coder3101/gemma-4-31B-it-heretic).
The demo supports typed text, UTF-8 text-file attachments, image input, token
streaming, an optional system prompt, and configurable sampling. Attach up to
two files per message; text files are limited to 64 KB.

## Runtime

- Checkpoint: [`0ppxnhximxr/gemma-4-31B-it-heretic-FP8-Dynamic`](https://huggingface.co/0ppxnhximxr/gemma-4-31B-it-heretic-FP8-Dynamic)
- Quantization: FP8 dynamic weights and activations in compressed-tensors format
- Hardware: Hugging Face ZeroGPU, xlarge request size
- Context guard: 16,384 input tokens plus up to 1,536 generated tokens
- Default response limit: 1,024 generated tokens

The model weights are fetched only by the Hugging Face Space runtime. Local
setup does not download model weights until `run.bat` starts the real app.

## Local utilities

- `setup.bat` creates a Python 3.12 virtual environment and installs packages.
- `run.bat` starts the Gradio app.
- `settings.bat` opens a local browser settings page with a Save button.

The model is large and needs a compatible NVIDIA GPU for local inference.

## Important

This checkpoint is intentionally refusal-reduced. Its output can be incorrect,
biased, offensive, or unsafe. Do not use it as a source of professional,
medical, legal, financial, or safety-critical advice.