README.md

---
pipeline_tag: image-to-image
library_name: diffusers
tags:
  - FLUX
  - FLUX.2
  - Klein
  - quantization
  - svdquant
  - nunchaku
  - fp4
  - int4
base_model: black-forest-labs/FLUX.2-klein-4B
base_model_relation: quantized
license: apache-2.0
---

# Model Card (SVDQuant / Nunchaku)

> **Language**: English | [中文](README_CN.md)

## Model and upstream

- **Quantized weights repo**: [`tonera/FLUX.2-klein-4B-Nunchaku`](https://huggingface.co/tonera/FLUX.2-klein-4B-Nunchaku)
- **Official full-precision source**: [`black-forest-labs/FLUX.2-klein-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
- **Quantized Transformer in this repo**: `svdq-<precision>_r32-FLUX.2-klein-4B-Nunchaku.safetensors`; use `nunchaku.utils.get_precision()` for `<precision>` (commonly `fp4` or `int4`) so the file name matches your environment and Nunchaku build

## Option 1 (for users comfortable with coding)

- **Engine**: [vitoom-nunchaku](https://huggingface.co/tonera/vitoom-nunchaku) — community-maintained Nunchaku build with **FLUX.2 Klein** support (FP4/INT4 SVDQuant inference)
- **Framework**: Diffusers with `Flux2KleinPipeline` support (official examples install from source):

```bash
pip install "git+https://github.com/huggingface/diffusers.git"
```

### Install vitoom-nunchaku (recommended)

Upstream [Nunchaku](https://github.com/nunchaku-ai/nunchaku) has not merged FLUX.2 Klein support for a long time ([PR #926](https://github.com/nunchaku-ai/nunchaku/pull/926) still pending). **Do not copy patch files manually.** Install the prebuilt wheel that matches your platform, Python, and CUDA:

1. Install matching **PyTorch 2.11** (cu128 or cu130) first.
2. Download and install a wheel from **[tonera/vitoom-nunchaku](https://huggingface.co/tonera/vitoom-nunchaku)**.

Example (x86_64, Python 3.11, CUDA 13.0):

```bash
pip install torch==2.11.* torchvision==0.26.* torchaudio==2.11.* \
  --index-url https://download.pytorch.org/whl/cu130

hf download tonera/vitoom-nunchaku \
  nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl \
  --local-dir ./wheels

pip install ./wheels/nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl
```

For other platforms (cu128, cp310, ARM64 aarch64), see the wheel table in the [vitoom-nunchaku README](https://huggingface.co/tonera/vitoom-nunchaku).

Verify:

```bash
python -c "import nunchaku; from nunchaku import NunchakuFlux2Transformer2DModel; print(nunchaku.__version__)"
```

## Minimal example (image-to-image + quantized Transformer)

Assumes **vitoom-nunchaku** is installed and weights are available locally or at `tonera/FLUX.2-klein-4B-Nunchaku`; set `REPO` to your directory or Hugging Face model id.

```python
import torch
from diffusers import Flux2KleinPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFlux2Transformer2DModel
from nunchaku.utils import get_precision

REPO = "tonera/FLUX.2-klein-4B-Nunchaku"  # or local absolute path
NAME = "FLUX.2-klein-4B-Nunchaku"

transformer = NunchakuFlux2Transformer2DModel.from_pretrained(
    f"{REPO}/svdq-{get_precision()}_r32-{NAME}.safetensors",
    torch_dtype=torch.bfloat16,
)
pipe = Flux2KleinPipeline.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, transformer=transformer
)

pipe.to("cuda")

ref = load_image("https://example.com/your_ref.png").convert("RGB")
image = pipe(
    prompt="Describe your edit in English…",
    image=ref,
    guidance_scale=1.0,  # matches official Klein examples; tune if needed
    num_inference_steps=4,  # common for the distilled model; see Diffusers docs otherwise
    generator=torch.Generator("cpu").manual_seed(1),
).images[0]
image.save("flux2_klein_4b_nunchaku.png")
```

For text-to-image, omit `image` (behavior per current Diffusers `Flux2KleinPipeline` docs). Use `pipe.enable_model_cpu_offload()` or similar if VRAM is tight.

## Recommended inference settings

| Parameter | Suggested value | Notes |
|-----------|-----------------|-------|
| `num_inference_steps` | 4 | Common for distilled Klein models |
| `guidance_scale` | 1.0 | Matches official Klein examples |

### If the steps above feel too difficult, install the vitoom platform instead (see below).

## Option 2 (recommended: vitoom)

We recommend deploying and running via [vitoom](https://github.com/tonera/vitoom): it ships a complete **vitoom-nunchaku** runtime, Web UI, multi-reference editing, and LoRA management—no manual wheel install or file copying. See the [vitoom repo](https://github.com/tonera/vitoom) and [docker-usage-en.md](https://github.com/tonera/vitoom/blob/main/docker-usage-en.md) for setup.

### 1. Prepare the environment

You need Docker, Docker Compose, and an NVIDIA GPU for inference (driver must support **CUDA 13.0**, matching vitoom’s `cu130` inference images). Confirm GPU access:

```bash
docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi
```

### 2. Generate config and pull images

In the vitoom repo root, run the setup wizard (recommended):

```bash
git clone https://github.com/tonera/vitoom.git
cd vitoom
python scripts/setup_vitoom.py
```

Or copy `.env.example` to `.env`, fill in required fields (`VITOOM_BACKEND_URL`, `VITOOM_WS_URL`, `VITOOM_INFERENCE_UPLOAD_AUTH_SECRET`, etc.), then pull images:

```bash
python scripts/load_vitoom_images.py --components backend,visual
```

### 3. Start services

Start Backend:

```bash
docker compose up -d backend
```

Open `http://127.0.0.1:8888` in your browser (port follows `VITOOM_SERVER_PORT` in `.env`).

Start the Visual inference service (FLUX.2 Klein is an image model; this profile is required):

```bash
docker compose -f docker-compose.inference.release.yml --profile visual up -d
```

Optional: run `python scripts/download_initial_models.py` to batch-download common models; or place weights under the host `resources/models` directory (override via `VITOOM_MODELS_HOST_DIR` in `.env`).

### 4. Use this model in the Web UI

1. Log in to the vitoom Web UI, open **Models**, download and activate **`tonera/FLUX.2-klein-4B-Nunchaku`**
2. Open the **Image** workspace, select the activated FLUX.2 Klein 4B model, and run text-to-image or image editing (multi-reference editing supported)

You can also run the Python example above directly inside the vitoom Visual inference container.

## LoRA

FLUX.2 Klein supports LoRA on the quantized Transformer when using **vitoom-nunchaku** (wheel or vitoom Visual image)—no manual file copying required:

```python
lora_path = "/path/to/your_lora.safetensors"
transformer.update_lora_params(lora_path)
transformer.set_lora_strength(0.8)
```

## License and compliance

These quantized weights are derived from **FLUX.2-klein-4B**. The upstream model is under [Apache 2.0](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B); confirm upstream license terms and Black Forest Labs’ acceptable use policy before commercial use or redistribution.