README.md

---
license: apache-2.0
base_model:
- black-forest-labs/FLUX.2-klein-base-4B
pipeline_tag: image-to-image
tags:
- flux
- flux2
- klein
- depth
- controlnet
- refcontrol
- img2img
- image
- editing
- lora
library_name: diffusers
---

# RefControl FLUX.2 Klein 4B - Reference Depth LoRA

## 📝 Short description
A LoRA for **FLUX.2 Klein 4B Base** that fuses a **reference image** (left) with a **depth map** (right).  
It preserves **identity and style** from the reference while following the **pose and structure** from the depth map.

**Trigger word:** `refcontrol`

<Gallery />

---

## 📊 Examples

<table>
  <thead>
    <tr>
      <th>Reference</th>
      <th>Depth</th>
      <th>Result</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/1ref.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/1d.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/1res.png" width="320" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/2ref.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/2d.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/2res.png" width="320" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/3ref.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/3d.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/3res.png" width="320" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/4ref.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/4d.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/4res.png" width="320" /></td>
    </tr>
    <tr>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/5ref.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/5d.png" width="320" /></td>
      <td><img src="https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/resolve/main/images/5res.png" width="320" /></td>
    </tr>
  </tbody>
</table>

---

## 📖 Extended description
This LoRA was primarily trained on **humans**, but it also works with **objects** and stylized characters.  
Its main purpose is to **preserve identity** - facial features, clothing, style, or object characteristics - from the reference image, while adapting the subject to the **pose and composition** defined by the depth map.

Part of the **RefControl** family: reference + control fusion for consistent, controllable generation on **FLUX.2 Klein 4B Base**.

---

## ⚙️ How to use
1. Use first image as depth reference, second image input.
2. Add the trigger word **`refcontrol`** in your prompt.
3. Adjust LoRA weight (recommended **0.8-1.0**) depending on how strongly you want to preserve identity.

## ComfyUI requirenments
For extracting depth you need to install nodes with depth estimation in workflow used:

https://github.com/Fannovel16/comfyui_controlnet_aux

### Base model
Use with [`black-forest-labs/FLUX.2-klein-base-4B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-4B).  
The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, `guidance_scale` ~4.0).

### ✅ Example prompt

```text
refcontrol
```

---

## 🎯 What it does
- Preserves **character or object identity** across generations.
- Embeds the subject into a **new pose or scene** defined by the depth map.
- Works best when the depth map has **similar proportions and scale** to the reference.

---

## ⚡ Tips
- Best results when the depth map is not drastically different in object scale from the reference.
- Combine with **text prompts** to refine background, lighting, or mood.
- Can be combined with other **RefControl** LoRAs (pose, lineart, canny) on compatible pipelines for multi-constraint generation.

---

## 📌 Use cases
- Human portraits in different poses.
- Consistent **character design** across multiple scenes.
- Object transformations (cars, furniture, props) with depth-guided placement.
- Storyboarding, comics, or animation frame generation.

---

## 📦 Files
- **Weights:** [`flux2_klein_4b_refcontrol_depth.safetensors`](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/blob/main/flux2_klein_4b_refcontrol_depth.safetensors)
- **ComfyUI workflow:** [`image_flux2_klein_image_edit_4b_refcontrol_depth.json`](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora/blob/main/image_flux2_klein_image_edit_4b_refcontrol_depth.json)
- **Repo:** [thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora](https://huggingface.co/thedeoxen/refcontrol-FLUX.2-klein-4B-reference-depth-lora)