---
license: cc-by-nc-4.0
base_model: stabilityai/stable-diffusion-3-medium-diffusers
pipeline_tag: image-to-image
language: en
tags:
  - super-resolution
  - image-super-resolution
  - extreme-zoom
  - chain-of-zoom
  - diffusion
  - privileged-distillation
  - faithfulness
---

# OracleZoom

**Privileged-Latent Distillation for faithful extreme super-resolution.**
OracleZoom drives Chain-of-Zoom's recursive 4x super-resolution out to 256x while staying *faithful*, adding real detail instead of hallucinating. This repo ships one ready-to-use file: the **merged super-resolution transformer**.

[![Code](https://img.shields.io/badge/Code-OPD--Zoom-black?logo=github)](https://github.com/dipta007/OPD-Zoom)
[![Base](https://img.shields.io/badge/Backbone-OSEDiff%20/%20SD3-blue)](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers)
[![Paper](https://img.shields.io/badge/Paper-WACV%202027%20(in%20submission)-red)](https://github.com/dipta007/OPD-Zoom)
[![License](https://img.shields.io/badge/License-CC--BY--NC--4.0-lightgrey)](https://creativecommons.org/licenses/by-nc/4.0/)

## What's in this repo
- **`merged_transformer.safetensors`** (fp32, ~8.35 GB): the complete OracleZoom super-resolution transformer, Stable Diffusion 3 + Chain-of-Zoom's SR module + our distilled adapter, all merged into one set of weights. This is all you need.

## Quickstart (one command)
Zooms your images 4x -> 16x -> 64x -> 256x with the merged transformer. Needs one NVIDIA GPU (~16 GB).

```bash
# 1. Get the pipeline (Chain-of-Zoom is included as a submodule)
git clone --recursive https://github.com/dipta007/OPD-Zoom
cd OPD-Zoom

# 2. Install dependencies (Python 3.10)
pip install -r ref/coz/requirements.txt
pip install -U "huggingface_hub[cli]"

# 3. Download the merged model into the pipeline
hf download dipta007/OracleZoom merged_transformer.safetensors --local-dir ckpt/OracleZoom

# 4. Put your images in ./inputs, then run the 4-step zoom
python -m opd_zoom.teacher.oracle_infer \
  --mode student \
  --full_transformer ckpt/OracleZoom/merged_transformer.safetensors \
  --gt_dir ./inputs --out ./outputs --rec_num 4
```

**Results:** `outputs/per-scale/scale1/<name>.png` ... `scale4/<name>.png` are your image at **4x / 16x / 64x / 256x**.

> Notes: Stable Diffusion 3-medium and the Qwen2.5-VL prompter download automatically on first run (a HuggingFace login may be needed for SD3). Chain-of-Zoom's own SR and VLM checkpoints must sit under `ref/coz/ckpt/` (`SR_LoRA`, `SR_VAE`, `VLM_LoRA`); see the [Chain-of-Zoom](https://github.com/bryanswkim/Chain-of-Zoom) repo to fetch them. That is the only extra download.

## Method
Recursive SR (Chain-of-Zoom) reuses a 4x backbone step after step to reach 16x-256x. Each step is **blind**: it sees only a blurred crop of its own previous output and must invent the missing detail, so errors compound and the invention may be hallucinated.

**Privileged-latent distillation.** A *privileged teacher* is shown the ground-truth high-resolution patch **at training time only** and distills its real detail into the blind student, in **decode space**. Only a small adapter is trained; the backbone, VAE, and prompter stay frozen. **A KL leash** to the deployed backbone keeps a deep sharpness reward from drifting into a metric-gaming texture, so detail stays faithful. (Trained: rank-16 adapter, 7.1M params, 1,000 curated 4K images; beta_reward 0.4, beta_kl 8.0. The released weights above have this adapter already merged in.)

## Results
Under Chain-of-Zoom's exact protocol on a curated 4K benchmark and six test sets:

| Axis | Metric | Ours | CoZ / best baseline |
|---|---|---|---|
| Sharpness (no-reference) | CLIPIQA @256x | **0.71** | <= 0.58 |
| Fidelity @4x (ground truth exists) | LPIPS | **0.20** | 0.22 (CoZ) |
| Deep faithfulness (MLLM judge, 64-256x) | preferred vs CoZ | **68-78%** | - |
| Deep faithfulness | hallucination rate vs CoZ | **2-5x lower** | - |

Sharpness is the axis prior methods are built for; the decisive gap is **faithfulness**, verified by full-reference metrics at 4x and by two cross-family MLLM judges plus a blinded human study past 4x.

## Intended Use
- **In-scope:** research on faithful extreme (recursive) super-resolution of natural photographs.
- **Out-of-scope:** forensic/evidentiary use (detail past 4x is generated, not recovered); real-camera-zoom claims (the benchmark uses synthetic center-crop zoom).

## Citation
```bibtex
@inproceedings{dipta2027oraclezoom,
  title={OracleZoom: Privileged-Latent Distillation for Faithful Extreme Super-Resolution},
  author={Shubhashis Roy Dipta},
  year={2027},
  note={In submission, WACV 2027},
  url={https://github.com/dipta007/OPD-Zoom}
}
```
Please also cite Chain-of-Zoom and OSEDiff, whose components this builds on.

## License
Released for **research, non-commercial** use (CC-BY-NC-4.0). Built on OSEDiff / Stable Diffusion 3 and used with a Qwen2.5-VL prompter inside Chain-of-Zoom; the respective upstream licenses apply to those components.