metadata
license: other
tags:
- text-to-image
- diffusion
- pixeldit
- nvidia
- pixel-space
base_model: nvidia/PixelDiT-1300M-1024px
PixelDiT 1.3B — Diffusers-Compatible Conversion
This is an unofficial HuggingFace-compatible conversion of NVIDIA's PixelDiT-1300M-1024px model.
All credit goes to the original authors at NVIDIA. This repo only provides a PreTrainedModel wrapper to enable from_pretrained, save_pretrained, and LoRA fine-tuning via peft.
I do not own this model. Original weights, architecture, and training are the work of NVIDIA Research. Please refer to their original repository for license terms.
What is PixelDiT?
PixelDiT is a 1.3B parameter pixel-space diffusion transformer — no VAE, generates images directly in pixel space. Text conditioning uses Gemma-2-2B with a chi_prompt prefix to produce rich visual descriptions.
- Architecture: MMDiT patch blocks + pixel pathway (PiT blocks)
- Text encoder: Gemma-2-2B (
Efficient-Large-Model/gemma-2-2b-it) - Resolution: up to 1024×1024
- Sampler: Flow matching (DPM-Solver++ recommended, 20 steps)
Usage
from pixeldit import PixelDiTPipeline
pipe = PixelDiTPipeline(pretrained="madtune/pixeldit-diffusers")
img = pipe("a white horse running in a meadow at sunset", height=512, width=512)[0]
img.save("out.jpg")
Install the package:
git clone https://github.com/madtune/pixeldit-diffusers
cd pixeldit-diffusers
pip install transformers accelerate safetensors pillow
LoRA fine-tuning
from pixeldit import PixelDiTModel
from peft import get_peft_model, LoraConfig
model = PixelDiTModel.from_pretrained("madtune/pixeldit-diffusers")
lora_cfg = LoraConfig(target_modules=["qkv_x", "qkv_y", "proj_x", "proj_y"])
model = get_peft_model(model, lora_cfg)
model.print_trainable_parameters()
Credits
- Original model: NVIDIA Research
- Diffusers conversion: madtune
- Paper: PixelDiT: Pixel-Space Diffusion Transformers for Text-to-Image Generation — NVIDIA