File size: 2,272 Bytes
0573629 fe7e8a6 0573629 fe7e8a6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
license: other
tags:
- text-to-image
- diffusion
- pixeldit
- nvidia
- pixel-space
base_model: nvidia/PixelDiT-1300M-1024px
---
# PixelDiT 1.3B — Diffusers-Compatible Conversion
This is an **unofficial** HuggingFace-compatible conversion of NVIDIA's [PixelDiT-1300M-1024px](https://huggingface.co/nvidia/PixelDiT-1300M-1024px) model.
All credit goes to the original authors at NVIDIA. This repo only provides a `PreTrainedModel` wrapper to enable `from_pretrained`, `save_pretrained`, and LoRA fine-tuning via `peft`.
> **I do not own this model.** Original weights, architecture, and training are the work of NVIDIA Research. Please refer to their [original repository](https://huggingface.co/nvidia/PixelDiT-1300M-1024px) for license terms.
---
## What is PixelDiT?
PixelDiT is a 1.3B parameter pixel-space diffusion transformer — no VAE, generates images directly in pixel space. Text conditioning uses Gemma-2-2B with a chi_prompt prefix to produce rich visual descriptions.
- **Architecture**: MMDiT patch blocks + pixel pathway (PiT blocks)
- **Text encoder**: Gemma-2-2B (`Efficient-Large-Model/gemma-2-2b-it`)
- **Resolution**: up to 1024×1024
- **Sampler**: Flow matching (DPM-Solver++ recommended, 20 steps)
---
## Usage
```python
from pixeldit import PixelDiTPipeline
pipe = PixelDiTPipeline(pretrained="madtune/pixeldit-diffusers")
img = pipe("a white horse running in a meadow at sunset", height=512, width=512)[0]
img.save("out.jpg")
```
Install the package:
```bash
git clone https://github.com/madtune/pixeldit-diffusers
cd pixeldit-diffusers
pip install transformers accelerate safetensors pillow
```
---
## LoRA fine-tuning
```python
from pixeldit import PixelDiTModel
from peft import get_peft_model, LoraConfig
model = PixelDiTModel.from_pretrained("madtune/pixeldit-diffusers")
lora_cfg = LoraConfig(target_modules=["qkv_x", "qkv_y", "proj_x", "proj_y"])
model = get_peft_model(model, lora_cfg)
model.print_trainable_parameters()
```
---
## Credits
- **Original model**: [NVIDIA Research](https://huggingface.co/nvidia/PixelDiT-1300M-1024px)
- **Diffusers conversion**: [madtune](https://huggingface.co/madtune)
- **Paper**: *PixelDiT: Pixel-Space Diffusion Transformers for Text-to-Image Generation* — NVIDIA
|