Zhengrui commited on
Commit
bfc4d68
·
verified ·
1 Parent(s): 3949f46

Add model card

Browse files
Files changed (1) hide show
  1. README.md +86 -0
README.md ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - 3d-generation
6
+ - voxel-generation
7
+ - 3d-editing
8
+ - diffusion
9
+ - image-to-3d
10
+ - text-to-3d
11
+ - trellis
12
+ pipeline_tag: image-to-3d
13
+ ---
14
+
15
+ # DVD: Discrete Voxel Diffusion for 3D Generation and Editing
16
+
17
+ This repository hosts the pretrained DVD checkpoints for **Discrete Voxel Diffusion for 3D Generation and Editing**.
18
+
19
+ DVD is a discrete diffusion pipeline that generates or edits a pure `64^3` voxel grid. The voxel output can be used directly, exported as a cubified voxel mesh, or passed as the sparse-structure anchor for [TRELLIS](https://github.com/microsoft/TRELLIS) stage 2 to produce final mesh and 3D Gaussian assets.
20
+
21
+ - Paper: [arXiv:2605.07971](https://arxiv.org/abs/2605.07971)
22
+ - Spaces: [DVD Image](https://huggingface.co/spaces/Zhengrui/dvd-image), [DVD Text](https://huggingface.co/spaces/Zhengrui/dvd-text)
23
+
24
+ ## Checkpoints
25
+
26
+ | Variant | Files | Use case |
27
+ | --- | --- | --- |
28
+ | Image generation | `dvd_img.json`, `dvd_img.safetensors` | image-conditioned voxel generation |
29
+ | Image editing | `dvd_img_BSP_ft.json`, `dvd_img_BSP_ft.safetensors` | image-conditioned voxel editing |
30
+ | Text generation | `dvd_text.json`, `dvd_text.safetensors` | text-conditioned voxel generation |
31
+ | Text editing | `dvd_text_BSP_ft.json`, `dvd_text_BSP_ft.safetensors` | text-conditioned voxel editing |
32
+
33
+ ## Minimal Usage
34
+
35
+ ```python
36
+ from dvd import DVDImageToVoxelPipeline, DVDTextToVoxelPipeline
37
+
38
+ repo_id = "Zhengrui/dvd"
39
+
40
+ image_dvd = DVDImageToVoxelPipeline.from_pretrained(repo_id, variant="base", device="cuda")
41
+ image_edit_dvd = DVDImageToVoxelPipeline.from_pretrained(repo_id, variant="bsp", device="cuda")
42
+
43
+ text_dvd = DVDTextToVoxelPipeline.from_pretrained(repo_id, variant="base", device="cuda")
44
+ text_edit_dvd = DVDTextToVoxelPipeline.from_pretrained(repo_id, variant="bsp", device="cuda")
45
+ ```
46
+
47
+ Example image-conditioned generation:
48
+
49
+ ```python
50
+ from PIL import Image
51
+ from dvd import DVDImageToVoxelPipeline, export_cubified_voxels
52
+
53
+ image = Image.open("assets/example_image/T.png")
54
+ dvd = DVDImageToVoxelPipeline.from_pretrained("Zhengrui/dvd", variant="base", device="cuda")
55
+ voxels = dvd.sample_voxels(image, seed=42, steps=256)
56
+ export_cubified_voxels(voxels, "dvd_voxels.glb")
57
+ ```
58
+
59
+ ## Voxel Convention
60
+
61
+ DVD stores voxel outputs as:
62
+
63
+ ```text
64
+ samples: [B, D, H, W]
65
+ coords: [N, 4] as [batch, x, y, z]
66
+ ```
67
+
68
+ Saved `.npy` coordinate files usually omit the batch dimension and can be loaded back with `as_voxel_output(coords, resolution=64)`.
69
+
70
+ ## Notes
71
+
72
+ These files are the DVD voxel-stage checkpoints. TRELLIS stage-2 checkpoints are loaded separately from the official TRELLIS model repositories, such as `microsoft/TRELLIS-image-large` and `microsoft/TRELLIS-text-large`.
73
+
74
+ ## Citation
75
+
76
+ ```bibtex
77
+ @misc{xiang2026dvddiscretevoxeldiffusion,
78
+ title={DVD: Discrete Voxel Diffusion for 3D Generation and Editing},
79
+ author={Zhengrui Xiang and Jiaqi Wu and Fupeng Sun and Heliang Zheng and Yingzhen Li},
80
+ year={2026},
81
+ eprint={2605.07971},
82
+ archivePrefix={arXiv},
83
+ primaryClass={cs.CV},
84
+ url={https://arxiv.org/abs/2605.07971},
85
+ }
86
+ ```