ApacheOne commited on
Commit
f8377e9
Β·
verified Β·
1 Parent(s): 6015b01

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +145 -18
README.md CHANGED
@@ -12,42 +12,169 @@ tags:
12
  - custom-runtime-required
13
  ---
14
 
 
15
 
16
- ### A4 runtime isnt public yet
17
- cant use these models without it, still working on the comfyUI node for loader to pipe into core ksampler
18
 
19
- # WIP for W2A4 block-level rollout recovery
20
 
 
21
 
 
22
 
 
23
 
 
24
 
25
- # OrbitQuant SDXL UNet Exports
26
 
27
- Custom UNet-only OrbitQuant exports for CreaPrompt Lightning Hyper-SDXL.
 
 
 
 
 
 
 
 
 
28
 
29
- These files are not normal SDXL checkpoints. They contain only `model.diffusion_model.*`
30
- UNet tensors plus `__orbitquant__` metadata/codebooks/rotations. They require a custom
31
- runtime that supports packed OrbitQuant weights and A4 activation quantization.
32
- examples/download (1).png
33
- ## Examples
34
 
35
  <table>
36
  <tr>
37
  <td align="center">
38
- <img src="examples/630df8e8-f7b1-4225-889b-247dcb55e2f0.png" width="360">
 
 
 
39
  <br>
40
- <strong>W4A4</strong>
41
  </td>
42
- <td align="center">
43
- <img src="examples/download (1).png" width="360">
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  <br>
45
- <strong>FP16</strong>
46
  </td>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  <td align="center">
48
- <img src="examples/download.png" width="360">
 
 
 
49
  <br>
50
- <strong>W2A4</strong>
51
  </td>
52
  </tr>
53
- </table>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  - custom-runtime-required
13
  ---
14
 
15
+ # OrbitQuant SDXL Exports
16
 
17
+ Experimental OrbitQuant and ConvRot quantized exports for CreaPrompt Lightning Hyper-SDXL.
 
18
 
19
+ > **Work in progress:** The native A4 runtime is not public yet. These models cannot currently be used without the custom runtime. A ComfyUI loader node that outputs a model compatible with the core KSampler is still under development.
20
 
21
+ ## W2A4 Recovery Status
22
 
23
+ WIP for W2A4 block-level rollout recovery.
24
 
25
+ ## Model Format
26
 
27
+ These are not standard SDXL checkpoints that can be loaded directly by existing SDXL loaders.
28
 
29
+ Depending on the export, the files can contain:
30
 
31
+ * OrbitQuant-packed SDXL UNet weights
32
+ * ConvRot-quantized SDXL UNet weights
33
+ * A4 activation-quantization metadata
34
+ * Quantized CLIP and text-encoder weights
35
+ * FP16 VAE weights
36
+ * `__orbitquant__` metadata
37
+ * Codebooks
38
+ * Rotations
39
+ * Quantization scales
40
+ * Runtime configuration
41
 
42
+ A custom runtime is required to execute the packed OrbitQuant and ConvRot weights without expanding them back into ordinary FP16 weights.
43
+
44
+ # Examples
45
+
46
+ ## Full OrbitQuant versus Full ConvRot
47
 
48
  <table>
49
  <tr>
50
  <td align="center">
51
+ <img
52
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/comparisons/FULL_OQW4A4_vs_FULL_CRW4A4_CLIPINT4_comparison.png"
53
+ width="760"
54
+ >
55
  <br>
56
+ <strong>Full OrbitQuant W4A4 versus Full ConvRot W4A4 β€” CLIP INT4</strong>
57
  </td>
58
+ </tr>
59
+ </table>
60
+
61
+ ## Full-UNet Tests
62
+
63
+ <table>
64
+ <tr>
65
+ <td align="center" width="50%">
66
+ <img
67
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/full_unet/convrot_w4a4_clip_int4/FULLUNET_CRW4A4_CLIPINT4_test.png"
68
+ width="420"
69
+ >
70
+ <br>
71
+ <strong>Full ConvRot W4A4 UNet β€” CLIP INT4</strong>
72
+ </td>
73
+ <td align="center" width="50%">
74
+ <img
75
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/full_unet/orbitquant_w4a4_clip_int4/FULLUNET_OQW4A4_CLIPINT4_test.png"
76
+ width="420"
77
+ >
78
+ <br>
79
+ <strong>Full OrbitQuant W4A4 UNet β€” CLIP INT4</strong>
80
+ </td>
81
+ </tr>
82
+ </table>
83
+
84
+ ## Mixed OrbitQuant and ConvRot UNet β€” CLIP FP16
85
+
86
+ <table>
87
+ <tr>
88
+ <td align="center" width="50%">
89
+ <img
90
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/mixed_unet/clip_fp16/creaprompt_SDXL_OQW4A4_CRW4A4_CLIPFP16_test.png"
91
+ width="420"
92
+ >
93
+ <br>
94
+ <strong>OrbitQuant W4A4 + ConvRot W4A4 β€” CLIP FP16</strong>
95
+ </td>
96
+ <td align="center" width="50%">
97
+ <img
98
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/mixed_unet/clip_fp16/creaprompt_SDXL_OQW4A4_CRW8A8_CLIPFP16_test.png"
99
+ width="420"
100
+ >
101
+ <br>
102
+ <strong>OrbitQuant W4A4 + ConvRot W8A8 β€” CLIP FP16</strong>
103
+ </td>
104
+ </tr>
105
+ </table>
106
+
107
+ ## Mixed OrbitQuant and ConvRot UNet β€” CLIP INT4
108
+
109
+ <table>
110
+ <tr>
111
+ <td align="center" width="50%">
112
+ <img
113
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/mixed_unet/clip_int4/creaprompt_SDXL_OQW4A4_CRW4A4_CLIPINT4_test.png"
114
+ width="420"
115
+ >
116
+ <br>
117
+ <strong>OrbitQuant W4A4 + ConvRot W4A4 β€” CLIP INT4</strong>
118
+ </td>
119
+ <td align="center" width="50%">
120
+ <img
121
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/mixed_unet/clip_int4/creaprompt_SDXL_OQW4A4_CRW8A8_CLIPINT4_test.png"
122
+ width="420"
123
+ >
124
+ <br>
125
+ <strong>OrbitQuant W4A4 + ConvRot W8A8 β€” CLIP INT4</strong>
126
+ </td>
127
+ </tr>
128
+ </table>
129
+
130
+ ## Mixed OrbitQuant and ConvRot UNet β€” CLIP INT8
131
+
132
+ <table>
133
+ <tr>
134
+ <td align="center" width="50%">
135
+ <img
136
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/mixed_unet/clip_int8/creaprompt_OQW4A4_CRW8A8_test.png"
137
+ width="420"
138
+ >
139
  <br>
140
+ <strong>OrbitQuant W4A4 + ConvRot W8A8 β€” CLIP INT8</strong>
141
  </td>
142
+ <td align="center" width="50%">
143
+ <img
144
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/mixed_unet/clip_int8/first_model_test.png"
145
+ width="420"
146
+ >
147
+ <br>
148
+ <strong>First Mixed-Quant Model Test β€” CLIP INT8</strong>
149
+ </td>
150
+ </tr>
151
+ </table>
152
+
153
+ ## Root Example
154
+
155
+ <table>
156
+ <tr>
157
  <td align="center">
158
+ <img
159
+ src="hybrid_quants/orbitW4A4convrotW4A4quant/examples/creaprompt_SDXL_OQW4A4_CRW8A8_CLIPINT4_test.png"
160
+ width="520"
161
+ >
162
  <br>
163
+ <strong>OrbitQuant W4A4 + ConvRot W8A8 β€” CLIP INT4</strong>
164
  </td>
165
  </tr>
166
+ </table>
167
+
168
+ ## Runtime Availability
169
+
170
+ The following components are still required for normal use:
171
+
172
+ * Native packed OrbitQuant weight execution
173
+ * Native A4 activation quantization
174
+ * Native ConvRot quantized operators
175
+ * ComfyUI model loader
176
+ * Integration with the core ComfyUI KSampler
177
+ * Runtime validation
178
+ * Performance tuning
179
+
180
+ Until the custom runtime is released, these files should be considered experimental development exports.