Hugging Face's logo

ApacheOne
/
orbitquant_unets

Text-to-Image
unet
orbitquant
quantization
w4a4
w3a4
w2a4
custom-runtime-required
sdxl
Model card Files
xet
 Community
ApacheOne commited on
Commit
18e9055
·
verified ·
1 Parent(s): b511348

Delete adaptertestsfor2bitday2

Browse files
Files changed (15) hide show
  1. adaptertestsfor2bitday2/creaprompt_W2A4_FULLRECOVERY_722_rank8_W4bridge_realLatents.safetensors +0 -3
  2. adaptertestsfor2bitday2/creaprompt_W2A4_FULLRECOVERY_722_rank8_W4bridge_realLatents_state.json +0 -0
  3. adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE1B_FFN_AttnOut_ProjIO_rankmix_gamma_beta.safetensors +0 -3
  4. adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE1B_FFN_AttnOut_ProjIO_rankmix_gamma_beta_state.json +0 -0
  5. adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE2_CrossAttnKV_ProjOut_rankmix_gamma_beta.safetensors +0 -3
  6. adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE2_CrossAttnKV_ProjOut_rankmix_gamma_beta_state.json +0 -0
  7. adaptertestsfor2bitday2/download.png +0 -3
  8. adaptertestsfor2bitday2/orbitquant_W4_vs_W2_all_layers.csv +0 -0
  9. adaptertestsfor2bitday2/orbitquant_W4_vs_W2_by_suffix.csv +0 -13
  10. adaptertestsfor2bitday2/orbitquant_W4_vs_W2_diagnostic_report.json +0 -1448
  11. adaptertestsfor2bitday2/orbitquant_W4_vs_W2_diagnostic_report.xlsx +0 -3
  12. adaptertestsfor2bitday2/orbitquant_W4_vs_W2_next_training_plan.json +0 -0
  13. adaptertestsfor2bitday2/orbitquant_W4_vs_W2_top_training_targets.csv +0 -0
  14. adaptertestsfor2bitday2/temp.txt +0 -0
  15. adaptertestsfor2bitday2/w4_w2_compare_condition_cache_captions_only_random_latents_1024x1024_8.pt +0 -3
adaptertestsfor2bitday2/creaprompt_W2A4_FULLRECOVERY_722_rank8_W4bridge_realLatents.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:ff20304bee1bdd49cf4d14dd85a73d469438208f7361fa1d55548870b82152df
3
- size 49108608
 
 
 
 
adaptertestsfor2bitday2/creaprompt_W2A4_FULLRECOVERY_722_rank8_W4bridge_realLatents_state.json DELETED
The diff for this file is too large to render. See raw diff
 
adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE1B_FFN_AttnOut_ProjIO_rankmix_gamma_beta.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:7c9c61aa0b499e2a43e1703ef54bc67ed78bcccc75effa4e4b62502690c1ca29
3
- size 404958616
 
 
 
 
adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE1B_FFN_AttnOut_ProjIO_rankmix_gamma_beta_state.json DELETED
The diff for this file is too large to render. See raw diff
 
adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE2_CrossAttnKV_ProjOut_rankmix_gamma_beta.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3b43365440fa7ba1de07b263102b20b392e50e6dd71df25d805fd7c47949ee4a
3
- size 163569480
 
 
 
 
adaptertestsfor2bitday2/creaprompt_W2A4_recovery_PHASE2_CrossAttnKV_ProjOut_rankmix_gamma_beta_state.json DELETED
The diff for this file is too large to render. See raw diff
 
adaptertestsfor2bitday2/download.png DELETED

Git LFS Details

  • SHA256: b70f8b70ca4f1774270c7f0ffe14fab4d0bd45bb8afa9cf63e39c4eeb9a9c5dc
  • Pointer size: 132 Bytes
  • Size of remote file: 1.54 MB
adaptertestsfor2bitday2/orbitquant_W4_vs_W2_all_layers.csv DELETED
The diff for this file is too large to render. See raw diff
 
adaptertestsfor2bitday2/orbitquant_W4_vs_W2_by_suffix.csv DELETED
@@ -1,13 +0,0 @@
1
- suffix,count,old_rank128_count,phase1b_count,phase2_count,composite_count,w4_mse_mean,w2_raw_mse_mean,w2_old_rank128_mse_mean,w2_phase1b_mse_mean,w2_phase2_mse_mean,w2_composite_mse_mean,w2_raw_to_w4_ratio_mean,w2_composite_to_w4_ratio_mean,composite_improvement_vs_raw_mean,phase1b_improvement_vs_raw_mean,phase2_improvement_vs_raw_mean,remaining_gap_score_mean,remaining_gap_score_max
2
- attn1.to_q,70,0,0,0,0,0.008834530145039174,0.06604109280267541,0.06604109280267541,0.06604109280267541,0.06604109280267541,0.06604109280267541,7.5747877444341,7.5747877444341,0.9999999999832079,0.9999999999832079,0.9999999999832079,0.42929220002638296,0.8156589509387167
3
- attn1.to_k,70,0,0,0,0,0.00869125885917444,0.06352441829714772,0.06352441829714772,0.06352441829714772,0.06352441829714772,0.06352441829714772,7.395005113804398,7.395005113804398,0.9999999999825412,0.9999999999825412,0.9999999999825412,0.4027360624185252,0.7311920707443591
4
- attn1.to_v,70,0,0,0,0,0.0078085727364238244,0.055820106880796354,0.055820106880796354,0.055820106880796354,0.055820106880796354,0.055820106880796354,7.1828383652128185,7.1828383652128185,0.999999999979725,0.999999999979725,0.999999999979725,0.34468865166305246,0.8469947038879494
5
- proj_out,11,11,11,8,11,0.08516350478106238,0.5946647835556756,0.35843181815421715,0.21572238788637635,0.2251953121226329,0.20004504513866328,6.913556153067888,2.452973561882824,2.969396590218453,2.8728746957207307,2.388979858331337,0.2912881384368135,0.7227891036499625
6
- attn2.to_q,70,0,0,0,0,0.004732320876079327,0.037790954292618806,0.037790954292618806,0.037790954292618806,0.037790954292618806,0.037790954292618806,8.31881567261658,8.31881567261658,0.9999999999712236,0.9999999999712236,0.9999999999712236,0.2711913227898678,0.6841755529628081
7
- ff.net.2,70,70,70,0,70,0.008616097393292431,0.05920763128883756,0.04301157893223346,0.02601208476651441,0.05920763128883756,0.02601208476651441,6.932097684619555,3.2009256575082317,2.3014962630105096,2.3014962630105096,0.9999999999747406,0.0572061937220024,0.20667232622681672
8
- ff.net.0.proj,70,70,70,0,70,0.006887027408722171,0.04879796355012424,0.03164028028936164,0.0176630408599336,0.04879796355012424,0.0176630408599336,7.171405831067578,2.504632836281881,2.998213235077801,2.998213235077801,0.9999999999754532,0.02951947878339631,0.1137982287591066
9
- proj_in,11,11,11,0,11,0.008933692221838206,0.06406978043038157,0.03873854998329824,0.018474196746806285,0.06406978043038157,0.018474196746806285,7.050454051901254,1.9995457179267833,3.6405517751576513,3.6405517751576513,0.9999999999808292,0.020491349940082455,0.05859008625009349
10
- attn2.to_out.0,70,70,70,0,70,0.0003375165737176238,0.002381115644698648,0.002395683516781632,0.00222385320224494,0.002381115644698648,0.00222385320224494,7.598942937360687,6.7306297362361756,1.8347576429611256,1.8347576429611256,0.9999999981489446,0.012840779013027317,0.05913571060692841
11
- attn1.to_out.0,70,70,70,0,70,0.0042316013468252445,0.030931819595006883,0.01690724441420515,0.005962002335374044,0.030931819595006883,0.005962002335374044,7.476497691796118,1.4532056634829924,5.661566033087392,5.661566033087392,0.9999999999555428,0.0034474704133703057,0.021837045500065322
12
- attn2.to_v,70,0,0,70,70,0.05053253279267145,0.33406722963388474,0.33406722963388474,0.33406722963388474,0.020737389160785824,0.020737389160785824,6.628456469971321,0.4083913035125052,16.325964238694546,0.9999999999957944,16.325964238694546,0.0,0.0
13
- attn2.to_k,70,0,0,70,70,0.0519350185824026,0.42513413368059055,0.42513413368059055,0.42513413368059055,0.024206796191200346,0.024206796191200346,8.66438299826568,0.47265911793929566,18.609512818870797,0.9999999999969618,18.609512818870797,0.0,0.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adaptertestsfor2bitday2/orbitquant_W4_vs_W2_diagnostic_report.json DELETED
@@ -1,1448 +0,0 @@
1
- {
2
- "diagnostic_type": "FP16 teacher layer output vs W4A4 vs W2A4 raw/adapted variants",
3
- "teacher": "/content/creaprompt_sdxl/creapromptLightning_creapromtHypersdxlV1.safetensors",
4
- "w4_model": "/content/creaprompt_sdxl/orbitquant_full/creaprompt_FULL_OrbitQuant_UNet_W4A4_with_VAE_CLIPs.safetensors",
5
- "w2_model": "/content/creaprompt_sdxl/orbitquant_full/creaprompt_FULL_OrbitQuant_UNet_W2A4_with_VAE_CLIPs.safetensors",
6
- "old_rank128_adapter": "/content/creaprompt_sdxl/adapter_training/checkpoints/creaprompt_W2A4_recovery_FFN_AttnOut_rank128_from_rank8.safetensors",
7
- "phase1b_adapter": "/content/creaprompt_sdxl/adapter_training/checkpoints/creaprompt_W2A4_recovery_PHASE1B_FFN_AttnOut_ProjIO_rankmix_gamma_beta.safetensors",
8
- "phase2_adapter": "/content/creaprompt_sdxl/adapter_training/checkpoints/creaprompt_W2A4_recovery_PHASE2_CrossAttnKV_ProjOut_rankmix_gamma_beta.safetensors",
9
- "dataset_mode": "captions_only_random_latents",
10
- "diag_width": 1024,
11
- "diag_height": 1024,
12
- "diag_samples": 8,
13
- "timesteps": [
14
- 999,
15
- 850,
16
- 700,
17
- 500,
18
- 300,
19
- 150,
20
- 50
21
- ],
22
- "max_tokens_per_layer": 2048,
23
- "total_layers": 722,
24
- "outputs": {
25
- "all_layers_csv": "/content/creaprompt_sdxl/diagnostics_w4_w2_compare/orbitquant_W4_vs_W2_all_layers.csv",
26
- "top_training_targets_csv": "/content/creaprompt_sdxl/diagnostics_w4_w2_compare/orbitquant_W4_vs_W2_top_training_targets.csv",
27
- "suffix_summary_csv": "/content/creaprompt_sdxl/diagnostics_w4_w2_compare/orbitquant_W4_vs_W2_by_suffix.csv",
28
- "plan_json": "/content/creaprompt_sdxl/diagnostics_w4_w2_compare/orbitquant_W4_vs_W2_next_training_plan.json",
29
- "report_json": "/content/creaprompt_sdxl/diagnostics_w4_w2_compare/orbitquant_W4_vs_W2_diagnostic_report.json",
30
- "xlsx": "/content/creaprompt_sdxl/diagnostics_w4_w2_compare/orbitquant_W4_vs_W2_diagnostic_report.xlsx"
31
- },
32
- "suffix_summary": [
33
- {
34
- "suffix": "attn1.to_q",
35
- "count": 70,
36
- "old_rank128_count": 0,
37
- "phase1b_count": 0,
38
- "phase2_count": 0,
39
- "composite_count": 0,
40
- "w4_mse_mean": 0.008834530145039174,
41
- "w2_raw_mse_mean": 0.06604109280267541,
42
- "w2_old_rank128_mse_mean": 0.06604109280267541,
43
- "w2_phase1b_mse_mean": 0.06604109280267541,
44
- "w2_phase2_mse_mean": 0.06604109280267541,
45
- "w2_composite_mse_mean": 0.06604109280267541,
46
- "w2_raw_to_w4_ratio_mean": 7.5747877444341,
47
- "w2_composite_to_w4_ratio_mean": 7.5747877444341,
48
- "composite_improvement_vs_raw_mean": 0.9999999999832079,
49
- "phase1b_improvement_vs_raw_mean": 0.9999999999832079,
50
- "phase2_improvement_vs_raw_mean": 0.9999999999832079,
51
- "remaining_gap_score_mean": 0.42929220002638296,
52
- "remaining_gap_score_max": 0.8156589509387167
53
- },
54
- {
55
- "suffix": "attn1.to_k",
56
- "count": 70,
57
- "old_rank128_count": 0,
58
- "phase1b_count": 0,
59
- "phase2_count": 0,
60
- "composite_count": 0,
61
- "w4_mse_mean": 0.00869125885917444,
62
- "w2_raw_mse_mean": 0.06352441829714772,
63
- "w2_old_rank128_mse_mean": 0.06352441829714772,
64
- "w2_phase1b_mse_mean": 0.06352441829714772,
65
- "w2_phase2_mse_mean": 0.06352441829714772,
66
- "w2_composite_mse_mean": 0.06352441829714772,
67
- "w2_raw_to_w4_ratio_mean": 7.395005113804398,
68
- "w2_composite_to_w4_ratio_mean": 7.395005113804398,
69
- "composite_improvement_vs_raw_mean": 0.9999999999825412,
70
- "phase1b_improvement_vs_raw_mean": 0.9999999999825412,
71
- "phase2_improvement_vs_raw_mean": 0.9999999999825412,
72
- "remaining_gap_score_mean": 0.4027360624185252,
73
- "remaining_gap_score_max": 0.7311920707443591
74
- },
75
- {
76
- "suffix": "attn1.to_v",
77
- "count": 70,
78
- "old_rank128_count": 0,
79
- "phase1b_count": 0,
80
- "phase2_count": 0,
81
- "composite_count": 0,
82
- "w4_mse_mean": 0.0078085727364238244,
83
- "w2_raw_mse_mean": 0.055820106880796354,
84
- "w2_old_rank128_mse_mean": 0.055820106880796354,
85
- "w2_phase1b_mse_mean": 0.055820106880796354,
86
- "w2_phase2_mse_mean": 0.055820106880796354,
87
- "w2_composite_mse_mean": 0.055820106880796354,
88
- "w2_raw_to_w4_ratio_mean": 7.1828383652128185,
89
- "w2_composite_to_w4_ratio_mean": 7.1828383652128185,
90
- "composite_improvement_vs_raw_mean": 0.999999999979725,
91
- "phase1b_improvement_vs_raw_mean": 0.999999999979725,
92
- "phase2_improvement_vs_raw_mean": 0.999999999979725,
93
- "remaining_gap_score_mean": 0.34468865166305246,
94
- "remaining_gap_score_max": 0.8469947038879494
95
- },
96
- {
97
- "suffix": "proj_out",
98
- "count": 11,
99
- "old_rank128_count": 11,
100
- "phase1b_count": 11,
101
- "phase2_count": 8,
102
- "composite_count": 11,
103
- "w4_mse_mean": 0.08516350478106238,
104
- "w2_raw_mse_mean": 0.5946647835556756,
105
- "w2_old_rank128_mse_mean": 0.35843181815421715,
106
- "w2_phase1b_mse_mean": 0.21572238788637635,
107
- "w2_phase2_mse_mean": 0.2251953121226329,
108
- "w2_composite_mse_mean": 0.20004504513866328,
109
- "w2_raw_to_w4_ratio_mean": 6.913556153067888,
110
- "w2_composite_to_w4_ratio_mean": 2.452973561882824,
111
- "composite_improvement_vs_raw_mean": 2.969396590218453,
112
- "phase1b_improvement_vs_raw_mean": 2.8728746957207307,
113
- "phase2_improvement_vs_raw_mean": 2.388979858331337,
114
- "remaining_gap_score_mean": 0.2912881384368135,
115
- "remaining_gap_score_max": 0.7227891036499625
116
- },
117
- {
118
- "suffix": "attn2.to_q",
119
- "count": 70,
120
- "old_rank128_count": 0,
121
- "phase1b_count": 0,
122
- "phase2_count": 0,
123
- "composite_count": 0,
124
- "w4_mse_mean": 0.004732320876079327,
125
- "w2_raw_mse_mean": 0.037790954292618806,
126
- "w2_old_rank128_mse_mean": 0.037790954292618806,
127
- "w2_phase1b_mse_mean": 0.037790954292618806,
128
- "w2_phase2_mse_mean": 0.037790954292618806,
129
- "w2_composite_mse_mean": 0.037790954292618806,
130
- "w2_raw_to_w4_ratio_mean": 8.31881567261658,
131
- "w2_composite_to_w4_ratio_mean": 8.31881567261658,
132
- "composite_improvement_vs_raw_mean": 0.9999999999712236,
133
- "phase1b_improvement_vs_raw_mean": 0.9999999999712236,
134
- "phase2_improvement_vs_raw_mean": 0.9999999999712236,
135
- "remaining_gap_score_mean": 0.2711913227898678,
136
- "remaining_gap_score_max": 0.6841755529628081
137
- },
138
- {
139
- "suffix": "ff.net.2",
140
- "count": 70,
141
- "old_rank128_count": 70,
142
- "phase1b_count": 70,
143
- "phase2_count": 0,
144
- "composite_count": 70,
145
- "w4_mse_mean": 0.008616097393292431,
146
- "w2_raw_mse_mean": 0.05920763128883756,
147
- "w2_old_rank128_mse_mean": 0.04301157893223346,
148
- "w2_phase1b_mse_mean": 0.02601208476651441,
149
- "w2_phase2_mse_mean": 0.05920763128883756,
150
- "w2_composite_mse_mean": 0.02601208476651441,
151
- "w2_raw_to_w4_ratio_mean": 6.932097684619555,
152
- "w2_composite_to_w4_ratio_mean": 3.2009256575082317,
153
- "composite_improvement_vs_raw_mean": 2.3014962630105096,
154
- "phase1b_improvement_vs_raw_mean": 2.3014962630105096,
155
- "phase2_improvement_vs_raw_mean": 0.9999999999747406,
156
- "remaining_gap_score_mean": 0.0572061937220024,
157
- "remaining_gap_score_max": 0.20667232622681672
158
- },
159
- {
160
- "suffix": "ff.net.0.proj",
161
- "count": 70,
162
- "old_rank128_count": 70,
163
- "phase1b_count": 70,
164
- "phase2_count": 0,
165
- "composite_count": 70,
166
- "w4_mse_mean": 0.006887027408722171,
167
- "w2_raw_mse_mean": 0.04879796355012424,
168
- "w2_old_rank128_mse_mean": 0.03164028028936164,
169
- "w2_phase1b_mse_mean": 0.0176630408599336,
170
- "w2_phase2_mse_mean": 0.04879796355012424,
171
- "w2_composite_mse_mean": 0.0176630408599336,
172
- "w2_raw_to_w4_ratio_mean": 7.171405831067578,
173
- "w2_composite_to_w4_ratio_mean": 2.504632836281881,
174
- "composite_improvement_vs_raw_mean": 2.998213235077801,
175
- "phase1b_improvement_vs_raw_mean": 2.998213235077801,
176
- "phase2_improvement_vs_raw_mean": 0.9999999999754532,
177
- "remaining_gap_score_mean": 0.02951947878339631,
178
- "remaining_gap_score_max": 0.1137982287591066
179
- },
180
- {
181
- "suffix": "proj_in",
182
- "count": 11,
183
- "old_rank128_count": 11,
184
- "phase1b_count": 11,
185
- "phase2_count": 0,
186
- "composite_count": 11,
187
- "w4_mse_mean": 0.008933692221838206,
188
- "w2_raw_mse_mean": 0.06406978043038157,
189
- "w2_old_rank128_mse_mean": 0.03873854998329824,
190
- "w2_phase1b_mse_mean": 0.018474196746806285,
191
- "w2_phase2_mse_mean": 0.06406978043038157,
192
- "w2_composite_mse_mean": 0.018474196746806285,
193
- "w2_raw_to_w4_ratio_mean": 7.050454051901254,
194
- "w2_composite_to_w4_ratio_mean": 1.9995457179267833,
195
- "composite_improvement_vs_raw_mean": 3.6405517751576513,
196
- "phase1b_improvement_vs_raw_mean": 3.6405517751576513,
197
- "phase2_improvement_vs_raw_mean": 0.9999999999808292,
198
- "remaining_gap_score_mean": 0.020491349940082455,
199
- "remaining_gap_score_max": 0.05859008625009349
200
- },
201
- {
202
- "suffix": "attn2.to_out.0",
203
- "count": 70,
204
- "old_rank128_count": 70,
205
- "phase1b_count": 70,
206
- "phase2_count": 0,
207
- "composite_count": 70,
208
- "w4_mse_mean": 0.0003375165737176238,
209
- "w2_raw_mse_mean": 0.002381115644698648,
210
- "w2_old_rank128_mse_mean": 0.002395683516781632,
211
- "w2_phase1b_mse_mean": 0.00222385320224494,
212
- "w2_phase2_mse_mean": 0.002381115644698648,
213
- "w2_composite_mse_mean": 0.00222385320224494,
214
- "w2_raw_to_w4_ratio_mean": 7.598942937360687,
215
- "w2_composite_to_w4_ratio_mean": 6.7306297362361756,
216
- "composite_improvement_vs_raw_mean": 1.8347576429611256,
217
- "phase1b_improvement_vs_raw_mean": 1.8347576429611256,
218
- "phase2_improvement_vs_raw_mean": 0.9999999981489446,
219
- "remaining_gap_score_mean": 0.012840779013027317,
220
- "remaining_gap_score_max": 0.05913571060692841
221
- },
222
- {
223
- "suffix": "attn1.to_out.0",
224
- "count": 70,
225
- "old_rank128_count": 70,
226
- "phase1b_count": 70,
227
- "phase2_count": 0,
228
- "composite_count": 70,
229
- "w4_mse_mean": 0.0042316013468252445,
230
- "w2_raw_mse_mean": 0.030931819595006883,
231
- "w2_old_rank128_mse_mean": 0.01690724441420515,
232
- "w2_phase1b_mse_mean": 0.005962002335374044,
233
- "w2_phase2_mse_mean": 0.030931819595006883,
234
- "w2_composite_mse_mean": 0.005962002335374044,
235
- "w2_raw_to_w4_ratio_mean": 7.476497691796118,
236
- "w2_composite_to_w4_ratio_mean": 1.4532056634829924,
237
- "composite_improvement_vs_raw_mean": 5.661566033087392,
238
- "phase1b_improvement_vs_raw_mean": 5.661566033087392,
239
- "phase2_improvement_vs_raw_mean": 0.9999999999555428,
240
- "remaining_gap_score_mean": 0.0034474704133703057,
241
- "remaining_gap_score_max": 0.021837045500065322
242
- },
243
- {
244
- "suffix": "attn2.to_v",
245
- "count": 70,
246
- "old_rank128_count": 0,
247
- "phase1b_count": 0,
248
- "phase2_count": 70,
249
- "composite_count": 70,
250
- "w4_mse_mean": 0.05053253279267145,
251
- "w2_raw_mse_mean": 0.33406722963388474,
252
- "w2_old_rank128_mse_mean": 0.33406722963388474,
253
- "w2_phase1b_mse_mean": 0.33406722963388474,
254
- "w2_phase2_mse_mean": 0.020737389160785824,
255
- "w2_composite_mse_mean": 0.020737389160785824,
256
- "w2_raw_to_w4_ratio_mean": 6.628456469971321,
257
- "w2_composite_to_w4_ratio_mean": 0.4083913035125052,
258
- "composite_improvement_vs_raw_mean": 16.325964238694546,
259
- "phase1b_improvement_vs_raw_mean": 0.9999999999957944,
260
- "phase2_improvement_vs_raw_mean": 16.325964238694546,
261
- "remaining_gap_score_mean": 0.0,
262
- "remaining_gap_score_max": 0.0
263
- },
264
- {
265
- "suffix": "attn2.to_k",
266
- "count": 70,
267
- "old_rank128_count": 0,
268
- "phase1b_count": 0,
269
- "phase2_count": 70,
270
- "composite_count": 70,
271
- "w4_mse_mean": 0.0519350185824026,
272
- "w2_raw_mse_mean": 0.42513413368059055,
273
- "w2_old_rank128_mse_mean": 0.42513413368059055,
274
- "w2_phase1b_mse_mean": 0.42513413368059055,
275
- "w2_phase2_mse_mean": 0.024206796191200346,
276
- "w2_composite_mse_mean": 0.024206796191200346,
277
- "w2_raw_to_w4_ratio_mean": 8.66438299826568,
278
- "w2_composite_to_w4_ratio_mean": 0.47265911793929566,
279
- "composite_improvement_vs_raw_mean": 18.609512818870797,
280
- "phase1b_improvement_vs_raw_mean": 0.9999999999969618,
281
- "phase2_improvement_vs_raw_mean": 18.609512818870797,
282
- "remaining_gap_score_mean": 0.0,
283
- "remaining_gap_score_max": 0.0
284
- }
285
- ],
286
- "top_40_training_targets": [
287
- {
288
- "training_priority_rank": 1,
289
- "module_name": "up_blocks.1.attentions.0.transformer_blocks.1.attn1.to_v",
290
- "ldm_key": "model.diffusion_model.output_blocks.3.1.transformer_blocks.1.attn1.to_v.weight",
291
- "suffix": "attn1.to_v",
292
- "shape": [
293
- 640,
294
- 640
295
- ],
296
- "has_old_rank128": false,
297
- "has_phase1b": false,
298
- "has_phase2": false,
299
- "composite_source": "raw",
300
- "w4_mse_mean": 0.016171995251040374,
301
- "w2_raw_mse_mean": 0.125401715614966,
302
- "w2_old_rank128_mse_mean": 0.125401715614966,
303
- "w2_phase1b_mse_mean": 0.125401715614966,
304
- "w2_phase2_mse_mean": 0.125401715614966,
305
- "w2_composite_mse_mean": 0.125401715614966,
306
- "w2_raw_to_w4_mse_ratio": 7.754251325243521,
307
- "w2_composite_to_w4_mse_ratio": 7.754251325243521,
308
- "old_rank128_improvement_vs_raw": 0.9999999999920256,
309
- "phase1b_improvement_vs_raw": 0.9999999999920256,
310
- "phase2_improvement_vs_raw": 0.9999999999920256,
311
- "composite_improvement_vs_raw": 0.9999999999920256,
312
- "remaining_w2_specific_gap_score": 0.8469947038879494,
313
- "recommended_action": "add_adapter_missing_layer",
314
- "recommended_rank": 256
315
- },
316
- {
317
- "training_priority_rank": 2,
318
- "module_name": "up_blocks.1.attentions.1.transformer_blocks.1.attn1.to_q",
319
- "ldm_key": "model.diffusion_model.output_blocks.4.1.transformer_blocks.1.attn1.to_q.weight",
320
- "suffix": "attn1.to_q",
321
- "shape": [
322
- 640,
323
- 640
324
- ],
325
- "has_old_rank128": false,
326
- "has_phase1b": false,
327
- "has_phase2": false,
328
- "composite_source": "raw",
329
- "w4_mse_mean": 0.01798078600716378,
330
- "w2_raw_mse_mean": 0.130427686364523,
331
- "w2_old_rank128_mse_mean": 0.130427686364523,
332
- "w2_phase1b_mse_mean": 0.130427686364523,
333
- "w2_phase2_mse_mean": 0.130427686364523,
334
- "w2_composite_mse_mean": 0.130427686364523,
335
- "w2_raw_to_w4_mse_ratio": 7.253725521526435,
336
- "w2_composite_to_w4_mse_ratio": 7.253725521526435,
337
- "old_rank128_improvement_vs_raw": 0.9999999999923329,
338
- "phase1b_improvement_vs_raw": 0.9999999999923329,
339
- "phase2_improvement_vs_raw": 0.9999999999923329,
340
- "composite_improvement_vs_raw": 0.9999999999923329,
341
- "remaining_w2_specific_gap_score": 0.8156589509387167,
342
- "recommended_action": "add_adapter_missing_layer",
343
- "recommended_rank": 256
344
- },
345
- {
346
- "training_priority_rank": 3,
347
- "module_name": "up_blocks.1.attentions.0.transformer_blocks.1.attn1.to_q",
348
- "ldm_key": "model.diffusion_model.output_blocks.3.1.transformer_blocks.1.attn1.to_q.weight",
349
- "suffix": "attn1.to_q",
350
- "shape": [
351
- 640,
352
- 640
353
- ],
354
- "has_old_rank128": false,
355
- "has_phase1b": false,
356
- "has_phase2": false,
357
- "composite_source": "raw",
358
- "w4_mse_mean": 0.01650474433388029,
359
- "w2_raw_mse_mean": 0.12441772927663156,
360
- "w2_old_rank128_mse_mean": 0.12441772927663156,
361
- "w2_phase1b_mse_mean": 0.12441772927663156,
362
- "w2_phase2_mse_mean": 0.12441772927663156,
363
- "w2_composite_mse_mean": 0.12441772927663156,
364
- "w2_raw_to_w4_mse_ratio": 7.5383009122833515,
365
- "w2_composite_to_w4_mse_ratio": 7.5383009122833515,
366
- "old_rank128_improvement_vs_raw": 0.9999999999919625,
367
- "phase1b_improvement_vs_raw": 0.9999999999919625,
368
- "phase2_improvement_vs_raw": 0.9999999999919625,
369
- "composite_improvement_vs_raw": 0.9999999999919625,
370
- "remaining_w2_specific_gap_score": 0.8134805528411615,
371
- "recommended_action": "add_adapter_missing_layer",
372
- "recommended_rank": 256
373
- },
374
- {
375
- "training_priority_rank": 4,
376
- "module_name": "down_blocks.1.attentions.1.transformer_blocks.1.attn1.to_q",
377
- "ldm_key": "model.diffusion_model.input_blocks.5.1.transformer_blocks.1.attn1.to_q.weight",
378
- "suffix": "attn1.to_q",
379
- "shape": [
380
- 640,
381
- 640
382
- ],
383
- "has_old_rank128": false,
384
- "has_phase1b": false,
385
- "has_phase2": false,
386
- "composite_source": "raw",
387
- "w4_mse_mean": 0.016551085664624616,
388
- "w2_raw_mse_mean": 0.12307428023112672,
389
- "w2_old_rank128_mse_mean": 0.12307428023112672,
390
- "w2_phase1b_mse_mean": 0.12307428023112672,
391
- "w2_phase2_mse_mean": 0.12307428023112672,
392
- "w2_composite_mse_mean": 0.12307428023112672,
393
- "w2_raw_to_w4_mse_ratio": 7.436024603917248,
394
- "w2_composite_to_w4_mse_ratio": 7.436024603917248,
395
- "old_rank128_improvement_vs_raw": 0.9999999999918748,
396
- "phase1b_improvement_vs_raw": 0.9999999999918748,
397
- "phase2_improvement_vs_raw": 0.9999999999918748,
398
- "composite_improvement_vs_raw": 0.9999999999918748,
399
- "remaining_w2_specific_gap_score": 0.7921090956843738,
400
- "recommended_action": "add_adapter_missing_layer",
401
- "recommended_rank": 256
402
- },
403
- {
404
- "training_priority_rank": 5,
405
- "module_name": "up_blocks.1.attentions.0.transformer_blocks.0.attn1.to_q",
406
- "ldm_key": "model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn1.to_q.weight",
407
- "suffix": "attn1.to_q",
408
- "shape": [
409
- 640,
410
- 640
411
- ],
412
- "has_old_rank128": false,
413
- "has_phase1b": false,
414
- "has_phase2": false,
415
- "composite_source": "raw",
416
- "w4_mse_mean": 0.015716641160127307,
417
- "w2_raw_mse_mean": 0.11642114937837635,
418
- "w2_old_rank128_mse_mean": 0.11642114937837635,
419
- "w2_phase1b_mse_mean": 0.11642114937837635,
420
- "w2_phase2_mse_mean": 0.11642114937837635,
421
- "w2_composite_mse_mean": 0.11642114937837635,
422
- "w2_raw_to_w4_mse_ratio": 7.407508270044756,
423
- "w2_composite_to_w4_mse_ratio": 7.407508270044756,
424
- "old_rank128_improvement_vs_raw": 0.9999999999914104,
425
- "phase1b_improvement_vs_raw": 0.9999999999914104,
426
- "phase2_improvement_vs_raw": 0.9999999999914104,
427
- "composite_improvement_vs_raw": 0.9999999999914104,
428
- "remaining_w2_specific_gap_score": 0.7459694774574699,
429
- "recommended_action": "add_adapter_missing_layer",
430
- "recommended_rank": 256
431
- },
432
- {
433
- "training_priority_rank": 6,
434
- "module_name": "up_blocks.1.attentions.1.transformer_blocks.1.attn1.to_k",
435
- "ldm_key": "model.diffusion_model.output_blocks.4.1.transformer_blocks.1.attn1.to_k.weight",
436
- "suffix": "attn1.to_k",
437
- "shape": [
438
- 640,
439
- 640
440
- ],
441
- "has_old_rank128": false,
442
- "has_phase1b": false,
443
- "has_phase2": false,
444
- "composite_source": "raw",
445
- "w4_mse_mean": 0.01650732191878238,
446
- "w2_raw_mse_mean": 0.11842691459293876,
447
- "w2_old_rank128_mse_mean": 0.11842691459293876,
448
- "w2_phase1b_mse_mean": 0.11842691459293876,
449
- "w2_phase2_mse_mean": 0.11842691459293876,
450
- "w2_composite_mse_mean": 0.11842691459293876,
451
- "w2_raw_to_w4_mse_ratio": 7.174205190183874,
452
- "w2_composite_to_w4_mse_ratio": 7.174205190183874,
453
- "old_rank128_improvement_vs_raw": 0.999999999991556,
454
- "phase1b_improvement_vs_raw": 0.999999999991556,
455
- "phase2_improvement_vs_raw": 0.999999999991556,
456
- "composite_improvement_vs_raw": 0.999999999991556,
457
- "remaining_w2_specific_gap_score": 0.7311920707443591,
458
- "recommended_action": "add_adapter_missing_layer",
459
- "recommended_rank": 256
460
- },
461
- {
462
- "training_priority_rank": 7,
463
- "module_name": "mid_block.attentions.0.proj_out",
464
- "ldm_key": "model.diffusion_model.middle_block.1.proj_out.weight",
465
- "suffix": "proj_out",
466
- "shape": [
467
- 1280,
468
- 1280
469
- ],
470
- "has_old_rank128": true,
471
- "has_phase1b": true,
472
- "has_phase2": true,
473
- "composite_source": "phase2",
474
- "w4_mse_mean": 0.18414856466863835,
475
- "w2_raw_mse_mean": 1.3472472195114409,
476
- "w2_old_rank128_mse_mean": 0.85862555035523,
477
- "w2_phase1b_mse_mean": 0.4570868414427553,
478
- "w2_phase2_mse_mean": 0.4683431349694729,
479
- "w2_composite_mse_mean": 0.4683431349694729,
480
- "w2_raw_to_w4_mse_ratio": 7.316088626204586,
481
- "w2_composite_to_w4_mse_ratio": 2.543289630357305,
482
- "old_rank128_improvement_vs_raw": 1.5690742244421791,
483
- "phase1b_improvement_vs_raw": 2.9474644582986103,
484
- "phase2_improvement_vs_raw": 2.8766242502868744,
485
- "composite_improvement_vs_raw": 2.8766242502868744,
486
- "remaining_w2_specific_gap_score": 0.7227891036499625,
487
- "recommended_action": "train_block_level_or_rank256_plus_real_latents",
488
- "recommended_rank": 256
489
- },
490
- {
491
- "training_priority_rank": 8,
492
- "module_name": "up_blocks.0.attentions.1.proj_out",
493
- "ldm_key": "model.diffusion_model.output_blocks.1.1.proj_out.weight",
494
- "suffix": "proj_out",
495
- "shape": [
496
- 1280,
497
- 1280
498
- ],
499
- "has_old_rank128": true,
500
- "has_phase1b": true,
501
- "has_phase2": true,
502
- "composite_source": "phase2",
503
- "w4_mse_mean": 0.1777853944471904,
504
- "w2_raw_mse_mean": 1.3340980570231165,
505
- "w2_old_rank128_mse_mean": 0.7680485237921987,
506
- "w2_phase1b_mse_mean": 0.6045318022370338,
507
- "w2_phase2_mse_mean": 0.4564306310244969,
508
- "w2_composite_mse_mean": 0.4564306310244969,
509
- "w2_raw_to_w4_mse_ratio": 7.503980071950705,
510
- "w2_composite_to_w4_mse_ratio": 2.5673123061720813,
511
- "old_rank128_improvement_vs_raw": 1.7369970980926328,
512
- "phase1b_improvement_vs_raw": 2.2068285772297824,
513
- "phase2_improvement_vs_raw": 2.9228933518894173,
514
- "composite_improvement_vs_raw": 2.9228933518894173,
515
- "remaining_w2_specific_gap_score": 0.7153693449211498,
516
- "recommended_action": "train_block_level_or_rank256_plus_real_latents",
517
- "recommended_rank": 256
518
- },
519
- {
520
- "training_priority_rank": 9,
521
- "module_name": "up_blocks.1.attentions.0.transformer_blocks.1.attn1.to_k",
522
- "ldm_key": "model.diffusion_model.output_blocks.3.1.transformer_blocks.1.attn1.to_k.weight",
523
- "suffix": "attn1.to_k",
524
- "shape": [
525
- 640,
526
- 640
527
- ],
528
- "has_old_rank128": false,
529
- "has_phase1b": false,
530
- "has_phase2": false,
531
- "composite_source": "raw",
532
- "w4_mse_mean": 0.01555688923690468,
533
- "w2_raw_mse_mean": 0.11352902118648801,
534
- "w2_old_rank128_mse_mean": 0.11352902118648801,
535
- "w2_phase1b_mse_mean": 0.11352902118648801,
536
- "w2_phase2_mse_mean": 0.11352902118648801,
537
- "w2_composite_mse_mean": 0.11352902118648801,
538
- "w2_raw_to_w4_mse_ratio": 7.29766854094919,
539
- "w2_composite_to_w4_mse_ratio": 7.29766854094919,
540
- "old_rank128_improvement_vs_raw": 0.9999999999911916,
541
- "phase1b_improvement_vs_raw": 0.9999999999911916,
542
- "phase2_improvement_vs_raw": 0.9999999999911916,
543
- "composite_improvement_vs_raw": 0.9999999999911916,
544
- "remaining_w2_specific_gap_score": 0.7149681452181973,
545
- "recommended_action": "add_adapter_missing_layer",
546
- "recommended_rank": 256
547
- },
548
- {
549
- "training_priority_rank": 10,
550
- "module_name": "up_blocks.1.attentions.1.transformer_blocks.0.attn1.to_q",
551
- "ldm_key": "model.diffusion_model.output_blocks.4.1.transformer_blocks.0.attn1.to_q.weight",
552
- "suffix": "attn1.to_q",
553
- "shape": [
554
- 640,
555
- 640
556
- ],
557
- "has_old_rank128": false,
558
- "has_phase1b": false,
559
- "has_phase2": false,
560
- "composite_source": "raw",
561
- "w4_mse_mean": 0.015393487393989094,
562
- "w2_raw_mse_mean": 0.11155489552766085,
563
- "w2_old_rank128_mse_mean": 0.11155489552766085,
564
- "w2_phase1b_mse_mean": 0.11155489552766085,
565
- "w2_phase2_mse_mean": 0.11155489552766085,
566
- "w2_composite_mse_mean": 0.11155489552766085,
567
- "w2_raw_to_w4_mse_ratio": 7.246889068423464,
568
- "w2_composite_to_w4_mse_ratio": 7.246889068423464,
569
- "old_rank128_improvement_vs_raw": 0.9999999999910357,
570
- "phase1b_improvement_vs_raw": 0.9999999999910357,
571
- "phase2_improvement_vs_raw": 0.9999999999910357,
572
- "composite_improvement_vs_raw": 0.9999999999910357,
573
- "remaining_w2_specific_gap_score": 0.6968710574081131,
574
- "recommended_action": "add_adapter_missing_layer",
575
- "recommended_rank": 256
576
- },
577
- {
578
- "training_priority_rank": 11,
579
- "module_name": "down_blocks.1.attentions.1.transformer_blocks.1.attn1.to_k",
580
- "ldm_key": "model.diffusion_model.input_blocks.5.1.transformer_blocks.1.attn1.to_k.weight",
581
- "suffix": "attn1.to_k",
582
- "shape": [
583
- 640,
584
- 640
585
- ],
586
- "has_old_rank128": false,
587
- "has_phase1b": false,
588
- "has_phase2": false,
589
- "composite_source": "raw",
590
- "w4_mse_mean": 0.015574711058954043,
591
- "w2_raw_mse_mean": 0.11173469506736312,
592
- "w2_old_rank128_mse_mean": 0.11173469506736312,
593
- "w2_phase1b_mse_mean": 0.11173469506736312,
594
- "w2_phase2_mse_mean": 0.11173469506736312,
595
- "w2_composite_mse_mean": 0.11173469506736312,
596
- "w2_raw_to_w4_mse_ratio": 7.174110302094608,
597
- "w2_composite_to_w4_mse_ratio": 7.174110302094608,
598
- "old_rank128_improvement_vs_raw": 0.9999999999910502,
599
- "phase1b_improvement_vs_raw": 0.9999999999910502,
600
- "phase2_improvement_vs_raw": 0.9999999999910502,
601
- "composite_improvement_vs_raw": 0.9999999999910502,
602
- "remaining_w2_specific_gap_score": 0.6898623319239804,
603
- "recommended_action": "add_adapter_missing_layer",
604
- "recommended_rank": 256
605
- },
606
- {
607
- "training_priority_rank": 12,
608
- "module_name": "up_blocks.0.attentions.2.transformer_blocks.9.attn2.to_q",
609
- "ldm_key": "model.diffusion_model.output_blocks.2.1.transformer_blocks.9.attn2.to_q.weight",
610
- "suffix": "attn2.to_q",
611
- "shape": [
612
- 1280,
613
- 1280
614
- ],
615
- "has_old_rank128": false,
616
- "has_phase1b": false,
617
- "has_phase2": false,
618
- "composite_source": "raw",
619
- "w4_mse_mean": 0.006283481646927872,
620
- "w2_raw_mse_mean": 0.06878376293129154,
621
- "w2_old_rank128_mse_mean": 0.06878376293129154,
622
- "w2_phase1b_mse_mean": 0.06878376293129154,
623
- "w2_phase2_mse_mean": 0.06878376293129154,
624
- "w2_composite_mse_mean": 0.06878376293129154,
625
- "w2_raw_to_w4_mse_ratio": 10.94675958096808,
626
- "w2_composite_to_w4_mse_ratio": 10.94675958096808,
627
- "old_rank128_improvement_vs_raw": 0.9999999999854616,
628
- "phase1b_improvement_vs_raw": 0.9999999999854616,
629
- "phase2_improvement_vs_raw": 0.9999999999854616,
630
- "composite_improvement_vs_raw": 0.9999999999854616,
631
- "remaining_w2_specific_gap_score": 0.6841755529628081,
632
- "recommended_action": "add_adapter_missing_layer",
633
- "recommended_rank": 256
634
- },
635
- {
636
- "training_priority_rank": 13,
637
- "module_name": "down_blocks.1.attentions.1.transformer_blocks.0.attn1.to_q",
638
- "ldm_key": "model.diffusion_model.input_blocks.5.1.transformer_blocks.0.attn1.to_q.weight",
639
- "suffix": "attn1.to_q",
640
- "shape": [
641
- 640,
642
- 640
643
- ],
644
- "has_old_rank128": false,
645
- "has_phase1b": false,
646
- "has_phase2": false,
647
- "composite_source": "raw",
648
- "w4_mse_mean": 0.012902505862127458,
649
- "w2_raw_mse_mean": 0.0985980766958424,
650
- "w2_old_rank128_mse_mean": 0.0985980766958424,
651
- "w2_phase1b_mse_mean": 0.0985980766958424,
652
- "w2_phase2_mse_mean": 0.0985980766958424,
653
- "w2_composite_mse_mean": 0.0985980766958424,
654
- "w2_raw_to_w4_mse_ratio": 7.641777321536753,
655
- "w2_composite_to_w4_mse_ratio": 7.641777321536753,
656
- "old_rank128_improvement_vs_raw": 0.9999999999898578,
657
- "phase1b_improvement_vs_raw": 0.9999999999898578,
658
- "phase2_improvement_vs_raw": 0.9999999999898578,
659
- "composite_improvement_vs_raw": 0.9999999999898578,
660
- "remaining_w2_specific_gap_score": 0.6548664697532293,
661
- "recommended_action": "add_adapter_missing_layer",
662
- "recommended_rank": 256
663
- },
664
- {
665
- "training_priority_rank": 14,
666
- "module_name": "up_blocks.1.attentions.1.transformer_blocks.1.attn1.to_v",
667
- "ldm_key": "model.diffusion_model.output_blocks.4.1.transformer_blocks.1.attn1.to_v.weight",
668
- "suffix": "attn1.to_v",
669
- "shape": [
670
- 640,
671
- 640
672
- ],
673
- "has_old_rank128": false,
674
- "has_phase1b": false,
675
- "has_phase2": false,
676
- "composite_source": "raw",
677
- "w4_mse_mean": 0.014045619937990392,
678
- "w2_raw_mse_mean": 0.10288021394184657,
679
- "w2_old_rank128_mse_mean": 0.10288021394184657,
680
- "w2_phase1b_mse_mean": 0.10288021394184657,
681
- "w2_phase2_mse_mean": 0.10288021394184657,
682
- "w2_composite_mse_mean": 0.10288021394184657,
683
- "w2_raw_to_w4_mse_ratio": 7.3247186232238075,
684
- "w2_composite_to_w4_mse_ratio": 7.3247186232238075,
685
- "old_rank128_improvement_vs_raw": 0.9999999999902799,
686
- "phase1b_improvement_vs_raw": 0.9999999999902799,
687
- "phase2_improvement_vs_raw": 0.9999999999902799,
688
- "composite_improvement_vs_raw": 0.9999999999902799,
689
- "remaining_w2_specific_gap_score": 0.6506884050865713,
690
- "recommended_action": "add_adapter_missing_layer",
691
- "recommended_rank": 256
692
- },
693
- {
694
- "training_priority_rank": 15,
695
- "module_name": "up_blocks.1.attentions.1.transformer_blocks.0.attn1.to_k",
696
- "ldm_key": "model.diffusion_model.output_blocks.4.1.transformer_blocks.0.attn1.to_k.weight",
697
- "suffix": "attn1.to_k",
698
- "shape": [
699
- 640,
700
- 640
701
- ],
702
- "has_old_rank128": false,
703
- "has_phase1b": false,
704
- "has_phase2": false,
705
- "composite_source": "raw",
706
- "w4_mse_mean": 0.015184641383322222,
707
- "w2_raw_mse_mean": 0.10630956571549177,
708
- "w2_old_rank128_mse_mean": 0.10630956571549177,
709
- "w2_phase1b_mse_mean": 0.10630956571549177,
710
- "w2_phase2_mse_mean": 0.10630956571549177,
711
- "w2_composite_mse_mean": 0.10630956571549177,
712
- "w2_raw_to_w4_mse_ratio": 7.0011245590069615,
713
- "w2_composite_to_w4_mse_ratio": 7.0011245590069615,
714
- "old_rank128_improvement_vs_raw": 0.9999999999905934,
715
- "phase1b_improvement_vs_raw": 0.9999999999905934,
716
- "phase2_improvement_vs_raw": 0.9999999999905934,
717
- "composite_improvement_vs_raw": 0.9999999999905934,
718
- "remaining_w2_specific_gap_score": 0.6379769456796033,
719
- "recommended_action": "add_adapter_missing_layer",
720
- "recommended_rank": 256
721
- },
722
- {
723
- "training_priority_rank": 16,
724
- "module_name": "up_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k",
725
- "ldm_key": "model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn1.to_k.weight",
726
- "suffix": "attn1.to_k",
727
- "shape": [
728
- 640,
729
- 640
730
- ],
731
- "has_old_rank128": false,
732
- "has_phase1b": false,
733
- "has_phase2": false,
734
- "composite_source": "raw",
735
- "w4_mse_mean": 0.015266332110123975,
736
- "w2_raw_mse_mean": 0.10559878458401986,
737
- "w2_old_rank128_mse_mean": 0.10559878458401986,
738
- "w2_phase1b_mse_mean": 0.10559878458401986,
739
- "w2_phase2_mse_mean": 0.10559878458401986,
740
- "w2_composite_mse_mean": 0.10559878458401986,
741
- "w2_raw_to_w4_mse_ratio": 6.917102537489944,
742
- "w2_composite_to_w4_mse_ratio": 6.917102537489944,
743
- "old_rank128_improvement_vs_raw": 0.9999999999905301,
744
- "phase1b_improvement_vs_raw": 0.9999999999905301,
745
- "phase2_improvement_vs_raw": 0.9999999999905301,
746
- "composite_improvement_vs_raw": 0.9999999999905301,
747
- "remaining_w2_specific_gap_score": 0.624838836224875,
748
- "recommended_action": "add_adapter_missing_layer",
749
- "recommended_rank": 256
750
- },
751
- {
752
- "training_priority_rank": 17,
753
- "module_name": "down_blocks.1.attentions.1.transformer_blocks.0.attn1.to_k",
754
- "ldm_key": "model.diffusion_model.input_blocks.5.1.transformer_blocks.0.attn1.to_k.weight",
755
- "suffix": "attn1.to_k",
756
- "shape": [
757
- 640,
758
- 640
759
- ],
760
- "has_old_rank128": false,
761
- "has_phase1b": false,
762
- "has_phase2": false,
763
- "composite_source": "raw",
764
- "w4_mse_mean": 0.0125871665908822,
765
- "w2_raw_mse_mean": 0.09344679423208747,
766
- "w2_old_rank128_mse_mean": 0.09344679423208747,
767
- "w2_phase1b_mse_mean": 0.09344679423208747,
768
- "w2_phase2_mse_mean": 0.09344679423208747,
769
- "w2_composite_mse_mean": 0.09344679423208747,
770
- "w2_raw_to_w4_mse_ratio": 7.423973739439805,
771
- "w2_composite_to_w4_mse_ratio": 7.423973739439805,
772
- "old_rank128_improvement_vs_raw": 0.9999999999892987,
773
- "phase1b_improvement_vs_raw": 0.9999999999892987,
774
- "phase2_improvement_vs_raw": 0.9999999999892987,
775
- "composite_improvement_vs_raw": 0.9999999999892987,
776
- "remaining_w2_specific_gap_score": 0.6002997521891889,
777
- "recommended_action": "add_adapter_missing_layer",
778
- "recommended_rank": 256
779
- },
780
- {
781
- "training_priority_rank": 18,
782
- "module_name": "down_blocks.1.attentions.1.transformer_blocks.1.attn1.to_v",
783
- "ldm_key": "model.diffusion_model.input_blocks.5.1.transformer_blocks.1.attn1.to_v.weight",
784
- "suffix": "attn1.to_v",
785
- "shape": [
786
- 640,
787
- 640
788
- ],
789
- "has_old_rank128": false,
790
- "has_phase1b": false,
791
- "has_phase2": false,
792
- "composite_source": "raw",
793
- "w4_mse_mean": 0.011522587821153658,
794
- "w2_raw_mse_mean": 0.08661618735641241,
795
- "w2_old_rank128_mse_mean": 0.08661618735641241,
796
- "w2_phase1b_mse_mean": 0.08661618735641241,
797
- "w2_phase2_mse_mean": 0.08661618735641241,
798
- "w2_composite_mse_mean": 0.08661618735641241,
799
- "w2_raw_to_w4_mse_ratio": 7.517077647252264,
800
- "w2_composite_to_w4_mse_ratio": 7.517077647252264,
801
- "old_rank128_improvement_vs_raw": 0.9999999999884548,
802
- "phase1b_improvement_vs_raw": 0.9999999999884548,
803
- "phase2_improvement_vs_raw": 0.9999999999884548,
804
- "composite_improvement_vs_raw": 0.9999999999884548,
805
- "remaining_w2_specific_gap_score": 0.5644844185182065,
806
- "recommended_action": "add_adapter_missing_layer",
807
- "recommended_rank": 256
808
- },
809
- {
810
- "training_priority_rank": 19,
811
- "module_name": "down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_q",
812
- "ldm_key": "model.diffusion_model.input_blocks.4.1.transformer_blocks.0.attn1.to_q.weight",
813
- "suffix": "attn1.to_q",
814
- "shape": [
815
- 640,
816
- 640
817
- ],
818
- "has_old_rank128": false,
819
- "has_phase1b": false,
820
- "has_phase2": false,
821
- "composite_source": "raw",
822
- "w4_mse_mean": 0.011824695875735156,
823
- "w2_raw_mse_mean": 0.08770120383373328,
824
- "w2_old_rank128_mse_mean": 0.08770120383373328,
825
- "w2_phase1b_mse_mean": 0.08770120383373328,
826
- "w2_phase2_mse_mean": 0.08770120383373328,
827
- "w2_composite_mse_mean": 0.08770120383373328,
828
- "w2_raw_to_w4_mse_ratio": 7.416783040169649,
829
- "w2_composite_to_w4_mse_ratio": 7.416783040169649,
830
- "old_rank128_improvement_vs_raw": 0.9999999999885976,
831
- "phase1b_improvement_vs_raw": 0.9999999999885976,
832
- "phase2_improvement_vs_raw": 0.9999999999885976,
833
- "composite_improvement_vs_raw": 0.9999999999885976,
834
- "remaining_w2_specific_gap_score": 0.5627595973701779,
835
- "recommended_action": "add_adapter_missing_layer",
836
- "recommended_rank": 256
837
- },
838
- {
839
- "training_priority_rank": 20,
840
- "module_name": "up_blocks.1.attentions.0.transformer_blocks.0.attn1.to_v",
841
- "ldm_key": "model.diffusion_model.output_blocks.3.1.transformer_blocks.0.attn1.to_v.weight",
842
- "suffix": "attn1.to_v",
843
- "shape": [
844
- 640,
845
- 640
846
- ],
847
- "has_old_rank128": false,
848
- "has_phase1b": false,
849
- "has_phase2": false,
850
- "composite_source": "raw",
851
- "w4_mse_mean": 0.013221082560318922,
852
- "w2_raw_mse_mean": 0.09296748986733812,
853
- "w2_old_rank128_mse_mean": 0.09296748986733812,
854
- "w2_phase1b_mse_mean": 0.09296748986733812,
855
- "w2_phase2_mse_mean": 0.09296748986733812,
856
- "w2_composite_mse_mean": 0.09296748986733812,
857
- "w2_raw_to_w4_mse_ratio": 7.031760783291242,
858
- "w2_composite_to_w4_mse_ratio": 7.031760783291242,
859
- "old_rank128_improvement_vs_raw": 0.9999999999892435,
860
- "phase1b_improvement_vs_raw": 0.9999999999892435,
861
- "phase2_improvement_vs_raw": 0.9999999999892435,
862
- "composite_improvement_vs_raw": 0.9999999999892435,
863
- "remaining_w2_specific_gap_score": 0.5607576595098677,
864
- "recommended_action": "add_adapter_missing_layer",
865
- "recommended_rank": 256
866
- },
867
- {
868
- "training_priority_rank": 21,
869
- "module_name": "down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k",
870
- "ldm_key": "model.diffusion_model.input_blocks.4.1.transformer_blocks.0.attn1.to_k.weight",
871
- "suffix": "attn1.to_k",
872
- "shape": [
873
- 640,
874
- 640
875
- ],
876
- "has_old_rank128": false,
877
- "has_phase1b": false,
878
- "has_phase2": false,
879
- "composite_source": "raw",
880
- "w4_mse_mean": 0.011692194467676538,
881
- "w2_raw_mse_mean": 0.08697623320456062,
882
- "w2_old_rank128_mse_mean": 0.08697623320456062,
883
- "w2_phase1b_mse_mean": 0.08697623320456062,
884
- "w2_phase2_mse_mean": 0.08697623320456062,
885
- "w2_composite_mse_mean": 0.08697623320456062,
886
- "w2_raw_to_w4_mse_ratio": 7.438828821875183,
887
- "w2_composite_to_w4_mse_ratio": 7.438828821875183,
888
- "old_rank128_improvement_vs_raw": 0.9999999999885025,
889
- "phase1b_improvement_vs_raw": 0.9999999999885025,
890
- "phase2_improvement_vs_raw": 0.9999999999885025,
891
- "composite_improvement_vs_raw": 0.9999999999885025,
892
- "remaining_w2_specific_gap_score": 0.560025077183101,
893
- "recommended_action": "add_adapter_missing_layer",
894
- "recommended_rank": 256
895
- },
896
- {
897
- "training_priority_rank": 22,
898
- "module_name": "up_blocks.0.attentions.1.transformer_blocks.8.attn1.to_v",
899
- "ldm_key": "model.diffusion_model.output_blocks.1.1.transformer_blocks.8.attn1.to_v.weight",
900
- "suffix": "attn1.to_v",
901
- "shape": [
902
- 1280,
903
- 1280
904
- ],
905
- "has_old_rank128": false,
906
- "has_phase1b": false,
907
- "has_phase2": false,
908
- "composite_source": "raw",
909
- "w4_mse_mean": 0.01207385812553444,
910
- "w2_raw_mse_mean": 0.08799946933452572,
911
- "w2_old_rank128_mse_mean": 0.08799946933452572,
912
- "w2_phase1b_mse_mean": 0.08799946933452572,
913
- "w2_phase2_mse_mean": 0.08799946933452572,
914
- "w2_composite_mse_mean": 0.08799946933452572,
915
- "w2_raw_to_w4_mse_ratio": 7.288429962675419,
916
- "w2_composite_to_w4_mse_ratio": 7.288429962675419,
917
- "old_rank128_improvement_vs_raw": 0.9999999999886362,
918
- "phase1b_improvement_vs_raw": 0.9999999999886362,
919
- "phase2_improvement_vs_raw": 0.9999999999886362,
920
- "composite_improvement_vs_raw": 0.9999999999886362,
921
- "remaining_w2_specific_gap_score": 0.5533784996700567,
922
- "recommended_action": "add_adapter_missing_layer",
923
- "recommended_rank": 256
924
- },
925
- {
926
- "training_priority_rank": 23,
927
- "module_name": "down_blocks.1.attentions.0.transformer_blocks.1.attn1.to_q",
928
- "ldm_key": "model.diffusion_model.input_blocks.4.1.transformer_blocks.1.attn1.to_q.weight",
929
- "suffix": "attn1.to_q",
930
- "shape": [
931
- 640,
932
- 640
933
- ],
934
- "has_old_rank128": false,
935
- "has_phase1b": false,
936
- "has_phase2": false,
937
- "composite_source": "raw",
938
- "w4_mse_mean": 0.012419876642525196,
939
- "w2_raw_mse_mean": 0.08907664686973606,
940
- "w2_old_rank128_mse_mean": 0.08907664686973606,
941
- "w2_phase1b_mse_mean": 0.08907664686973606,
942
- "w2_phase2_mse_mean": 0.08907664686973606,
943
- "w2_composite_mse_mean": 0.08907664686973606,
944
- "w2_raw_to_w4_mse_ratio": 7.1721039931724295,
945
- "w2_composite_to_w4_mse_ratio": 7.1721039931724295,
946
- "old_rank128_improvement_vs_raw": 0.9999999999887736,
947
- "phase1b_improvement_vs_raw": 0.9999999999887736,
948
- "phase2_improvement_vs_raw": 0.9999999999887736,
949
- "composite_improvement_vs_raw": 0.9999999999887736,
950
- "remaining_w2_specific_gap_score": 0.5497903278502804,
951
- "recommended_action": "add_adapter_missing_layer",
952
- "recommended_rank": 256
953
- },
954
- {
955
- "training_priority_rank": 24,
956
- "module_name": "up_blocks.1.attentions.2.transformer_blocks.1.attn1.to_q",
957
- "ldm_key": "model.diffusion_model.output_blocks.5.1.transformer_blocks.1.attn1.to_q.weight",
958
- "suffix": "attn1.to_q",
959
- "shape": [
960
- 640,
961
- 640
962
- ],
963
- "has_old_rank128": false,
964
- "has_phase1b": false,
965
- "has_phase2": false,
966
- "composite_source": "raw",
967
- "w4_mse_mean": 0.012714513893505293,
968
- "w2_raw_mse_mean": 0.0901942936969655,
969
- "w2_old_rank128_mse_mean": 0.0901942936969655,
970
- "w2_phase1b_mse_mean": 0.0901942936969655,
971
- "w2_phase2_mse_mean": 0.0901942936969655,
972
- "w2_composite_mse_mean": 0.0901942936969655,
973
- "w2_raw_to_w4_mse_ratio": 7.09380590129709,
974
- "w2_composite_to_w4_mse_ratio": 7.09380590129709,
975
- "old_rank128_improvement_vs_raw": 0.9999999999889128,
976
- "phase1b_improvement_vs_raw": 0.9999999999889128,
977
- "phase2_improvement_vs_raw": 0.9999999999889128,
978
- "composite_improvement_vs_raw": 0.9999999999889128,
979
- "remaining_w2_specific_gap_score": 0.549626519200985,
980
- "recommended_action": "add_adapter_missing_layer",
981
- "recommended_rank": 256
982
- },
983
- {
984
- "training_priority_rank": 25,
985
- "module_name": "up_blocks.0.attentions.1.transformer_blocks.7.attn1.to_v",
986
- "ldm_key": "model.diffusion_model.output_blocks.1.1.transformer_blocks.7.attn1.to_v.weight",
987
- "suffix": "attn1.to_v",
988
- "shape": [
989
- 1280,
990
- 1280
991
- ],
992
- "has_old_rank128": false,
993
- "has_phase1b": false,
994
- "has_phase2": false,
995
- "composite_source": "raw",
996
- "w4_mse_mean": 0.012551478842007262,
997
- "w2_raw_mse_mean": 0.08876089937984943,
998
- "w2_old_rank128_mse_mean": 0.08876089937984943,
999
- "w2_phase1b_mse_mean": 0.08876089937984943,
1000
- "w2_phase2_mse_mean": 0.08876089937984943,
1001
- "w2_composite_mse_mean": 0.08876089937984943,
1002
- "w2_raw_to_w4_mse_ratio": 7.071748316677466,
1003
- "w2_composite_to_w4_mse_ratio": 7.071748316677466,
1004
- "old_rank128_improvement_vs_raw": 0.9999999999887337,
1005
- "phase1b_improvement_vs_raw": 0.9999999999887337,
1006
- "phase2_improvement_vs_raw": 0.9999999999887337,
1007
- "composite_improvement_vs_raw": 0.9999999999887337,
1008
- "remaining_w2_specific_gap_score": 0.5389338414034505,
1009
- "recommended_action": "add_adapter_missing_layer",
1010
- "recommended_rank": 256
1011
- },
1012
- {
1013
- "training_priority_rank": 26,
1014
- "module_name": "up_blocks.1.attentions.2.transformer_blocks.0.attn1.to_k",
1015
- "ldm_key": "model.diffusion_model.output_blocks.5.1.transformer_blocks.0.attn1.to_k.weight",
1016
- "suffix": "attn1.to_k",
1017
- "shape": [
1018
- 640,
1019
- 640
1020
- ],
1021
- "has_old_rank128": false,
1022
- "has_phase1b": false,
1023
- "has_phase2": false,
1024
- "composite_source": "raw",
1025
- "w4_mse_mean": 0.011307834963580328,
1026
- "w2_raw_mse_mean": 0.08250298537313938,
1027
- "w2_old_rank128_mse_mean": 0.08250298537313938,
1028
- "w2_phase1b_mse_mean": 0.08250298537313938,
1029
- "w2_phase2_mse_mean": 0.08250298537313938,
1030
- "w2_composite_mse_mean": 0.08250298537313938,
1031
- "w2_raw_to_w4_mse_ratio": 7.296090333080073,
1032
- "w2_composite_to_w4_mse_ratio": 7.296090333080073,
1033
- "old_rank128_improvement_vs_raw": 0.9999999999878791,
1034
- "phase1b_improvement_vs_raw": 0.9999999999878791,
1035
- "phase2_improvement_vs_raw": 0.9999999999878791,
1036
- "composite_improvement_vs_raw": 0.9999999999878791,
1037
- "remaining_w2_specific_gap_score": 0.5194462486653656,
1038
- "recommended_action": "add_adapter_missing_layer",
1039
- "recommended_rank": 256
1040
- },
1041
- {
1042
- "training_priority_rank": 27,
1043
- "module_name": "down_blocks.1.attentions.0.transformer_blocks.1.attn1.to_k",
1044
- "ldm_key": "model.diffusion_model.input_blocks.4.1.transformer_blocks.1.attn1.to_k.weight",
1045
- "suffix": "attn1.to_k",
1046
- "shape": [
1047
- 640,
1048
- 640
1049
- ],
1050
- "has_old_rank128": false,
1051
- "has_phase1b": false,
1052
- "has_phase2": false,
1053
- "composite_source": "raw",
1054
- "w4_mse_mean": 0.012032134258853537,
1055
- "w2_raw_mse_mean": 0.08493821855102267,
1056
- "w2_old_rank128_mse_mean": 0.08493821855102267,
1057
- "w2_phase1b_mse_mean": 0.08493821855102267,
1058
- "w2_phase2_mse_mean": 0.08493821855102267,
1059
- "w2_composite_mse_mean": 0.08493821855102267,
1060
- "w2_raw_to_w4_mse_ratio": 7.059281148019419,
1061
- "w2_composite_to_w4_mse_ratio": 7.059281148019419,
1062
- "old_rank128_improvement_vs_raw": 0.9999999999882266,
1063
- "phase1b_improvement_vs_raw": 0.9999999999882266,
1064
- "phase2_improvement_vs_raw": 0.9999999999882266,
1065
- "composite_improvement_vs_raw": 0.9999999999882266,
1066
- "remaining_w2_specific_gap_score": 0.5146645464196242,
1067
- "recommended_action": "add_adapter_missing_layer",
1068
- "recommended_rank": 256
1069
- },
1070
- {
1071
- "training_priority_rank": 28,
1072
- "module_name": "up_blocks.0.attentions.1.transformer_blocks.8.attn1.to_q",
1073
- "ldm_key": "model.diffusion_model.output_blocks.1.1.transformer_blocks.8.attn1.to_q.weight",
1074
- "suffix": "attn1.to_q",
1075
- "shape": [
1076
- 1280,
1077
- 1280
1078
- ],
1079
- "has_old_rank128": false,
1080
- "has_phase1b": false,
1081
- "has_phase2": false,
1082
- "composite_source": "raw",
1083
- "w4_mse_mean": 0.01177227936152901,
1084
- "w2_raw_mse_mean": 0.08393190681402173,
1085
- "w2_old_rank128_mse_mean": 0.08393190681402173,
1086
- "w2_phase1b_mse_mean": 0.08393190681402173,
1087
- "w2_phase2_mse_mean": 0.08393190681402173,
1088
- "w2_composite_mse_mean": 0.08393190681402173,
1089
- "w2_raw_to_w4_mse_ratio": 7.129622414600161,
1090
- "w2_composite_to_w4_mse_ratio": 7.129622414600161,
1091
- "old_rank128_improvement_vs_raw": 0.9999999999880855,
1092
- "phase1b_improvement_vs_raw": 0.9999999999880855,
1093
- "phase2_improvement_vs_raw": 0.9999999999880855,
1094
- "composite_improvement_vs_raw": 0.9999999999880855,
1095
- "remaining_w2_specific_gap_score": 0.5144708973144891,
1096
- "recommended_action": "add_adapter_missing_layer",
1097
- "recommended_rank": 256
1098
- },
1099
- {
1100
- "training_priority_rank": 29,
1101
- "module_name": "up_blocks.1.attentions.2.transformer_blocks.0.attn1.to_q",
1102
- "ldm_key": "model.diffusion_model.output_blocks.5.1.transformer_blocks.0.attn1.to_q.weight",
1103
- "suffix": "attn1.to_q",
1104
- "shape": [
1105
- 640,
1106
- 640
1107
- ],
1108
- "has_old_rank128": false,
1109
- "has_phase1b": false,
1110
- "has_phase2": false,
1111
- "composite_source": "raw",
1112
- "w4_mse_mean": 0.011715796842638935,
1113
- "w2_raw_mse_mean": 0.0831726472824812,
1114
- "w2_old_rank128_mse_mean": 0.0831726472824812,
1115
- "w2_phase1b_mse_mean": 0.0831726472824812,
1116
- "w2_phase2_mse_mean": 0.0831726472824812,
1117
- "w2_composite_mse_mean": 0.0831726472824812,
1118
- "w2_raw_to_w4_mse_ratio": 7.09918824920898,
1119
- "w2_composite_to_w4_mse_ratio": 7.09918824920898,
1120
- "old_rank128_improvement_vs_raw": 0.9999999999879767,
1121
- "phase1b_improvement_vs_raw": 0.9999999999879767,
1122
- "phase2_improvement_vs_raw": 0.9999999999879767,
1123
- "composite_improvement_vs_raw": 0.9999999999879767,
1124
- "remaining_w2_specific_gap_score": 0.5072856329680118,
1125
- "recommended_action": "add_adapter_missing_layer",
1126
- "recommended_rank": 256
1127
- },
1128
- {
1129
- "training_priority_rank": 30,
1130
- "module_name": "up_blocks.1.attentions.2.transformer_blocks.1.attn1.to_k",
1131
- "ldm_key": "model.diffusion_model.output_blocks.5.1.transformer_blocks.1.attn1.to_k.weight",
1132
- "suffix": "attn1.to_k",
1133
- "shape": [
1134
- 640,
1135
- 640
1136
- ],
1137
- "has_old_rank128": false,
1138
- "has_phase1b": false,
1139
- "has_phase2": false,
1140
- "composite_source": "raw",
1141
- "w4_mse_mean": 0.011052693789159613,
1142
- "w2_raw_mse_mean": 0.0802028871008328,
1143
- "w2_old_rank128_mse_mean": 0.0802028871008328,
1144
- "w2_phase1b_mse_mean": 0.0802028871008328,
1145
- "w2_phase2_mse_mean": 0.0802028871008328,
1146
- "w2_composite_mse_mean": 0.0802028871008328,
1147
- "w2_raw_to_w4_mse_ratio": 7.25641084639825,
1148
- "w2_composite_to_w4_mse_ratio": 7.25641084639825,
1149
- "old_rank128_improvement_vs_raw": 0.9999999999875315,
1150
- "phase1b_improvement_vs_raw": 0.9999999999875315,
1151
- "phase2_improvement_vs_raw": 0.9999999999875315,
1152
- "composite_improvement_vs_raw": 0.9999999999875315,
1153
- "remaining_w2_specific_gap_score": 0.501782212777361,
1154
- "recommended_action": "add_adapter_missing_layer",
1155
- "recommended_rank": 256
1156
- },
1157
- {
1158
- "training_priority_rank": 31,
1159
- "module_name": "up_blocks.0.attentions.0.transformer_blocks.6.attn1.to_q",
1160
- "ldm_key": "model.diffusion_model.output_blocks.0.1.transformer_blocks.6.attn1.to_q.weight",
1161
- "suffix": "attn1.to_q",
1162
- "shape": [
1163
- 1280,
1164
- 1280
1165
- ],
1166
- "has_old_rank128": false,
1167
- "has_phase1b": false,
1168
- "has_phase2": false,
1169
- "composite_source": "raw",
1170
- "w4_mse_mean": 0.010030013425940914,
1171
- "w2_raw_mse_mean": 0.076130835339427,
1172
- "w2_old_rank128_mse_mean": 0.076130835339427,
1173
- "w2_phase1b_mse_mean": 0.076130835339427,
1174
- "w2_phase2_mse_mean": 0.076130835339427,
1175
- "w2_composite_mse_mean": 0.076130835339427,
1176
- "w2_raw_to_w4_mse_ratio": 7.590302435182919,
1177
- "w2_composite_to_w4_mse_ratio": 7.590302435182919,
1178
- "old_rank128_improvement_vs_raw": 0.9999999999868646,
1179
- "phase1b_improvement_vs_raw": 0.9999999999868646,
1180
- "phase2_improvement_vs_raw": 0.9999999999868646,
1181
- "composite_improvement_vs_raw": 0.9999999999868646,
1182
- "remaining_w2_specific_gap_score": 0.5017252295375259,
1183
- "recommended_action": "add_adapter_missing_layer",
1184
- "recommended_rank": 256
1185
- },
1186
- {
1187
- "training_priority_rank": 32,
1188
- "module_name": "up_blocks.0.attentions.1.transformer_blocks.7.attn1.to_k",
1189
- "ldm_key": "model.diffusion_model.output_blocks.1.1.transformer_blocks.7.attn1.to_k.weight",
1190
- "suffix": "attn1.to_k",
1191
- "shape": [
1192
- 1280,
1193
- 1280
1194
- ],
1195
- "has_old_rank128": false,
1196
- "has_phase1b": false,
1197
- "has_phase2": false,
1198
- "composite_source": "raw",
1199
- "w4_mse_mean": 0.012142672136958157,
1200
- "w2_raw_mse_mean": 0.08435673732310534,
1201
- "w2_old_rank128_mse_mean": 0.08435673732310534,
1202
- "w2_phase1b_mse_mean": 0.08435673732310534,
1203
- "w2_phase2_mse_mean": 0.08435673732310534,
1204
- "w2_composite_mse_mean": 0.08435673732310534,
1205
- "w2_raw_to_w4_mse_ratio": 6.9471312709996536,
1206
- "w2_composite_to_w4_mse_ratio": 6.9471312709996536,
1207
- "old_rank128_improvement_vs_raw": 0.9999999999881455,
1208
- "phase1b_improvement_vs_raw": 0.9999999999881455,
1209
- "phase2_improvement_vs_raw": 0.9999999999881455,
1210
- "composite_improvement_vs_raw": 0.9999999999881455,
1211
- "remaining_w2_specific_gap_score": 0.5016805904606905,
1212
- "recommended_action": "add_adapter_missing_layer",
1213
- "recommended_rank": 256
1214
- },
1215
- {
1216
- "training_priority_rank": 33,
1217
- "module_name": "up_blocks.0.attentions.0.transformer_blocks.4.attn1.to_k",
1218
- "ldm_key": "model.diffusion_model.output_blocks.0.1.transformer_blocks.4.attn1.to_k.weight",
1219
- "suffix": "attn1.to_k",
1220
- "shape": [
1221
- 1280,
1222
- 1280
1223
- ],
1224
- "has_old_rank128": false,
1225
- "has_phase1b": false,
1226
- "has_phase2": false,
1227
- "composite_source": "raw",
1228
- "w4_mse_mean": 0.010702501915927445,
1229
- "w2_raw_mse_mean": 0.07881842686661653,
1230
- "w2_old_rank128_mse_mean": 0.07881842686661653,
1231
- "w2_phase1b_mse_mean": 0.07881842686661653,
1232
- "w2_phase2_mse_mean": 0.07881842686661653,
1233
- "w2_composite_mse_mean": 0.07881842686661653,
1234
- "w2_raw_to_w4_mse_ratio": 7.364486124684041,
1235
- "w2_composite_to_w4_mse_ratio": 7.364486124684041,
1236
- "old_rank128_improvement_vs_raw": 0.9999999999873126,
1237
- "phase1b_improvement_vs_raw": 0.9999999999873126,
1238
- "phase2_improvement_vs_raw": 0.9999999999873126,
1239
- "composite_improvement_vs_raw": 0.9999999999873126,
1240
- "remaining_w2_specific_gap_score": 0.5016387841693692,
1241
- "recommended_action": "add_adapter_missing_layer",
1242
- "recommended_rank": 256
1243
- },
1244
- {
1245
- "training_priority_rank": 34,
1246
- "module_name": "up_blocks.0.attentions.0.transformer_blocks.4.attn1.to_q",
1247
- "ldm_key": "model.diffusion_model.output_blocks.0.1.transformer_blocks.4.attn1.to_q.weight",
1248
- "suffix": "attn1.to_q",
1249
- "shape": [
1250
- 1280,
1251
- 1280
1252
- ],
1253
- "has_old_rank128": false,
1254
- "has_phase1b": false,
1255
- "has_phase2": false,
1256
- "composite_source": "raw",
1257
- "w4_mse_mean": 0.010684565009017075,
1258
- "w2_raw_mse_mean": 0.0787019906565547,
1259
- "w2_old_rank128_mse_mean": 0.0787019906565547,
1260
- "w2_phase1b_mse_mean": 0.0787019906565547,
1261
- "w2_phase2_mse_mean": 0.0787019906565547,
1262
- "w2_composite_mse_mean": 0.0787019906565547,
1263
- "w2_raw_to_w4_mse_ratio": 7.365951780233393,
1264
- "w2_composite_to_w4_mse_ratio": 7.365951780233393,
1265
- "old_rank128_improvement_vs_raw": 0.9999999999872937,
1266
- "phase1b_improvement_vs_raw": 0.9999999999872937,
1267
- "phase2_improvement_vs_raw": 0.9999999999872937,
1268
- "composite_improvement_vs_raw": 0.9999999999872937,
1269
- "remaining_w2_specific_gap_score": 0.5010130775353723,
1270
- "recommended_action": "add_adapter_missing_layer",
1271
- "recommended_rank": 256
1272
- },
1273
- {
1274
- "training_priority_rank": 35,
1275
- "module_name": "up_blocks.0.attentions.0.transformer_blocks.9.attn2.to_q",
1276
- "ldm_key": "model.diffusion_model.output_blocks.0.1.transformer_blocks.9.attn2.to_q.weight",
1277
- "suffix": "attn2.to_q",
1278
- "shape": [
1279
- 1280,
1280
- 1280
1281
- ],
1282
- "has_old_rank128": false,
1283
- "has_phase1b": false,
1284
- "has_phase2": false,
1285
- "composite_source": "raw",
1286
- "w4_mse_mean": 0.004660304186732641,
1287
- "w2_raw_mse_mean": 0.050641766084092,
1288
- "w2_old_rank128_mse_mean": 0.050641766084092,
1289
- "w2_phase1b_mse_mean": 0.050641766084092,
1290
- "w2_phase2_mse_mean": 0.050641766084092,
1291
- "w2_composite_mse_mean": 0.050641766084092,
1292
- "w2_raw_to_w4_mse_ratio": 10.866622444388234,
1293
- "w2_composite_to_w4_mse_ratio": 10.866622444388234,
1294
- "old_rank128_improvement_vs_raw": 0.9999999999802535,
1295
- "phase1b_improvement_vs_raw": 0.9999999999802535,
1296
- "phase2_improvement_vs_raw": 0.9999999999802535,
1297
- "composite_improvement_vs_raw": 0.9999999999802535,
1298
- "remaining_w2_specific_gap_score": 0.49966318587962766,
1299
- "recommended_action": "add_adapter_missing_layer",
1300
- "recommended_rank": 256
1301
- },
1302
- {
1303
- "training_priority_rank": 36,
1304
- "module_name": "mid_block.attentions.0.transformer_blocks.4.attn1.to_v",
1305
- "ldm_key": "model.diffusion_model.middle_block.1.transformer_blocks.4.attn1.to_v.weight",
1306
- "suffix": "attn1.to_v",
1307
- "shape": [
1308
- 1280,
1309
- 1280
1310
- ],
1311
- "has_old_rank128": false,
1312
- "has_phase1b": false,
1313
- "has_phase2": false,
1314
- "composite_source": "raw",
1315
- "w4_mse_mean": 0.010012863469975335,
1316
- "w2_raw_mse_mean": 0.07585773031626429,
1317
- "w2_old_rank128_mse_mean": 0.07585773031626429,
1318
- "w2_phase1b_mse_mean": 0.07585773031626429,
1319
- "w2_phase2_mse_mean": 0.07585773031626429,
1320
- "w2_composite_mse_mean": 0.07585773031626429,
1321
- "w2_raw_to_w4_mse_ratio": 7.576027630473136,
1322
- "w2_composite_to_w4_mse_ratio": 7.576027630473136,
1323
- "old_rank128_improvement_vs_raw": 0.9999999999868173,
1324
- "phase1b_improvement_vs_raw": 0.9999999999868173,
1325
- "phase2_improvement_vs_raw": 0.9999999999868173,
1326
- "composite_improvement_vs_raw": 0.9999999999868173,
1327
- "remaining_w2_specific_gap_score": 0.49884253055230965,
1328
- "recommended_action": "add_adapter_missing_layer",
1329
- "recommended_rank": 256
1330
- },
1331
- {
1332
- "training_priority_rank": 37,
1333
- "module_name": "up_blocks.0.attentions.0.transformer_blocks.0.attn1.to_k",
1334
- "ldm_key": "model.diffusion_model.output_blocks.0.1.transformer_blocks.0.attn1.to_k.weight",
1335
- "suffix": "attn1.to_k",
1336
- "shape": [
1337
- 1280,
1338
- 1280
1339
- ],
1340
- "has_old_rank128": false,
1341
- "has_phase1b": false,
1342
- "has_phase2": false,
1343
- "composite_source": "raw",
1344
- "w4_mse_mean": 0.006486877957026341,
1345
- "w2_raw_mse_mean": 0.060104410363627334,
1346
- "w2_old_rank128_mse_mean": 0.060104410363627334,
1347
- "w2_phase1b_mse_mean": 0.060104410363627334,
1348
- "w2_phase2_mse_mean": 0.060104410363627334,
1349
- "w2_composite_mse_mean": 0.060104410363627334,
1350
- "w2_raw_to_w4_mse_ratio": 9.265537405287388,
1351
- "w2_composite_to_w4_mse_ratio": 9.265537405287388,
1352
- "old_rank128_improvement_vs_raw": 0.9999999999833623,
1353
- "phase1b_improvement_vs_raw": 0.9999999999833623,
1354
- "phase2_improvement_vs_raw": 0.9999999999833623,
1355
- "composite_improvement_vs_raw": 0.9999999999833623,
1356
- "remaining_w2_specific_gap_score": 0.4967952520925702,
1357
- "recommended_action": "add_adapter_missing_layer",
1358
- "recommended_rank": 256
1359
- },
1360
- {
1361
- "training_priority_rank": 38,
1362
- "module_name": "up_blocks.0.attentions.1.transformer_blocks.7.attn1.to_q",
1363
- "ldm_key": "model.diffusion_model.output_blocks.1.1.transformer_blocks.7.attn1.to_q.weight",
1364
- "suffix": "attn1.to_q",
1365
- "shape": [
1366
- 1280,
1367
- 1280
1368
- ],
1369
- "has_old_rank128": false,
1370
- "has_phase1b": false,
1371
- "has_phase2": false,
1372
- "composite_source": "raw",
1373
- "w4_mse_mean": 0.012249513346302723,
1374
- "w2_raw_mse_mean": 0.08413445058145694,
1375
- "w2_old_rank128_mse_mean": 0.08413445058145694,
1376
- "w2_phase1b_mse_mean": 0.08413445058145694,
1377
- "w2_phase2_mse_mean": 0.08413445058145694,
1378
- "w2_composite_mse_mean": 0.08413445058145694,
1379
- "w2_raw_to_w4_mse_ratio": 6.868391273681325,
1380
- "w2_composite_to_w4_mse_ratio": 6.868391273681325,
1381
- "old_rank128_improvement_vs_raw": 0.9999999999881142,
1382
- "phase1b_improvement_vs_raw": 0.9999999999881142,
1383
- "phase2_improvement_vs_raw": 0.9999999999881142,
1384
- "composite_improvement_vs_raw": 0.9999999999881142,
1385
- "remaining_w2_specific_gap_score": 0.49373387561506293,
1386
- "recommended_action": "add_adapter_missing_layer",
1387
- "recommended_rank": 256
1388
- },
1389
- {
1390
- "training_priority_rank": 39,
1391
- "module_name": "mid_block.attentions.0.transformer_blocks.4.attn1.to_q",
1392
- "ldm_key": "model.diffusion_model.middle_block.1.transformer_blocks.4.attn1.to_q.weight",
1393
- "suffix": "attn1.to_q",
1394
- "shape": [
1395
- 1280,
1396
- 1280
1397
- ],
1398
- "has_old_rank128": false,
1399
- "has_phase1b": false,
1400
- "has_phase2": false,
1401
- "composite_source": "raw",
1402
- "w4_mse_mean": 0.008935813060296434,
1403
- "w2_raw_mse_mean": 0.07091100793331861,
1404
- "w2_old_rank128_mse_mean": 0.07091100793331861,
1405
- "w2_phase1b_mse_mean": 0.07091100793331861,
1406
- "w2_phase2_mse_mean": 0.07091100793331861,
1407
- "w2_composite_mse_mean": 0.07091100793331861,
1408
- "w2_raw_to_w4_mse_ratio": 7.935596620799343,
1409
- "w2_composite_to_w4_mse_ratio": 7.935596620799343,
1410
- "old_rank128_improvement_vs_raw": 0.9999999999858977,
1411
- "phase1b_improvement_vs_raw": 0.9999999999858977,
1412
- "phase2_improvement_vs_raw": 0.9999999999858977,
1413
- "composite_improvement_vs_raw": 0.9999999999858977,
1414
- "remaining_w2_specific_gap_score": 0.49181014700773557,
1415
- "recommended_action": "add_adapter_missing_layer",
1416
- "recommended_rank": 256
1417
- },
1418
- {
1419
- "training_priority_rank": 40,
1420
- "module_name": "up_blocks.0.attentions.0.transformer_blocks.1.attn1.to_k",
1421
- "ldm_key": "model.diffusion_model.output_blocks.0.1.transformer_blocks.1.attn1.to_k.weight",
1422
- "suffix": "attn1.to_k",
1423
- "shape": [
1424
- 1280,
1425
- 1280
1426
- ],
1427
- "has_old_rank128": false,
1428
- "has_phase1b": false,
1429
- "has_phase2": false,
1430
- "composite_source": "raw",
1431
- "w4_mse_mean": 0.007506641941810293,
1432
- "w2_raw_mse_mean": 0.06461270240002445,
1433
- "w2_old_rank128_mse_mean": 0.06461270240002445,
1434
- "w2_phase1b_mse_mean": 0.06461270240002445,
1435
- "w2_phase2_mse_mean": 0.06461270240002445,
1436
- "w2_composite_mse_mean": 0.06461270240002445,
1437
- "w2_raw_to_w4_mse_ratio": 8.60740433502482,
1438
- "w2_composite_to_w4_mse_ratio": 8.60740433502482,
1439
- "old_rank128_improvement_vs_raw": 0.999999999984523,
1440
- "phase1b_improvement_vs_raw": 0.999999999984523,
1441
- "phase2_improvement_vs_raw": 0.999999999984523,
1442
- "composite_improvement_vs_raw": 0.999999999984523,
1443
- "remaining_w2_specific_gap_score": 0.491534952344222,
1444
- "recommended_action": "add_adapter_missing_layer",
1445
- "recommended_rank": 256
1446
- }
1447
- ]
1448
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adaptertestsfor2bitday2/orbitquant_W4_vs_W2_diagnostic_report.xlsx DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6090b8819e0936338bfd36a5e24f51be39770f8a6dad831b66d359e13632cc32
3
- size 708266
 
 
 
 
adaptertestsfor2bitday2/orbitquant_W4_vs_W2_next_training_plan.json DELETED
The diff for this file is too large to render. See raw diff
 
adaptertestsfor2bitday2/orbitquant_W4_vs_W2_top_training_targets.csv DELETED
The diff for this file is too large to render. See raw diff
 
adaptertestsfor2bitday2/temp.txt DELETED
File without changes
adaptertestsfor2bitday2/w4_w2_compare_condition_cache_captions_only_random_latents_1024x1024_8.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0ee428ea02342c6d3d918e67bad07ad166327f4a4000a2fd9048562d7b79a781
3
- size 3605469