Hugging Face's logo

FINAL-Bench
/
AETHER-7B-7Attn-base

Text Generation
Transformers
Safetensors
Korean
English
aether_v2_7way
aether
vidraft
open-weight
heterogeneous-attention
custom_code
Model card Files
xet
 Community
AETHER-7B-7Attn-base
File size: 16,060 Bytes
b4e1b81
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
<svg xmlns="http://www.w3.org/2000/svg" width="634" height="700" viewBox="0 0 634 700" font-family="Inter,Segoe UI,sans-serif">
<rect width="634" height="700" fill="#0d1117"/>
<text x="317.0" y="42" fill="#e6edf3" font-size="25" font-weight="700" text-anchor="middle">AETHER-7B-7Attn — Attention Layout</text>
<text x="317.0" y="70" fill="#8b949e" font-size="14" text-anchor="middle">7 heterogeneous mechanisms · 7x7 Latin square · 49 layers</text>
<text x="44" y="138.0" fill="#8b949e" font-size="12" text-anchor="end">R1</text>
<text x="95.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C1</text>
<text x="44" y="212.0" fill="#8b949e" font-size="12" text-anchor="end">R2</text>
<text x="169.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C2</text>
<text x="44" y="286.0" fill="#8b949e" font-size="12" text-anchor="end">R3</text>
<text x="243.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C3</text>
<text x="44" y="360.0" fill="#8b949e" font-size="12" text-anchor="end">R4</text>
<text x="317.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C4</text>
<text x="44" y="434.0" fill="#8b949e" font-size="12" text-anchor="end">R5</text>
<text x="391.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C5</text>
<text x="44" y="508.0" fill="#8b949e" font-size="12" text-anchor="end">R6</text>
<text x="465.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C6</text>
<text x="44" y="582.0" fill="#8b949e" font-size="12" text-anchor="end">R7</text>
<text x="539.0" y="84" fill="#8b949e" font-size="12" text-anchor="middle">C7</text>
<rect x="61" y="99" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="95.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="95.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L1</text>
<rect x="135" y="99" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="169.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="169.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L2</text>
<rect x="209" y="99" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="243.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="243.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L3</text>
<rect x="283" y="99" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="317.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="317.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L4</text>
<rect x="357" y="99" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="391.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="391.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L5</text>
<rect x="431" y="99" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="465.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="465.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L6</text>
<rect x="505" y="99" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="539.0" y="141.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="539.0" y="159" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L7</text>
<rect x="61" y="173" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="95.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="95.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L8</text>
<rect x="135" y="173" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="169.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="169.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L9</text>
<rect x="209" y="173" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="243.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="243.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L10</text>
<rect x="283" y="173" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="317.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="317.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L11</text>
<rect x="357" y="173" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="391.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="391.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L12</text>
<rect x="431" y="173" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="465.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="465.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L13</text>
<rect x="505" y="173" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="539.0" y="215.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="539.0" y="233" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L14</text>
<rect x="61" y="247" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="95.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="95.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L15</text>
<rect x="135" y="247" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="169.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="169.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L16</text>
<rect x="209" y="247" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="243.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="243.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L17</text>
<rect x="283" y="247" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="317.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="317.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L18</text>
<rect x="357" y="247" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="391.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="391.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L19</text>
<rect x="431" y="247" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="465.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="465.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L20</text>
<rect x="505" y="247" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="539.0" y="289.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="539.0" y="307" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L21</text>
<rect x="61" y="321" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="95.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="95.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L22</text>
<rect x="135" y="321" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="169.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="169.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L23</text>
<rect x="209" y="321" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="243.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="243.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L24</text>
<rect x="283" y="321" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="317.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="317.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L25</text>
<rect x="357" y="321" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="391.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="391.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L26</text>
<rect x="431" y="321" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="465.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="465.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L27</text>
<rect x="505" y="321" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="539.0" y="363.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="539.0" y="381" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L28</text>
<rect x="61" y="395" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="95.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="95.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L29</text>
<rect x="135" y="395" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="169.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="169.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L30</text>
<rect x="209" y="395" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="243.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="243.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L31</text>
<rect x="283" y="395" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="317.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="317.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L32</text>
<rect x="357" y="395" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="391.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="391.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L33</text>
<rect x="431" y="395" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="465.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="465.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L34</text>
<rect x="505" y="395" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="539.0" y="437.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="539.0" y="455" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L35</text>
<rect x="61" y="469" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="95.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="95.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L36</text>
<rect x="135" y="469" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="169.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="169.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L37</text>
<rect x="209" y="469" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="243.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="243.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L38</text>
<rect x="283" y="469" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="317.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="317.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L39</text>
<rect x="357" y="469" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="391.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="391.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L40</text>
<rect x="431" y="469" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="465.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="465.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L41</text>
<rect x="505" y="469" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="539.0" y="511.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="539.0" y="529" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L42</text>
<rect x="61" y="543" width="68" height="68" rx="10" fill="#FF6B9D" opacity="0.88"/>
<text x="95.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">G</text>
<text x="95.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L43</text>
<rect x="135" y="543" width="68" height="68" rx="10" fill="#6C8EF5" opacity="0.88"/>
<text x="169.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">A</text>
<text x="169.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L44</text>
<rect x="209" y="543" width="68" height="68" rx="10" fill="#F59E6C" opacity="0.88"/>
<text x="243.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">B</text>
<text x="243.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L45</text>
<rect x="283" y="543" width="68" height="68" rx="10" fill="#7BC96F" opacity="0.88"/>
<text x="317.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">C</text>
<text x="317.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L46</text>
<rect x="357" y="543" width="68" height="68" rx="10" fill="#C77DFF" opacity="0.88"/>
<text x="391.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">D</text>
<text x="391.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L47</text>
<rect x="431" y="543" width="68" height="68" rx="10" fill="#FFD166" opacity="0.88"/>
<text x="465.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">E</text>
<text x="465.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L48</text>
<rect x="505" y="543" width="68" height="68" rx="10" fill="#4ECDC4" opacity="0.88"/>
<text x="539.0" y="585.0" fill="#0d1117" font-size="24" font-weight="700" text-anchor="middle">F</text>
<text x="539.0" y="603" fill="#0d1117" font-size="10" opacity="0.65" text-anchor="middle">L49</text>
<text x="58" y="644" fill="#8b949e" font-size="13">Each of the 7 mechanisms (A-G) appears exactly once per row and per column.</text>
<text x="58" y="666" fill="#6e7681" font-size="12">L1-L49 = layer index. Mechanism identities and assignment recipe are proprietary.</text>
</svg>