mp-juuuns commited on
Commit
5f2cadf
·
verified ·
1 Parent(s): 7b8cd30

Add bilingual 4L and MALINT benchmark evidence

Browse files
.gitattributes CHANGED
@@ -1,3 +1,6 @@
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
  benchmark/figures/02_per_label_transfer_metrics.png filter=lfs diff=lfs merge=lfs -text
3
  benchmark/figures/04_compression_ladder.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
1
  *.safetensors filter=lfs diff=lfs merge=lfs -text
2
  benchmark/figures/02_per_label_transfer_metrics.png filter=lfs diff=lfs merge=lfs -text
3
  benchmark/figures/04_compression_ladder.png filter=lfs diff=lfs merge=lfs -text
4
+ benchmark/figures/cross_task/05_why_4l_edge_tradeoff.png filter=lfs diff=lfs merge=lfs -text
5
+ benchmark/figures/cross_task/06_malint_4l_quality.png filter=lfs diff=lfs merge=lfs -text
6
+ benchmark/figures/cross_task/07_malint_quality_resource.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -10,9 +10,11 @@ tags:
10
  - qwen3.5
11
  - classification-backbone
12
  - text-classification
 
13
  - knowledge-distillation
14
  - model-compression
15
  - edge-ai
 
16
  ---
17
 
18
  # Qwen3.5 Standalone 4L Classification Base
@@ -31,6 +33,26 @@ The repository follows one integrated 1-1-1 layout:
31
  classifier preserved as a separate task-specific lineage.
32
  3. **`distillation/`:** reusable 24L→8L→6L→4L platform for user-owned data.
33
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  ## Model at a glance
35
 
36
  | Item | Root model |
@@ -44,7 +66,7 @@ The repository follows one integrated 1-1-1 layout:
44
  | Root BF16 weights | 668,198,976 bytes (637.24 MiB) |
45
  | Output | hidden states; no LM head, task head, labels, or thresholds |
46
  | Task-free KD data | 4,096 unlabeled WikiText-103 raw rows |
47
- | Downstream evidence | one English 14-label SemEval-derived transfer probe |
48
 
49
  ## Load the headless base
50
 
@@ -107,6 +129,126 @@ The chart reports the actual unsharded safetensors parameter counts and BF16
107
  file sizes. It does not imply that downstream quality scales monotonically with
108
  layer count.
109
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
110
  ## Downstream benchmark contract
111
 
112
  Because the root is headless, it cannot be scored as a classifier without
@@ -245,6 +387,10 @@ hardware.
245
  including article-level probability vectors
246
  - [`benchmark/figures/`](benchmark/figures/): Matplotlib PNG/PDF figures,
247
  per-figure source CSVs, and a hash manifest
 
 
 
 
248
 
249
  ## Multilingual scope
250
 
@@ -255,12 +401,13 @@ multilingual upstream benchmarks such as MMMLU, MMLU-ProX, NOVA-63, INCLUDE,
255
  Global PIQA, PolyMATH, WMT24++, and MAXIFE. This is why this repository carries
256
  the Hugging Face `multilingual` language and model tag.
257
 
258
- However, this four-layer root was distilled on English WikiText and its only
259
- downstream transfer probe is English SemEval-derived data. **Tokenizer coverage
260
- and inherited architecture support do not establish retained classification
261
- quality in all 201 languages.** No multilingual downstream score is reported
262
- for this root release. Users should fine-tune and evaluate on each intended
263
- language and domain before making performance claims.
 
264
 
265
  ## Separate SemEval model
266
 
@@ -281,12 +428,19 @@ tokenizer. Its complete reports remain under
281
  - A classification head and labeled training are required before root-model
282
  predictions have task meaning.
283
  - Task-free KD used only 4,096 rows from one English WikiText shard.
284
- - Only one downstream task and one hardware/software setting were measured.
 
 
 
285
  - The 55-article test split was previously opened; results are exploratory.
286
  - Per-label support ranges from 7 to 45 articles and rare-label estimates are
287
  unstable.
288
  - Three seeds describe run-to-run variation; they are not a confidence
289
  interval.
 
 
 
 
290
  - Multilingual input support has not been validated as multilingual downstream
291
  classification quality for this four-layer root.
292
  - Default layer maps are documented structural choices, not universal optima.
@@ -302,6 +456,12 @@ and data terms. Qwen3.5 is attributed to Qwen. Transformers, PyTorch,
302
  Hugging Face Hub, WikiText, and SemEval remain the work of their respective
303
  authors. No WikiText or SemEval source records are redistributed.
304
 
 
 
 
 
 
 
305
  The WikiText page currently has a license wording discrepancy: metadata lists
306
  CC BY-SA 3.0 and GFDL, while prose says CC BY-SA 4.0. Users should inspect
307
  [`Salesforce/wikitext`](https://huggingface.co/datasets/Salesforce/wikitext)
 
10
  - qwen3.5
11
  - classification-backbone
12
  - text-classification
13
+ - multilabel-classification
14
  - knowledge-distillation
15
  - model-compression
16
  - edge-ai
17
+ - malint
18
  ---
19
 
20
  # Qwen3.5 Standalone 4L Classification Base
 
33
  classifier preserved as a separate task-specific lineage.
34
  3. **`distillation/`:** reusable 24L→8L→6L→4L platform for user-owned data.
35
 
36
+ ## 한국어 요약
37
+
38
+ 이 저장소의 루트 모델은 Qwen3.5-0.8B의 text backbone을 **4개 layer로 축소한
39
+ 분류 전용 기반 모델**이다. LM head와 task label, 분류 head가 없으므로 채팅 모델이나
40
+ 즉시 사용할 수 있는 범용 분류기가 아니다. 사용자가 자신의 label과 데이터로 새 head를
41
+ 붙여 학습하는 출발점이다.
42
+
43
+ 4L을 선택한 근거는 한 숫자가 아니라 세 범주의 측정이다. 첫째, full-vocabulary
44
+ task-free backbone은 24L의 752.39M parameter/1,435.11 MiB에서 4L의
45
+ 334.10M/637.24 MiB로 줄었다. 둘째, 동일 V128k/F16 UNO Q 깊이 비교에서 역사적
46
+ 4L proxy는 8L 대비 artifact `-27.74%`, PP128 latency `-49.84%`, peak RSS
47
+ `-21.81%`, core-seconds `-48.96%`를 기록했다. 셋째, 별도 task 적응 결과에서
48
+ 4L endpoint가 propaganda와 MALINT 모두에서 실제 분류 가능성을 보였다.
49
+
50
+ 다만 이 근거는 “4L이 항상 8L/24L보다 정확하다”는 뜻이 아니다. UNO Q의 4L은
51
+ historical early-path proxy이고, propaganda 24L/8L/4L endpoint는 계보와 vocabulary가
52
+ 달라 순수 depth ablation이 아니다. MALINT의 Qwen student 세 모델은 모두 4L이므로,
53
+ MALINT는 **4L 내부 pooling/KD interface 개선**을 검증할 뿐 4L-vs-8L/24L 우월성을
54
+ 검증하지 않는다. 아래 표와 그림은 이 경계를 수치와 함께 명시한다.
55
+
56
  ## Model at a glance
57
 
58
  | Item | Root model |
 
66
  | Root BF16 weights | 668,198,976 bytes (637.24 MiB) |
67
  | Output | hidden states; no LM head, task head, labels, or thresholds |
68
  | Task-free KD data | 4,096 unlabeled WikiText-103 raw rows |
69
+ | Downstream evidence | English 14-label propaganda transfer and 5-label MALINT adaptation studies under separate protocols |
70
 
71
  ## Load the headless base
72
 
 
129
  file sizes. It does not imply that downstream quality scales monotonically with
130
  layer count.
131
 
132
+ ## Why four layers? / 왜 4L인가?
133
+
134
+ The choice is an **edge-oriented operating point**, not a universal optimum.
135
+ The task-free full-vocabulary root has 334,096,704 parameters and a 637.24 MiB
136
+ BF16 weight file, versus 752,393,024 parameters and 1,435.11 MiB for the 24L
137
+ text teacher: a `55.59%` reduction in both parameter count and raw BF16 bytes.
138
+
139
+ A separate controlled Arduino UNO Q chain held vocabulary (`128k`), precision
140
+ (`F16`), runtime, and benchmark protocol fixed while moving 8L→6L→4L. Values
141
+ below normalize the 8L arm to 100; lower is better.
142
+
143
+ | Controlled UNO Q arm | Artifact | PP128 latency | Peak RSS | Core-seconds |
144
+ |---|---:|---:|---:|---:|
145
+ | 8L | 100.00 | 100.00 | 100.00 | 100.00 |
146
+ | 6L | 85.59 | 76.37 | 87.51 | 76.17 |
147
+ | Historical 4L proxy | **72.26** | **50.16** | **78.19** | **51.04** |
148
+
149
+ Thus the matched 8L→4L route reduced artifact size by `27.74%`, PP128 latency
150
+ by `49.84%`, peak RSS by `21.81%`, and core-seconds by `48.96%`. CPU occupancy
151
+ remained near 3.6–3.7 saturated cores; the benefit was finishing sooner, not
152
+ lower active occupancy. The 4L point is a historical early-path proxy, not the
153
+ final common-head quality model.
154
+
155
+ The earlier English 14-label propaganda benchmark supplies task evidence under
156
+ one common five-epoch protocol. Its endpoints are operationally relevant but
157
+ not a pure depth study because lineage and vocabulary differ.
158
+
159
+ | Propaganda endpoint | Macro-F1 | Micro-F1 | Subset exact match | Test s/article | Peak CUDA allocated | Parameters |
160
+ |---|---:|---:|---:|---:|---:|---:|
161
+ | Qwen3.5 24L | 0.56426 ± 0.00626 | 0.62648 ± 0.01399 | 0.03030 ± 0.02777 | 0.58636 ± 0.00598 | 7,635,339,264 B | 752.41M |
162
+ | Qwen3.5 C8Q 8L | 0.53669 ± 0.01726 | 0.62388 ± 0.01362 | 0.00606 ± 0.01050 | 0.20666 ± 0.00207 | 4,257,894,400 B | 420.33M |
163
+ | Qwen3.5 final 4L | **0.58757 ± 0.00717** | **0.64798 ± 0.00187** | 0.00606 ± 0.01050 | **0.08544 ± 0.00060** | **2,143,655,936 B** | **210.90M** |
164
+
165
+ These results show that a compact four-layer endpoint can be useful; they do
166
+ not prove that reducing depth caused the quality difference. The headless root
167
+ also differs from the specialized 4L endpoint: the root keeps the full 248,320
168
+ token vocabulary and has 334.10M parameters.
169
+
170
+ ![Why four layers: controlled edge and propaganda evidence](benchmark/figures/cross_task/05_why_4l_edge_tradeoff.png)
171
+
172
+ 한국어로 요약하면, 4L 선택은 동일 조건 edge chain에서 확인된 latency·memory·artifact
173
+ 감소와 별도 task에서 확인된 분류 가능성을 함께 본 결과다. 그러나 propaganda endpoint의
174
+ 계보/vocabulary 차이 때문에 4L의 품질 우위를 depth 하나의 인과 효과로 해석하면 안 된다.
175
+
176
+ ## MALINT adaptation evidence / MALINT 적응 근거
177
+
178
+ MALINT is an English five-label malicious-intent multilabel task covering
179
+ changing political views (CPV), promoting social stereotypes/antagonisms
180
+ (PSSA), undermining international organizations/alliances (UIOA), promoting
181
+ anti-scientific views (PASV), and undermining public institutions (UCPI). No
182
+ MALINT row is redistributed here.
183
+
184
+ The strongest changed-model evidence is a frozen **3 repeats × 3 folds** matched
185
+ study over the combined 1,100-row development corpus. Each repeat reconstructs
186
+ exact out-of-fold predictions for all 1,100 rows at threshold `0.30`. There was
187
+ no fold checkpoint selection, threshold tuning, or official held-out access.
188
+
189
+ | Arm | Weighted-F1 | Macro-F1 | Micro-F1 | Subset exact match | Label-wise accuracy |
190
+ |---|---:|---:|---:|---:|---:|
191
+ | Rightmost token + hard labels (4L) | 0.52218 ± 0.01497 | 0.51096 ± 0.01628 | 0.53203 ± 0.01076 | 0.64030 ± 0.00367 | 0.87467 ± 0.00121 |
192
+ | Mean pooling + hard labels (4L) | 0.61180 ± 0.00886 | 0.60551 ± 0.00753 | 0.61769 ± 0.00746 | **0.66970 ± 0.00517** | **0.88764 ± 0.00175** |
193
+ | **Mean pooling + KD (4L)** | **0.63465 ± 0.01197** | **0.62818 ± 0.01423** | **0.63896 ± 0.00985** | 0.65576 ± 0.00319 | 0.87576 ± 0.00204 |
194
+ | Fresh RoBERTa teacher | 0.60468 ± 0.01580 | 0.59557 ± 0.01637 | 0.60462 ± 0.01777 | 0.62000 ± 0.00596 | 0.85079 ± 0.01087 |
195
+
196
+ Values are mean ± sample SD across three repeated OOF reconstructions. Subset
197
+ exact match requires all five labels for a row to match. Label-wise accuracy is
198
+ the mean over all row-label decisions and is imbalance-sensitive.
199
+
200
+ Mean+KD improved weighted F1 over rightmost-hard by `+0.11247`, with positive
201
+ deltas in `9/9` folds and corrected interval `[+0.07160, +0.15925]`. Mean
202
+ pooling alone improved it by `+0.08962` in `9/9` folds. The incremental KD
203
+ contrast was `+0.02285` in `8/9` folds, but its interval
204
+ `[-0.00155, +0.05059]` crosses zero. Therefore the strongest isolated evidence
205
+ is for masked mean pooling; independent KD and teacher superiority are not
206
+ established.
207
+
208
+ ![MALINT 4L repeated quality metrics](benchmark/figures/cross_task/06_malint_4l_quality.png)
209
+
210
+ All three Qwen student arms above are four-layer models. This is evidence that
211
+ a 4L backbone can support a materially better document interface under this
212
+ internal protocol. It is **not** a MALINT 4L-vs-8L/24L depth ablation. It is
213
+ also post-selection development evidence, not untouched confirmation or SOTA.
214
+ The older validation-selected rightmost 4L has an official fixed-threshold test
215
+ weighted F1 of `0.53749` with row-bootstrap interval `[0.49102, 0.58403]`; the
216
+ changed Mean+KD artifact did not reopen that test, so the two protocols must not
217
+ be compared as if they were one held-out leaderboard.
218
+
219
+ ### Quality–resource trade-off / 품질–자원 절충
220
+
221
+ A separate label-free same-input resource run used an RTX 5070 Ti, BF16,
222
+ sequence length 256, fresh processes, three counterbalanced rounds per batch,
223
+ and CUDA-resident token IDs/masks. Timing is model forward only.
224
+
225
+ | Batch | Rightmost ms/example | Mean+KD ms/example | Latency change | Rightmost peak allocated | Mean+KD peak allocated | Peak change |
226
+ |---:|---:|---:|---:|---:|---:|---:|
227
+ | 1 | 18.47463 | 19.31878 | **+4.569%** | 1,373,150,208 B | 1,374,199,296 B | **+0.076%** |
228
+ | 4 | 4.58975 | 4.67101 | **+1.770%** | 1,454,388,736 B | 1,455,437,824 B | **+0.072%** |
229
+
230
+ Mean+KD adds exactly five parameters. It is **not faster**; the supported claim
231
+ is higher internal F1 at a small measured forward-latency and peak-allocation
232
+ cost. Ambient `nvidia-smi` utilization is not reported as model occupancy.
233
+
234
+ ![MALINT 4L quality and resource trade-off](benchmark/figures/cross_task/07_malint_quality_resource.png)
235
+
236
+ 한국어 해석: MALINT에서 확인된 핵심은 “4L보다 더 깊은 모델을 이겼다”가 아니라,
237
+ **깊이를 4L로 고정한 상태에서 문서 전체 mean pooling interface가 rightmost-token
238
+ interface보다 반복 내부평가 F1을 높였다**는 것이다. Mean+KD는 batch 1/4에서 각각
239
+ `4.569%`/`1.770%` 느렸고 peak allocation도 소폭 증가했다. 따라서 빠른 모델이
240
+ 아니라, 거의 같은 크기·memory에서 제한된 latency 비용으로 내부 품질을 높인 적응
241
+ 결과로 설명해야 한다.
242
+
243
+ ### Cross-task boundary / task 간 비교 경계
244
+
245
+ Propaganda and MALINT have different labels, split histories, thresholds,
246
+ metrics, and evaluation protocols. Their F1 values are not pooled or ranked
247
+ against each other. Together they support only a bounded statement: this 4L
248
+ platform has been adapted successfully to two English multilabel settings,
249
+ while generalization, multilingual quality, pure depth superiority, and SOTA
250
+ remain unproven.
251
+
252
  ## Downstream benchmark contract
253
 
254
  Because the root is headless, it cannot be scored as a classifier without
 
387
  including article-level probability vectors
388
  - [`benchmark/figures/`](benchmark/figures/): Matplotlib PNG/PDF figures,
389
  per-figure source CSVs, and a hash manifest
390
+ - [`benchmark/figures/cross_task/manifest.json`](benchmark/figures/cross_task/manifest.json):
391
+ immutable input/output hashes for figures 05–07
392
+ - [`benchmark/figures/cross_task/source_summary.json`](benchmark/figures/cross_task/source_summary.json):
393
+ aggregate-only source values and claim boundaries for figures 05–07
394
 
395
  ## Multilingual scope
396
 
 
401
  Global PIQA, PolyMATH, WMT24++, and MAXIFE. This is why this repository carries
402
  the Hugging Face `multilingual` language and model tag.
403
 
404
+ However, this four-layer root was distilled on English WikiText and its current
405
+ downstream evidence is limited to two English multilabel settings:
406
+ SemEval-derived propaganda and MALINT. **Tokenizer coverage and inherited
407
+ architecture support do not establish retained classification quality in all
408
+ 201 languages.** No multilingual downstream score is reported for this root
409
+ release. Users should fine-tune and evaluate on each intended language and
410
+ domain before making performance claims.
411
 
412
  ## Separate SemEval model
413
 
 
428
  - A classification head and labeled training are required before root-model
429
  predictions have task meaning.
430
  - Task-free KD used only 4,096 rows from one English WikiText shard.
431
+ - Only two English multilabel downstream settings and a limited set of
432
+ hardware/software conditions were measured.
433
+ - Cross-task evidence now covers two English multilabel settings, but they use
434
+ different protocols and cannot be pooled into one score.
435
  - The 55-article test split was previously opened; results are exploratory.
436
  - Per-label support ranges from 7 to 45 articles and rare-label estimates are
437
  unstable.
438
  - Three seeds describe run-to-run variation; they are not a confidence
439
  interval.
440
+ - The changed MALINT model has post-selection repeated-development evidence but
441
+ no newly opened official held-out score; MALINT is not a depth ablation.
442
+ - MALINT forward latency excludes tokenization, loading, disk, host-to-device
443
+ transfer, CPU, Mac, Arduino, and end-to-end application costs.
444
  - Multilingual input support has not been validated as multilingual downstream
445
  classification quality for this four-layer root.
446
  - Default layer maps are documented structural choices, not universal optima.
 
456
  Hugging Face Hub, WikiText, and SemEval remain the work of their respective
457
  authors. No WikiText or SemEval source records are redistributed.
458
 
459
+ No MALINT row is redistributed. The MALINT task/data attribution follows
460
+ Modzelewski et al., *MALicious INTent Dataset and Inoculating LLMs for Enhanced
461
+ Disinformation Detection*, EACL 2026
462
+ ([ACL Anthology](https://aclanthology.org/2026.eacl-long.144/)); the official
463
+ dataset repository states CC BY 4.0 for the dataset.
464
+
465
  The WikiText page currently has a license wording discrepancy: metadata lists
466
  CC BY-SA 3.0 and GFDL, while prose says CC BY-SA 4.0. Users should inspect
467
  [`Salesforce/wikitext`](https://huggingface.co/datasets/Salesforce/wikitext)
benchmark/figures/README.md CHANGED
@@ -1,6 +1,8 @@
1
  # Matplotlib benchmark figures
2
 
3
- All figures are deterministically rendered from the frozen v2 benchmark bundle.
 
 
4
  PNG files are intended for the Hugging Face model card; PDF files are vector
5
  exports for reports. Each plot has a compact source CSV beside it.
6
 
@@ -8,6 +10,25 @@ exports for reports. Each plot has a compact source CSV beside it.
8
  2. Per-label precision/recall/F1 for the task-agnostic base.
9
  3. Macro-F1 against article inference time and peak CUDA allocation.
10
  4. Parameters and BF16 size across the 24L→8L→6L→4L task-free ladder.
 
 
 
 
 
 
11
 
12
  The 55-article test split was previously opened. Error bars are not confidence
13
  intervals. The specialized checkpoint is a separate reference lineage.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # Matplotlib benchmark figures
2
 
3
+ Figures 01–04 are deterministically rendered from the frozen v2 task-agnostic
4
+ transfer bundle. Figures 05–07 are recomputed from frozen propaganda, UNO Q,
5
+ and MALINT aggregate records.
6
  PNG files are intended for the Hugging Face model card; PDF files are vector
7
  exports for reports. Each plot has a compact source CSV beside it.
8
 
 
10
  2. Per-label precision/recall/F1 for the task-agnostic base.
11
  3. Macro-F1 against article inference time and peak CUDA allocation.
12
  4. Parameters and BF16 size across the 24L→8L→6L→4L task-free ladder.
13
+ 5. Controlled UNO Q depth resources plus descriptive propaganda 24L/8L/4L
14
+ quality and resources.
15
+ 6. MALINT repeated-OOF weighted/macro/micro F1, subset exact match, and
16
+ label-wise accuracy.
17
+ 7. MALINT internal weighted-F1 gain beside measured CUDA-forward latency and
18
+ peak-allocation changes.
19
 
20
  The 55-article test split was previously opened. Error bars are not confidence
21
  intervals. The specialized checkpoint is a separate reference lineage.
22
+
23
+ `manifest.json` and `source_summary.json` cover figures 01–04.
24
+ `cross_task/manifest.json` and `cross_task/source_summary.json` cover figures
25
+ 05–07 and bind every frozen input by SHA-256. The latter bundle is aggregate
26
+ only and contains no raw text, IDs, row labels, logits, probabilities, or
27
+ predictions.
28
+
29
+ ## 한국어
30
+
31
+ 01–04는 기존 task-free base/propaganda transfer 결과이고, 05–07은 4L 선택 근거와
32
+ MALINT 내부 반복평가를 추가한다. MALINT의 Qwen student는 모두 4L이므로 06은
33
+ 4L-vs-8L/24L depth 비교가 아니다. 07은 Mean+KD가 더 빠르지 않으며 제한된 latency와
34
+ peak CUDA allocation 비용이 있음을 함께 표시한다.
benchmark/figures/cross_task/05_why_4l_edge_tradeoff.csv ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ scope,model,layers,metric,value,sample_sd,unit,normalized_reference,normalized_value,n,claim_boundary
2
+ uno_q_controlled_v128k_f16_depth_chain,8L,8,artifact_mib,100.0,,percent_of_8l,8L=100,100.0,3,4L is the historical early-path proxy
3
+ uno_q_controlled_v128k_f16_depth_chain,6L,6,artifact_mib,85.59176355424236,,percent_of_8l,8L=100,85.59176355424236,3,4L is the historical early-path proxy
4
+ uno_q_controlled_v128k_f16_depth_chain,4L proxy,4,artifact_mib,72.2607217304372,,percent_of_8l,8L=100,72.2607217304372,3,4L is the historical early-path proxy
5
+ uno_q_controlled_v128k_f16_depth_chain,8L,8,latency_pp128_ms,100.0,,percent_of_8l,8L=100,100.0,3,4L is the historical early-path proxy
6
+ uno_q_controlled_v128k_f16_depth_chain,6L,6,latency_pp128_ms,76.36565126547553,,percent_of_8l,8L=100,76.36565126547553,3,4L is the historical early-path proxy
7
+ uno_q_controlled_v128k_f16_depth_chain,4L proxy,4,latency_pp128_ms,50.16216836129361,,percent_of_8l,8L=100,50.16216836129361,3,4L is the historical early-path proxy
8
+ uno_q_controlled_v128k_f16_depth_chain,8L,8,peak_rss_mib,100.0,,percent_of_8l,8L=100,100.0,3,4L is the historical early-path proxy
9
+ uno_q_controlled_v128k_f16_depth_chain,6L,6,peak_rss_mib,87.50983221571457,,percent_of_8l,8L=100,87.50983221571457,3,4L is the historical early-path proxy
10
+ uno_q_controlled_v128k_f16_depth_chain,4L proxy,4,peak_rss_mib,78.19200407435702,,percent_of_8l,8L=100,78.19200407435702,3,4L is the historical early-path proxy
11
+ uno_q_controlled_v128k_f16_depth_chain,8L,8,mean_core_seconds_per_round,100.0,,percent_of_8l,8L=100,100.0,3,4L is the historical early-path proxy
12
+ uno_q_controlled_v128k_f16_depth_chain,6L,6,mean_core_seconds_per_round,76.16702179915283,,percent_of_8l,8L=100,76.16702179915283,3,4L is the historical early-path proxy
13
+ uno_q_controlled_v128k_f16_depth_chain,4L proxy,4,mean_core_seconds_per_round,51.03561371028788,,percent_of_8l,8L=100,51.03561371028788,3,4L is the historical early-path proxy
14
+ propaganda_operational_endpoint_quality,24L,24,macro_f1,0.5642638295960175,0.006258972649035587,score,,,3,same task protocol; lineage/vocabulary differ
15
+ propaganda_operational_endpoint_quality,24L,24,micro_f1,0.6264844890231888,0.01398932248989461,score,,,3,same task protocol; lineage/vocabulary differ
16
+ propaganda_operational_endpoint_quality,24L,24,subset_exact_match,0.0303030303030303,0.027773186030035393,score,,,3,same task protocol; lineage/vocabulary differ
17
+ propaganda_operational_endpoint_resource,24L,24,test_seconds_per_article,0.586362602903109,0.005979023788731576,seconds_per_article,24L=100,100.0,3,same task protocol; lineage/vocabulary differ
18
+ propaganda_operational_endpoint_resource,24L,24,peak_cuda_allocated_bytes,7635339264.0,0.0,bytes,24L=100,100.0,3,same task protocol; lineage/vocabulary differ
19
+ propaganda_operational_endpoint_resource,24L,24,parameters,752407360,,count,24L=100,100.0,3,same task protocol; lineage/vocabulary differ
20
+ propaganda_operational_endpoint_resource,24L,24,total_wall_seconds,12842.063612925334,94.80225434811064,seconds,24L=100,99.99999999999999,3,same task protocol; lineage/vocabulary differ
21
+ propaganda_operational_endpoint_quality,8L,8,macro_f1,0.5366873350531403,0.017264532146217884,score,,,3,same task protocol; lineage/vocabulary differ
22
+ propaganda_operational_endpoint_quality,8L,8,micro_f1,0.6238830315003392,0.013623163834245318,score,,,3,same task protocol; lineage/vocabulary differ
23
+ propaganda_operational_endpoint_quality,8L,8,subset_exact_match,0.006060606060606061,0.01049727762162956,score,,,3,same task protocol; lineage/vocabulary differ
24
+ propaganda_operational_endpoint_resource,8L,8,test_seconds_per_article,0.206657676709195,0.002071265185964238,seconds_per_article,24L=100,35.24400698237286,3,same task protocol; lineage/vocabulary differ
25
+ propaganda_operational_endpoint_resource,8L,8,peak_cuda_allocated_bytes,4257894400.0,3632373.815194686,bytes,24L=100,55.765621575921635,3,same task protocol; lineage/vocabulary differ
26
+ propaganda_operational_endpoint_resource,8L,8,parameters,420332480,,count,24L=100,55.865014398583234,3,same task protocol; lineage/vocabulary differ
27
+ propaganda_operational_endpoint_resource,8L,8,total_wall_seconds,4291.582853937677,44.49609583978631,seconds,24L=100,33.418171590571056,3,same task protocol; lineage/vocabulary differ
28
+ propaganda_operational_endpoint_quality,4L,4,macro_f1,0.5875679559555907,0.007165753447433348,score,,,3,same task protocol; lineage/vocabulary differ
29
+ propaganda_operational_endpoint_quality,4L,4,micro_f1,0.647978512363484,0.0018739735696546695,score,,,3,same task protocol; lineage/vocabulary differ
30
+ propaganda_operational_endpoint_quality,4L,4,subset_exact_match,0.006060606060606061,0.01049727762162956,score,,,3,same task protocol; lineage/vocabulary differ
31
+ propaganda_operational_endpoint_resource,4L,4,test_seconds_per_article,0.08543555729062946,0.0005980667826383295,seconds_per_article,24L=100,14.570430799582711,3,same task protocol; lineage/vocabulary differ
32
+ propaganda_operational_endpoint_resource,4L,4,peak_cuda_allocated_bytes,2143655936.0,0.0,bytes,24L=100,28.075451029493376,3,same task protocol; lineage/vocabulary differ
33
+ propaganda_operational_endpoint_resource,4L,4,parameters,210903360,,count,24L=100,28.030475406301182,3,same task protocol; lineage/vocabulary differ
34
+ propaganda_operational_endpoint_resource,4L,4,total_wall_seconds,1643.6388439506602,9.226529522850198,seconds,24L=100,12.798868573555138,3,same task protocol; lineage/vocabulary differ
benchmark/figures/cross_task/05_why_4l_edge_tradeoff.pdf ADDED
Binary file (27.2 kB). View file
 
benchmark/figures/cross_task/05_why_4l_edge_tradeoff.png ADDED

Git LFS Details

  • SHA256: cdc0a8a98cd13ad35a68b980d2e0750e2424b8932f7d27dc0ecd548fed823c74
  • Pointer size: 131 Bytes
  • Size of remote file: 179 kB
benchmark/figures/cross_task/06_malint_4l_quality.csv ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ arm,arm_label,role,layers,metric,mean,sample_sd,n_repeats,threshold
2
+ rightmost_hard,Rightmost + hard,qwen_4l_student,4,weighted_f1,0.522182806412052,0.014974460565445307,3,0.3
3
+ rightmost_hard,Rightmost + hard,qwen_4l_student,4,macro_f1,0.5109624469249431,0.016275524962473233,3,0.3
4
+ rightmost_hard,Rightmost + hard,qwen_4l_student,4,micro_f1,0.5320256865023298,0.010757300158510568,3,0.3
5
+ rightmost_hard,Rightmost + hard,qwen_4l_student,4,subset_exact_match,0.6403030303030303,0.003674047167570365,3,0.3
6
+ rightmost_hard,Rightmost + hard,qwen_4l_student,4,label_accuracy,0.8746666666666667,0.0012106051124507965,3,0.3
7
+ mean_hard,Mean pool + hard,qwen_4l_student,4,weighted_f1,0.6118049035859879,0.008860176348616246,3,0.3
8
+ mean_hard,Mean pool + hard,qwen_4l_student,4,macro_f1,0.6055064323204021,0.007533985719586246,3,0.3
9
+ mean_hard,Mean pool + hard,qwen_4l_student,4,micro_f1,0.6176900341883843,0.007461802499370809,3,0.3
10
+ mean_hard,Mean pool + hard,qwen_4l_student,4,subset_exact_match,0.6696969696969697,0.005169309730070291,3,0.3
11
+ mean_hard,Mean pool + hard,qwen_4l_student,4,label_accuracy,0.8876363636363637,0.0017533910474532954,3,0.3
12
+ mean_kd,Mean pool + KD,qwen_4l_student,4,weighted_f1,0.6346533875063399,0.011966996735426847,3,0.3
13
+ mean_kd,Mean pool + KD,qwen_4l_student,4,macro_f1,0.6281768437686408,0.014225316120600004,3,0.3
14
+ mean_kd,Mean pool + KD,qwen_4l_student,4,micro_f1,0.6389632888388834,0.009850482866928375,3,0.3
15
+ mean_kd,Mean pool + KD,qwen_4l_student,4,subset_exact_match,0.6557575757575758,0.003192622349349309,3,0.3
16
+ mean_kd,Mean pool + KD,qwen_4l_student,4,label_accuracy,0.8757575757575757,0.0020436018545560413,3,0.3
17
+ teacher,RoBERTa teacher,external_teacher,,weighted_f1,0.6046800630077942,0.01579900095351724,3,0.3
18
+ teacher,RoBERTa teacher,external_teacher,,macro_f1,0.5955721586042237,0.01637303032583919,3,0.3
19
+ teacher,RoBERTa teacher,external_teacher,,micro_f1,0.6046188233688233,0.01776761734134435,3,0.3
20
+ teacher,RoBERTa teacher,external_teacher,,subset_exact_match,0.62,0.005961307749365459,3,0.3
21
+ teacher,RoBERTa teacher,external_teacher,,label_accuracy,0.8507878787878788,0.01087469324477906,3,0.3
benchmark/figures/cross_task/06_malint_4l_quality.pdf ADDED
Binary file (26.4 kB). View file
 
benchmark/figures/cross_task/06_malint_4l_quality.png ADDED

Git LFS Details

  • SHA256: 2c5b22415d311ca5fa2b198d55b77f3570b57595d01f0b5031abde59ffc3e9e6
  • Pointer size: 131 Bytes
  • Size of remote file: 181 kB
benchmark/figures/cross_task/07_malint_quality_resource.csv ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ panel,batch,arm,metric,value,sample_sd,unit,ratio_vs_rightmost,percent_change,scope
2
+ quality,,rightmost_hard,weighted_f1,0.522182806412052,0.014974460565445307,score,1.0,0.0,post_selection_internal_repeated_cv
3
+ quality,,mean_kd,weighted_f1,0.6346533875063399,0.011966996735426847,score,1.2153854545060183,21.53854545060183,post_selection_internal_repeated_cv
4
+ resource,1,existing_rightmost,latency_ms_per_example,18.474629627574572,,ms_per_example,1.0,0.0,same_input_cuda_forward_only
5
+ resource,1,existing_rightmost,peak_allocated_bytes,1373150208,,bytes,1.0,0.0,same_input_cuda_forward_only
6
+ resource,1,existing_rightmost,parameter_count,334101824,,count,1.0,0.0,same_input_cuda_forward_only
7
+ resource,1,mean_kd,latency_ms_per_example,19.318781786253957,,ms_per_example,1.0456925078172845,4.569250781728451,same_input_cuda_forward_only
8
+ resource,1,mean_kd,peak_allocated_bytes,1374199296,,bytes,1.0007640009038254,0.07640009038254458,same_input_cuda_forward_only
9
+ resource,1,mean_kd,parameter_count,334101829,,count,1.0000000149654975,1.496549750790166e-06,same_input_cuda_forward_only
10
+ resource,4,existing_rightmost,latency_ms_per_example,4.589751928502863,,ms_per_example,1.0,0.0,same_input_cuda_forward_only
11
+ resource,4,existing_rightmost,peak_allocated_bytes,1454388736,,bytes,1.0,0.0,same_input_cuda_forward_only
12
+ resource,4,existing_rightmost,parameter_count,334101824,,count,1.0,0.0,same_input_cuda_forward_only
13
+ resource,4,mean_kd,latency_ms_per_example,4.671012829289292,,ms_per_example,1.0177048568315403,1.7704856831540328,same_input_cuda_forward_only
14
+ resource,4,mean_kd,peak_allocated_bytes,1455437824,,bytes,1.0007213257185181,0.07213257185181288,same_input_cuda_forward_only
15
+ resource,4,mean_kd,parameter_count,334101829,,count,1.0000000149654975,1.496549750790166e-06,same_input_cuda_forward_only
benchmark/figures/cross_task/07_malint_quality_resource.pdf ADDED
Binary file (29.3 kB). View file
 
benchmark/figures/cross_task/07_malint_quality_resource.png ADDED

Git LFS Details

  • SHA256: 1cb13cc695f3d59ae9ce75599b86867da624d6e0aae0897e0db8686f6db984ba
  • Pointer size: 131 Bytes
  • Size of remote file: 164 kB
benchmark/figures/cross_task/README.md ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Cross-task Matplotlib figures
2
+
3
+ This directory contains aggregate-only, deterministic plots for the existing
4
+ Qwen3.5 standalone 4L classification-base model card. PNG files are for the
5
+ Hugging Face README; PDF files are vector exports; each figure has a source CSV.
6
+
7
+ 1. `05_why_4l_edge_tradeoff`: controlled UNO Q depth-chain resources plus
8
+ descriptive propaganda endpoint quality/resources.
9
+ 2. `06_malint_4l_quality`: repeated-development F1 and accuracy-type metrics.
10
+ 3. `07_malint_quality_resource`: the primary internal quality contrast and its
11
+ measured CUDA-forward latency/peak-allocation cost.
12
+
13
+ Scientific boundaries are printed inside the figures. In particular, MALINT
14
+ does not compare 4L against 8L or 24L; all Qwen student arms keep four layers.
15
+ No raw text, IDs, row labels, logits, probabilities, or predictions are copied
16
+ into this public bundle.
benchmark/figures/cross_task/manifest.json ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "figures": [
3
+ "05_why_4l_edge_tradeoff",
4
+ "06_malint_4l_quality",
5
+ "07_malint_quality_resource"
6
+ ],
7
+ "files_before_manifest": [
8
+ {
9
+ "bytes": 5442,
10
+ "path": "05_why_4l_edge_tradeoff.csv",
11
+ "sha256": "afd203646513c49140a25ef2a562a064286a4fa5efb495e6a73f83f0106e7933"
12
+ },
13
+ {
14
+ "bytes": 27224,
15
+ "path": "05_why_4l_edge_tradeoff.pdf",
16
+ "sha256": "22f531ea586f9c79ac8f114345cec56e74ea5ed97aa3726f7310018666ae6312"
17
+ },
18
+ {
19
+ "bytes": 178965,
20
+ "path": "05_why_4l_edge_tradeoff.png",
21
+ "sha256": "cdc0a8a98cd13ad35a68b980d2e0750e2424b8932f7d27dc0ecd548fed823c74"
22
+ },
23
+ {
24
+ "bytes": 2118,
25
+ "path": "06_malint_4l_quality.csv",
26
+ "sha256": "25848237b9c77596a8c4b6224fa111f86c816e065804ccfdd6a4fdf6c63ef8d1"
27
+ },
28
+ {
29
+ "bytes": 26351,
30
+ "path": "06_malint_4l_quality.pdf",
31
+ "sha256": "9a1ae78b4e5871f5d5c5321db6c5d1fe0bf349a7869977b2649b44ccea633d04"
32
+ },
33
+ {
34
+ "bytes": 181392,
35
+ "path": "06_malint_4l_quality.png",
36
+ "sha256": "2c5b22415d311ca5fa2b198d55b77f3570b57595d01f0b5031abde59ffc3e9e6"
37
+ },
38
+ {
39
+ "bytes": 1800,
40
+ "path": "07_malint_quality_resource.csv",
41
+ "sha256": "aac7d3400cf8dcb3ac8b6fc317c27ded5343231c93476997011a3bb56b5e4d6c"
42
+ },
43
+ {
44
+ "bytes": 29309,
45
+ "path": "07_malint_quality_resource.pdf",
46
+ "sha256": "498338991976e64038096a12361f63830ced3b9c6eea3e1bdaed8d0e4efac606"
47
+ },
48
+ {
49
+ "bytes": 163562,
50
+ "path": "07_malint_quality_resource.png",
51
+ "sha256": "1cb13cc695f3d59ae9ce75599b86867da624d6e0aae0897e0db8686f6db984ba"
52
+ },
53
+ {
54
+ "bytes": 868,
55
+ "path": "README.md",
56
+ "sha256": "f3e79965f14ec0fd20d11c361d25c3a3abc4c9a20b5dbce9fac0660bb12789f0"
57
+ },
58
+ {
59
+ "bytes": 29752,
60
+ "path": "source_summary.json",
61
+ "sha256": "20ea292f85757440d8641199003c47674deb0c6aacea0ef8f9f5fcd2480bb5d0"
62
+ }
63
+ ],
64
+ "inputs": [
65
+ {
66
+ "bytes": 58857,
67
+ "path": "results/malint_qwen35_4l_repeated_cv/20260814-r3x3-terminal-v1/aggregate-v1/aggregate_result.json",
68
+ "role": "malint_repeated_aggregate",
69
+ "sha256": "e3d659c2f5aca645f040d03c57e4effa9eb2616f622aceb503c71236b6b01efa"
70
+ },
71
+ {
72
+ "bytes": 325527,
73
+ "path": "results/malint_qwen35_4l_repeated_cv/20260814-r3x3-terminal-v1/aggregate-v1/aggregate_oof_predictions.npz",
74
+ "role": "malint_repeated_oof",
75
+ "sha256": "a1d7827ebe4beecb49676ca42e0c986c5f4101c394d55c9965161481c347a3f1"
76
+ },
77
+ {
78
+ "bytes": 15407,
79
+ "path": "results/malint_qwen35_4l_resource_benchmark/20260814-matched-bf16-v2/resource_benchmark.json",
80
+ "role": "malint_resource_aggregate",
81
+ "sha256": "5337e44c907a05590c0ea3da6a0760baff22c3f930ed81c069c257060fd2e725"
82
+ },
83
+ {
84
+ "bytes": 3144,
85
+ "path": "results/final_broad_benchmark/20260729-v1/report/quality_matrix.csv",
86
+ "role": "propaganda_quality_matrix",
87
+ "sha256": "c6e316f7081e2a6dcda1ec4e95b57cdcc357ae49ed8b3f5a876c5b51e0df722f"
88
+ },
89
+ {
90
+ "bytes": 115661,
91
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-full24l-commonhead/seed41/report.json",
92
+ "role": "propaganda_seed_report_01",
93
+ "sha256": "1c572fb728755dbde0460e6b8eb9a84d078b128f8a42a9148a17c40fc0b27d82"
94
+ },
95
+ {
96
+ "bytes": 116024,
97
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-full24l-commonhead/seed42/report.json",
98
+ "role": "propaganda_seed_report_02",
99
+ "sha256": "055edfee92e39928aa9577f239c5b0389f3dda375a96fc69a2c000a0ed6edc36"
100
+ },
101
+ {
102
+ "bytes": 115639,
103
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-full24l-commonhead/seed43-retry2/report.json",
104
+ "role": "propaganda_seed_report_03",
105
+ "sha256": "84b2c316556db4aa5fdf7c08433a4808a5fe9ef6f620da6472f4f2f73dee2e4b"
106
+ },
107
+ {
108
+ "bytes": 114924,
109
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-c8q8l-commonhead/seed41/report.json",
110
+ "role": "propaganda_seed_report_04",
111
+ "sha256": "9234258589ffb24eaa92859639a66aeb3e55190f8aa82f798d88e791a282bf9c"
112
+ },
113
+ {
114
+ "bytes": 115800,
115
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-c8q8l-commonhead/seed42/report.json",
116
+ "role": "propaganda_seed_report_05",
117
+ "sha256": "8161dd1378081de9960136fddab800994529aa3c754674ca41bb4aa71515b3c6"
118
+ },
119
+ {
120
+ "bytes": 116039,
121
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-c8q8l-commonhead/seed43/report.json",
122
+ "role": "propaganda_seed_report_06",
123
+ "sha256": "ce0c76a2aa97a5b7c8cdf820af0bd4f8a26bf50d460612ca1db30bd497ca3d9d"
124
+ },
125
+ {
126
+ "bytes": 115985,
127
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-standalone4l-commonhead/seed41/report.json",
128
+ "role": "propaganda_seed_report_07",
129
+ "sha256": "ec5314140fff0bdd46e87ee00ad67bee7e61d82c7cfa34add1510dd7d5bdb3c0"
130
+ },
131
+ {
132
+ "bytes": 115972,
133
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-standalone4l-commonhead/seed42/report.json",
134
+ "role": "propaganda_seed_report_08",
135
+ "sha256": "e5c45a7dc41bebf4550da7c5763d0759c6d0922da9efda4048564c11c6d2f3af"
136
+ },
137
+ {
138
+ "bytes": 116177,
139
+ "path": "results/final_broad_benchmark/20260729-v1/trainable/qwen35-standalone4l-commonhead/seed43/report.json",
140
+ "role": "propaganda_seed_report_09",
141
+ "sha256": "e05bc2146d720706780fc45a1fb05911d5c2926caa0bcb12ffead946028eef0d"
142
+ },
143
+ {
144
+ "bytes": 827,
145
+ "path": "results/standalone4l_base/20260731-v1/huggingface_release_v3/benchmark/compression_ladder.csv",
146
+ "role": "task_free_compression_ladder",
147
+ "sha256": "2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b"
148
+ },
149
+ {
150
+ "bytes": 13963,
151
+ "path": "results/unoq_lineage_matrix/20260731-v1/analysis/analysis.json",
152
+ "role": "uno_q_depth_analysis",
153
+ "sha256": "c899a19e4433e3a40547a11cfcfc3b0a7b6f00c436aa8c1d742a5dc0399bf649"
154
+ }
155
+ ],
156
+ "schema": "qwen35-4l-cross-task-base-card-figures-manifest-v1",
157
+ "status": "complete"
158
+ }
benchmark/figures/cross_task/source_summary.json ADDED
@@ -0,0 +1,972 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "claim_boundaries": [
3
+ "The controlled UNO Q 4L point is a historical early-path proxy, not the final common-head quality endpoint.",
4
+ "The propaganda 24L/8L/4L endpoints differ in lineage and vocabulary; their comparison is descriptive, not a pure depth ablation.",
5
+ "All Qwen MALINT student arms are 4L; MALINT supports a 4L interface result, not 4L-versus-8L/24L depth superiority.",
6
+ "The MALINT result is post-selection internal repeated-fold evidence, not untouched held-out confirmation or SOTA evidence.",
7
+ "Mean+KD incurred measured CUDA-forward latency and peak-allocation increases; it is not claimed to be faster.",
8
+ "No ambient GPU utilization value is used as model occupancy."
9
+ ],
10
+ "malint_quality": [
11
+ {
12
+ "arm": "rightmost_hard",
13
+ "arm_label": "Rightmost + hard",
14
+ "layers": 4,
15
+ "mean": 0.522182806412052,
16
+ "metric": "weighted_f1",
17
+ "n_repeats": 3,
18
+ "role": "qwen_4l_student",
19
+ "sample_sd": 0.014974460565445307,
20
+ "threshold": 0.3
21
+ },
22
+ {
23
+ "arm": "rightmost_hard",
24
+ "arm_label": "Rightmost + hard",
25
+ "layers": 4,
26
+ "mean": 0.5109624469249431,
27
+ "metric": "macro_f1",
28
+ "n_repeats": 3,
29
+ "role": "qwen_4l_student",
30
+ "sample_sd": 0.016275524962473233,
31
+ "threshold": 0.3
32
+ },
33
+ {
34
+ "arm": "rightmost_hard",
35
+ "arm_label": "Rightmost + hard",
36
+ "layers": 4,
37
+ "mean": 0.5320256865023298,
38
+ "metric": "micro_f1",
39
+ "n_repeats": 3,
40
+ "role": "qwen_4l_student",
41
+ "sample_sd": 0.010757300158510568,
42
+ "threshold": 0.3
43
+ },
44
+ {
45
+ "arm": "rightmost_hard",
46
+ "arm_label": "Rightmost + hard",
47
+ "layers": 4,
48
+ "mean": 0.6403030303030303,
49
+ "metric": "subset_exact_match",
50
+ "n_repeats": 3,
51
+ "role": "qwen_4l_student",
52
+ "sample_sd": 0.003674047167570365,
53
+ "threshold": 0.3
54
+ },
55
+ {
56
+ "arm": "rightmost_hard",
57
+ "arm_label": "Rightmost + hard",
58
+ "layers": 4,
59
+ "mean": 0.8746666666666667,
60
+ "metric": "label_accuracy",
61
+ "n_repeats": 3,
62
+ "role": "qwen_4l_student",
63
+ "sample_sd": 0.0012106051124507965,
64
+ "threshold": 0.3
65
+ },
66
+ {
67
+ "arm": "mean_hard",
68
+ "arm_label": "Mean pool + hard",
69
+ "layers": 4,
70
+ "mean": 0.6118049035859879,
71
+ "metric": "weighted_f1",
72
+ "n_repeats": 3,
73
+ "role": "qwen_4l_student",
74
+ "sample_sd": 0.008860176348616246,
75
+ "threshold": 0.3
76
+ },
77
+ {
78
+ "arm": "mean_hard",
79
+ "arm_label": "Mean pool + hard",
80
+ "layers": 4,
81
+ "mean": 0.6055064323204021,
82
+ "metric": "macro_f1",
83
+ "n_repeats": 3,
84
+ "role": "qwen_4l_student",
85
+ "sample_sd": 0.007533985719586246,
86
+ "threshold": 0.3
87
+ },
88
+ {
89
+ "arm": "mean_hard",
90
+ "arm_label": "Mean pool + hard",
91
+ "layers": 4,
92
+ "mean": 0.6176900341883843,
93
+ "metric": "micro_f1",
94
+ "n_repeats": 3,
95
+ "role": "qwen_4l_student",
96
+ "sample_sd": 0.007461802499370809,
97
+ "threshold": 0.3
98
+ },
99
+ {
100
+ "arm": "mean_hard",
101
+ "arm_label": "Mean pool + hard",
102
+ "layers": 4,
103
+ "mean": 0.6696969696969697,
104
+ "metric": "subset_exact_match",
105
+ "n_repeats": 3,
106
+ "role": "qwen_4l_student",
107
+ "sample_sd": 0.005169309730070291,
108
+ "threshold": 0.3
109
+ },
110
+ {
111
+ "arm": "mean_hard",
112
+ "arm_label": "Mean pool + hard",
113
+ "layers": 4,
114
+ "mean": 0.8876363636363637,
115
+ "metric": "label_accuracy",
116
+ "n_repeats": 3,
117
+ "role": "qwen_4l_student",
118
+ "sample_sd": 0.0017533910474532954,
119
+ "threshold": 0.3
120
+ },
121
+ {
122
+ "arm": "mean_kd",
123
+ "arm_label": "Mean pool + KD",
124
+ "layers": 4,
125
+ "mean": 0.6346533875063399,
126
+ "metric": "weighted_f1",
127
+ "n_repeats": 3,
128
+ "role": "qwen_4l_student",
129
+ "sample_sd": 0.011966996735426847,
130
+ "threshold": 0.3
131
+ },
132
+ {
133
+ "arm": "mean_kd",
134
+ "arm_label": "Mean pool + KD",
135
+ "layers": 4,
136
+ "mean": 0.6281768437686408,
137
+ "metric": "macro_f1",
138
+ "n_repeats": 3,
139
+ "role": "qwen_4l_student",
140
+ "sample_sd": 0.014225316120600004,
141
+ "threshold": 0.3
142
+ },
143
+ {
144
+ "arm": "mean_kd",
145
+ "arm_label": "Mean pool + KD",
146
+ "layers": 4,
147
+ "mean": 0.6389632888388834,
148
+ "metric": "micro_f1",
149
+ "n_repeats": 3,
150
+ "role": "qwen_4l_student",
151
+ "sample_sd": 0.009850482866928375,
152
+ "threshold": 0.3
153
+ },
154
+ {
155
+ "arm": "mean_kd",
156
+ "arm_label": "Mean pool + KD",
157
+ "layers": 4,
158
+ "mean": 0.6557575757575758,
159
+ "metric": "subset_exact_match",
160
+ "n_repeats": 3,
161
+ "role": "qwen_4l_student",
162
+ "sample_sd": 0.003192622349349309,
163
+ "threshold": 0.3
164
+ },
165
+ {
166
+ "arm": "mean_kd",
167
+ "arm_label": "Mean pool + KD",
168
+ "layers": 4,
169
+ "mean": 0.8757575757575757,
170
+ "metric": "label_accuracy",
171
+ "n_repeats": 3,
172
+ "role": "qwen_4l_student",
173
+ "sample_sd": 0.0020436018545560413,
174
+ "threshold": 0.3
175
+ },
176
+ {
177
+ "arm": "teacher",
178
+ "arm_label": "RoBERTa teacher",
179
+ "layers": "",
180
+ "mean": 0.6046800630077942,
181
+ "metric": "weighted_f1",
182
+ "n_repeats": 3,
183
+ "role": "external_teacher",
184
+ "sample_sd": 0.01579900095351724,
185
+ "threshold": 0.3
186
+ },
187
+ {
188
+ "arm": "teacher",
189
+ "arm_label": "RoBERTa teacher",
190
+ "layers": "",
191
+ "mean": 0.5955721586042237,
192
+ "metric": "macro_f1",
193
+ "n_repeats": 3,
194
+ "role": "external_teacher",
195
+ "sample_sd": 0.01637303032583919,
196
+ "threshold": 0.3
197
+ },
198
+ {
199
+ "arm": "teacher",
200
+ "arm_label": "RoBERTa teacher",
201
+ "layers": "",
202
+ "mean": 0.6046188233688233,
203
+ "metric": "micro_f1",
204
+ "n_repeats": 3,
205
+ "role": "external_teacher",
206
+ "sample_sd": 0.01776761734134435,
207
+ "threshold": 0.3
208
+ },
209
+ {
210
+ "arm": "teacher",
211
+ "arm_label": "RoBERTa teacher",
212
+ "layers": "",
213
+ "mean": 0.62,
214
+ "metric": "subset_exact_match",
215
+ "n_repeats": 3,
216
+ "role": "external_teacher",
217
+ "sample_sd": 0.005961307749365459,
218
+ "threshold": 0.3
219
+ },
220
+ {
221
+ "arm": "teacher",
222
+ "arm_label": "RoBERTa teacher",
223
+ "layers": "",
224
+ "mean": 0.8507878787878788,
225
+ "metric": "label_accuracy",
226
+ "n_repeats": 3,
227
+ "role": "external_teacher",
228
+ "sample_sd": 0.01087469324477906,
229
+ "threshold": 0.3
230
+ }
231
+ ],
232
+ "malint_quality_contract": {
233
+ "claim_boundary": "Post-selection internal repeated-fold robustness and matched ablation only. This result is not untouched confirmation, official held-out evidence, all-label superiority, or SOTA evidence.",
234
+ "contrasts": {
235
+ "conditional_kd_mean_kd_minus_mean_hard": {
236
+ "corrected_resampled_t_interval": {
237
+ "confidence": 0.95,
238
+ "critical_t": 2.306004135204166,
239
+ "degrees_of_freedom": 8,
240
+ "formula": "sqrt((1/n + mean(n_eval/n_train)) * sample_variance)",
241
+ "interval": [
242
+ -0.0015478418507784523,
243
+ 0.05059146195651548
244
+ ],
245
+ "mean": 0.024521810052868513,
246
+ "mean_eval_train_ratio": 0.5000006195533516,
247
+ "n": 9,
248
+ "sample_variance": 0.00020913642539219796,
249
+ "standard_error": 0.011305119321192738,
250
+ "variance_correction": 0.6111117306644627
251
+ },
252
+ "fold_deltas": [
253
+ 0.04524702943141845,
254
+ 0.017956307877355804,
255
+ -0.0053068243793217285,
256
+ 0.015864642448092825,
257
+ 0.022064206421293142,
258
+ 0.03213041489613799,
259
+ 0.02962871459506977,
260
+ 0.026719118927120977,
261
+ 0.03639268025864939
262
+ ],
263
+ "fold_mean": 0.024521810052868513,
264
+ "fold_sample_standard_deviation": 0.014461549895920491,
265
+ "name": "conditional_kd_mean_kd_minus_mean_hard",
266
+ "per_label_mean_delta": {
267
+ "CPV": 0.0006438268748058898,
268
+ "PASV": 0.027106289374744647,
269
+ "PSSA": 0.058741661936162225,
270
+ "UCPI": 0.0247139382339762,
271
+ "UIOA": 0.0021463408215049995
272
+ },
273
+ "per_label_repeat_deltas": [
274
+ [
275
+ -0.01842723004694835,
276
+ 0.04865531704909942,
277
+ 0.015803620583568212,
278
+ -0.013921424268790505,
279
+ 0.03679497238444562
280
+ ],
281
+ [
282
+ 0.025183733463988223,
283
+ 0.070038654494613,
284
+ -0.0026296349009481412,
285
+ 0.021790722761596548,
286
+ 0.0004450534649754756
287
+ ],
288
+ [
289
+ -0.004825022792622202,
290
+ 0.05753101426477425,
291
+ -0.006734963218105072,
292
+ 0.0734495696314279,
293
+ 0.0369017888525075
294
+ ]
295
+ ],
296
+ "positive_fold_tail": {
297
+ "null_probability": 0.5,
298
+ "one_sided_probability_at_least_observed": 0.01953125,
299
+ "positive": 8,
300
+ "total": 9
301
+ },
302
+ "repeat_oof_deltas": [
303
+ 0.018545164631776245,
304
+ 0.020840853227636957,
305
+ 0.029159433901642884
306
+ ],
307
+ "repeat_oof_t_interval": {
308
+ "confidence": 0.95,
309
+ "critical_t": 4.302652729749462,
310
+ "degrees_of_freedom": 2,
311
+ "interval": [
312
+ 0.00897537793561895,
313
+ 0.03672158990508511
314
+ ],
315
+ "mean": 0.02284848392035203,
316
+ "n": 3,
317
+ "sample_standard_deviation": 0.005584676694496962,
318
+ "standard_error": 0.0032243145929048505
319
+ }
320
+ },
321
+ "mean_kd_minus_teacher": {
322
+ "corrected_resampled_t_interval": {
323
+ "confidence": 0.95,
324
+ "critical_t": 2.306004135204166,
325
+ "degrees_of_freedom": 8,
326
+ "formula": "sqrt((1/n + mean(n_eval/n_train)) * sample_variance)",
327
+ "interval": [
328
+ -0.03180974553628697,
329
+ 0.08853586985785872
330
+ ],
331
+ "mean": 0.02836306216078588,
332
+ "mean_eval_train_ratio": 0.5000006195533516,
333
+ "n": 9,
334
+ "sample_variance": 0.0011141913151793519,
335
+ "standard_error": 0.026093972156622066,
336
+ "variance_correction": 0.6111117306644627
337
+ },
338
+ "fold_deltas": [
339
+ -0.03162698747672954,
340
+ 0.06379890775996899,
341
+ 0.056964918230746675,
342
+ 0.04693837159160652,
343
+ 0.001345287770746495,
344
+ 0.014058020214096523,
345
+ 0.00030085368163068615,
346
+ 0.05808798200431842,
347
+ 0.04540020567068814
348
+ ],
349
+ "fold_mean": 0.02836306216078588,
350
+ "fold_sample_standard_deviation": 0.0333795044178213,
351
+ "name": "mean_kd_minus_teacher",
352
+ "per_label_mean_delta": {
353
+ "CPV": 0.03245133981428311,
354
+ "PASV": 0.04215462872506418,
355
+ "PSSA": 0.030550793916265178,
356
+ "UCPI": 0.006157663285133859,
357
+ "UIOA": 0.05170900008133931
358
+ },
359
+ "per_label_repeat_deltas": [
360
+ [
361
+ 0.010081541882876188,
362
+ 0.09069144184384492,
363
+ 0.058604490711147705,
364
+ -0.005178527872112815,
365
+ 0.0012970008399624255
366
+ ],
367
+ [
368
+ 0.05816392692322364,
369
+ -0.013100009776126686,
370
+ 0.017979371387083054,
371
+ 0.03578643578643581,
372
+ 0.01576007056748019
373
+ ],
374
+ [
375
+ 0.029108550636749497,
376
+ 0.014060949681077295,
377
+ 0.07854313814578717,
378
+ 0.09585597826086956,
379
+ 0.0014159184479589637
380
+ ]
381
+ ],
382
+ "positive_fold_tail": {
383
+ "null_probability": 0.5,
384
+ "one_sided_probability_at_least_observed": 0.01953125,
385
+ "positive": 8,
386
+ "total": 9
387
+ },
388
+ "repeat_oof_deltas": [
389
+ 0.032162246558158025,
390
+ 0.020416569493259984,
391
+ 0.037341157444219175
392
+ ],
393
+ "repeat_oof_t_interval": {
394
+ "confidence": 0.95,
395
+ "critical_t": 4.302652729749462,
396
+ "degrees_of_freedom": 2,
397
+ "interval": [
398
+ 0.00843082924454409,
399
+ 0.05151581975254736
400
+ ],
401
+ "mean": 0.029973324498545728,
402
+ "n": 3,
403
+ "sample_standard_deviation": 0.008672021342497449,
404
+ "standard_error": 0.005006793856509082
405
+ }
406
+ },
407
+ "pooling_mean_hard_minus_rightmost_hard": {
408
+ "corrected_resampled_t_interval": {
409
+ "confidence": 0.95,
410
+ "critical_t": 2.306004135204166,
411
+ "degrees_of_freedom": 8,
412
+ "formula": "sqrt((1/n + mean(n_eval/n_train)) * sample_variance)",
413
+ "interval": [
414
+ 0.056762831891190316,
415
+ 0.12504455147590826
416
+ ],
417
+ "mean": 0.0909036916835493,
418
+ "mean_eval_train_ratio": 0.5000006195533516,
419
+ "n": 9,
420
+ "sample_variance": 0.00035868079546522353,
421
+ "standard_error": 0.014805203195932804,
422
+ "variance_correction": 0.6111117306644627
423
+ },
424
+ "fold_deltas": [
425
+ 0.06678423059606675,
426
+ 0.08178035353463753,
427
+ 0.08672762777739151,
428
+ 0.09344774812516743,
429
+ 0.0878998694863643,
430
+ 0.07415570161139795,
431
+ 0.11094545718958193,
432
+ 0.08708564234043492,
433
+ 0.12930659449090132
434
+ ],
435
+ "fold_mean": 0.0909036916835493,
436
+ "fold_sample_standard_deviation": 0.01893886996273071,
437
+ "name": "pooling_mean_hard_minus_rightmost_hard",
438
+ "per_label_mean_delta": {
439
+ "CPV": 0.14860808426516173,
440
+ "PASV": 0.09778905085334237,
441
+ "PSSA": 0.07923875253833933,
442
+ "UCPI": 0.055020095369494,
443
+ "UIOA": 0.09206394395095778
444
+ },
445
+ "per_label_repeat_deltas": [
446
+ [
447
+ 0.09436936936936935,
448
+ 0.07977821265492502,
449
+ 0.10417829690385194,
450
+ 0.07872402692420255,
451
+ 0.03863953001237319
452
+ ],
453
+ [
454
+ 0.15668016194331985,
455
+ 0.04167051934042232,
456
+ 0.06934192200557099,
457
+ 0.11942791523123497,
458
+ 0.06908520415955133
459
+ ],
460
+ [
461
+ 0.194774721482796,
462
+ 0.11626752561967063,
463
+ 0.1026716129434504,
464
+ 0.09521521040458958,
465
+ 0.05733555193655748
466
+ ]
467
+ ],
468
+ "positive_fold_tail": {
469
+ "null_probability": 0.5,
470
+ "one_sided_probability_at_least_observed": 0.001953125,
471
+ "positive": 9,
472
+ "total": 9
473
+ },
474
+ "repeat_oof_deltas": [
475
+ 0.07574451062131737,
476
+ 0.08540665760778032,
477
+ 0.10771512329271016
478
+ ],
479
+ "repeat_oof_t_interval": {
480
+ "confidence": 0.95,
481
+ "critical_t": 4.302652729749462,
482
+ "degrees_of_freedom": 2,
483
+ "interval": [
484
+ 0.048890004893968025,
485
+ 0.13035418945390387
486
+ ],
487
+ "mean": 0.08962209717393595,
488
+ "n": 3,
489
+ "sample_standard_deviation": 0.01639687368671199,
490
+ "standard_error": 0.009466739436891459
491
+ }
492
+ },
493
+ "primary_mean_kd_minus_rightmost_hard": {
494
+ "corrected_resampled_t_interval": {
495
+ "confidence": 0.95,
496
+ "critical_t": 2.306004135204166,
497
+ "degrees_of_freedom": 8,
498
+ "formula": "sqrt((1/n + mean(n_eval/n_train)) * sample_variance)",
499
+ "interval": [
500
+ 0.07160480852626178,
501
+ 0.15924619494657383
502
+ ],
503
+ "mean": 0.1154255017364178,
504
+ "mean_eval_train_ratio": 0.5000006195533516,
505
+ "n": 9,
506
+ "sample_variance": 0.0005909050521203029,
507
+ "standard_error": 0.019002868442938105,
508
+ "variance_correction": 0.6111117306644627
509
+ },
510
+ "fold_deltas": [
511
+ 0.1120312600274852,
512
+ 0.09973666141199333,
513
+ 0.08142080339806979,
514
+ 0.10931239057326025,
515
+ 0.10996407590765744,
516
+ 0.10628611650753594,
517
+ 0.1405741717846517,
518
+ 0.1138047612675559,
519
+ 0.1656992747495507
520
+ ],
521
+ "fold_mean": 0.1154255017364178,
522
+ "fold_sample_standard_deviation": 0.024308538666902683,
523
+ "name": "primary_mean_kd_minus_rightmost_hard",
524
+ "per_label_mean_delta": {
525
+ "CPV": 0.14925191113996764,
526
+ "PASV": 0.12489534022808702,
527
+ "PSSA": 0.13798041447450154,
528
+ "UCPI": 0.0797340336034702,
529
+ "UIOA": 0.09421028477246278
530
+ },
531
+ "per_label_repeat_deltas": [
532
+ [
533
+ 0.075942139322421,
534
+ 0.12843352970402444,
535
+ 0.11998191748742015,
536
+ 0.06480260265541204,
537
+ 0.07543450239681881
538
+ ],
539
+ [
540
+ 0.18186389540730807,
541
+ 0.11170917383503531,
542
+ 0.06671228710462285,
543
+ 0.14121863799283152,
544
+ 0.0695302576245268
545
+ ],
546
+ [
547
+ 0.1899496986901738,
548
+ 0.17379853988444488,
549
+ 0.09593664972534532,
550
+ 0.16866478003601748,
551
+ 0.09423734078906498
552
+ ]
553
+ ],
554
+ "positive_fold_tail": {
555
+ "null_probability": 0.5,
556
+ "one_sided_probability_at_least_observed": 0.001953125,
557
+ "positive": 9,
558
+ "total": 9
559
+ },
560
+ "repeat_oof_deltas": [
561
+ 0.09428967525309362,
562
+ 0.10624751083541728,
563
+ 0.13687455719435304
564
+ ],
565
+ "repeat_oof_t_interval": {
566
+ "confidence": 0.95,
567
+ "critical_t": 4.302652729749462,
568
+ "degrees_of_freedom": 2,
569
+ "interval": [
570
+ 0.05790922361228608,
571
+ 0.1670319385762899
572
+ ],
573
+ "mean": 0.11247058109428798,
574
+ "n": 3,
575
+ "sample_standard_deviation": 0.02196390209122417,
576
+ "standard_error": 0.01268086478482286
577
+ }
578
+ }
579
+ },
580
+ "promotion_gate": {
581
+ "fold_gate_passed": true,
582
+ "observed_positive_fold_deltas": 9,
583
+ "observed_positive_repeat_oof_deltas": 3,
584
+ "passed": true,
585
+ "repeat_gate_passed": true,
586
+ "required_positive_fold_deltas": 7,
587
+ "required_positive_repeat_oof_deltas": 3
588
+ },
589
+ "study": {
590
+ "fold_count": 9,
591
+ "fold_runner_protocol": "malint-qwen35-4l-repeated-matched-cv/1.0",
592
+ "fold_runner_sha256": "c71af81f1182ce860c39c107dd19eff02f1addd8dd646e3eebaadedc4503067c",
593
+ "heldout_split_accessed": false,
594
+ "post_selection_internal_evidence": true,
595
+ "raw_text_written": false,
596
+ "repeat_count": 3,
597
+ "rows_per_repeat": 1100,
598
+ "terminal_evaluation_only": true,
599
+ "threshold": 0.3
600
+ }
601
+ },
602
+ "malint_resources": {
603
+ "all_resource_parity_gates_passed": true,
604
+ "claim_boundary": {
605
+ "development_labels_read": false,
606
+ "heldout_access": false,
607
+ "quality_scored": false,
608
+ "resource_only": true,
609
+ "sota_claim_supported": false,
610
+ "timing_scope": "CUDA model forward with CUDA-resident token IDs and masks; tokenization, loading, disk, and host-to-device transfer excluded"
611
+ },
612
+ "comparisons": {
613
+ "1": {
614
+ "existing_over_mean_kd_throughput_ratio": 1.0456925078172845,
615
+ "mean_kd_minus_existing_parameters": 5,
616
+ "mean_kd_minus_existing_serialized_weight_bytes": -220,
617
+ "mean_kd_over_existing_latency_ratio": 1.0456925078172845,
618
+ "mean_kd_over_existing_peak_allocated_ratio": 1.0007640009038254,
619
+ "paired_round_latency_ratio_summary": {
620
+ "maximum": 1.0527066487704841,
621
+ "mean": 1.0488242542281458,
622
+ "median": 1.0511611345543765,
623
+ "minimum": 1.042604979359577,
624
+ "n": 3,
625
+ "p05": 1.043460594879057,
626
+ "p95": 1.0525520973488733,
627
+ "sample_standard_deviation": 0.005441202851039159
628
+ },
629
+ "paired_round_latency_ratios": [
630
+ 1.0511611345543765,
631
+ 1.042604979359577,
632
+ 1.0527066487704841
633
+ ],
634
+ "resource_parity_gate": {
635
+ "four_layers_and_length_256_pass": true,
636
+ "latency_pass": true,
637
+ "latency_ratio": 1.0456925078172845,
638
+ "latency_ratio_limit": 1.05,
639
+ "parameter_pass": true,
640
+ "parameter_relative_difference": 1.4965497464629228e-08,
641
+ "parameter_relative_limit": 0.0001,
642
+ "passed": true,
643
+ "peak_allocated_ratio": 1.0007640009038254,
644
+ "peak_memory_pass": true,
645
+ "peak_memory_ratio_limit": 1.05
646
+ }
647
+ },
648
+ "4": {
649
+ "existing_over_mean_kd_throughput_ratio": 1.0177048568315403,
650
+ "mean_kd_minus_existing_parameters": 5,
651
+ "mean_kd_minus_existing_serialized_weight_bytes": -220,
652
+ "mean_kd_over_existing_latency_ratio": 1.0177048568315403,
653
+ "mean_kd_over_existing_peak_allocated_ratio": 1.0007213257185181,
654
+ "paired_round_latency_ratio_summary": {
655
+ "maximum": 1.0378811577131521,
656
+ "mean": 1.0299336811276214,
657
+ "median": 1.0272510844529572,
658
+ "minimum": 1.0246688012167542,
659
+ "n": 3,
660
+ "p05": 1.0249270295403745,
661
+ "p95": 1.0368181503871325,
662
+ "sample_standard_deviation": 0.007002773360331318
663
+ },
664
+ "paired_round_latency_ratios": [
665
+ 1.0272510844529572,
666
+ 1.0378811577131521,
667
+ 1.0246688012167542
668
+ ],
669
+ "resource_parity_gate": {
670
+ "four_layers_and_length_256_pass": true,
671
+ "latency_pass": true,
672
+ "latency_ratio": 1.0177048568315403,
673
+ "latency_ratio_limit": 1.05,
674
+ "parameter_pass": true,
675
+ "parameter_relative_difference": 1.4965497464629228e-08,
676
+ "parameter_relative_limit": 0.0001,
677
+ "passed": true,
678
+ "peak_allocated_ratio": 1.0007213257185181,
679
+ "peak_memory_pass": true,
680
+ "peak_memory_ratio_limit": 1.05
681
+ }
682
+ }
683
+ },
684
+ "rows": [
685
+ {
686
+ "arm": "existing_rightmost",
687
+ "batch": 1,
688
+ "latency_ms_per_example": 18.474629627574572,
689
+ "parameter_count": 334101824,
690
+ "peak_allocated_bytes": 1373150208,
691
+ "worker_count": 3
692
+ },
693
+ {
694
+ "arm": "mean_kd",
695
+ "batch": 1,
696
+ "latency_ms_per_example": 19.318781786253957,
697
+ "parameter_count": 334101829,
698
+ "peak_allocated_bytes": 1374199296,
699
+ "worker_count": 3
700
+ },
701
+ {
702
+ "arm": "existing_rightmost",
703
+ "batch": 4,
704
+ "latency_ms_per_example": 4.589751928502863,
705
+ "parameter_count": 334101824,
706
+ "peak_allocated_bytes": 1454388736,
707
+ "worker_count": 3
708
+ },
709
+ {
710
+ "arm": "mean_kd",
711
+ "batch": 4,
712
+ "latency_ms_per_example": 4.671012829289292,
713
+ "parameter_count": 334101829,
714
+ "peak_allocated_bytes": 1455437824,
715
+ "worker_count": 3
716
+ }
717
+ ]
718
+ },
719
+ "privacy": {
720
+ "aggregate_only": true,
721
+ "contains_article_ids": false,
722
+ "contains_logits_probabilities_or_predictions": false,
723
+ "contains_raw_text": false,
724
+ "contains_row_labels": false
725
+ },
726
+ "propaganda_operational_endpoints": [
727
+ {
728
+ "arm": "qwen35-full24l-commonhead",
729
+ "label": "24L",
730
+ "layers": 24,
731
+ "lineage_note": "same five-epoch common-head protocol",
732
+ "macro_f1": {
733
+ "mean": 0.5642638295960175,
734
+ "n": 3,
735
+ "sample_sd": 0.006258972649035587
736
+ },
737
+ "micro_f1": {
738
+ "mean": 0.6264844890231888,
739
+ "n": 3,
740
+ "sample_sd": 0.01398932248989461
741
+ },
742
+ "parameters": 752407360,
743
+ "peak_cuda_allocated_bytes": {
744
+ "mean": 7635339264.0,
745
+ "n": 3,
746
+ "sample_sd": 0.0
747
+ },
748
+ "subset_exact_match": {
749
+ "mean": 0.0303030303030303,
750
+ "n": 3,
751
+ "sample_sd": 0.027773186030035393
752
+ },
753
+ "test_articles": 55,
754
+ "test_seconds_per_article": {
755
+ "mean": 0.586362602903109,
756
+ "n": 3,
757
+ "sample_sd": 0.005979023788731576
758
+ },
759
+ "total_wall_seconds": {
760
+ "mean": 12842.063612925334,
761
+ "n": 3,
762
+ "sample_sd": 94.80225434811064
763
+ }
764
+ },
765
+ {
766
+ "arm": "qwen35-c8q8l-commonhead",
767
+ "label": "8L",
768
+ "layers": 8,
769
+ "lineage_note": "same five-epoch common-head protocol",
770
+ "macro_f1": {
771
+ "mean": 0.5366873350531403,
772
+ "n": 3,
773
+ "sample_sd": 0.017264532146217884
774
+ },
775
+ "micro_f1": {
776
+ "mean": 0.6238830315003392,
777
+ "n": 3,
778
+ "sample_sd": 0.013623163834245318
779
+ },
780
+ "parameters": 420332480,
781
+ "peak_cuda_allocated_bytes": {
782
+ "mean": 4257894400.0,
783
+ "n": 3,
784
+ "sample_sd": 3632373.815194686
785
+ },
786
+ "subset_exact_match": {
787
+ "mean": 0.006060606060606061,
788
+ "n": 3,
789
+ "sample_sd": 0.01049727762162956
790
+ },
791
+ "test_articles": 55,
792
+ "test_seconds_per_article": {
793
+ "mean": 0.206657676709195,
794
+ "n": 3,
795
+ "sample_sd": 0.002071265185964238
796
+ },
797
+ "total_wall_seconds": {
798
+ "mean": 4291.582853937677,
799
+ "n": 3,
800
+ "sample_sd": 44.49609583978631
801
+ }
802
+ },
803
+ {
804
+ "arm": "qwen35-standalone4l-commonhead",
805
+ "label": "4L",
806
+ "layers": 4,
807
+ "lineage_note": "same five-epoch common-head protocol",
808
+ "macro_f1": {
809
+ "mean": 0.5875679559555907,
810
+ "n": 3,
811
+ "sample_sd": 0.007165753447433348
812
+ },
813
+ "micro_f1": {
814
+ "mean": 0.647978512363484,
815
+ "n": 3,
816
+ "sample_sd": 0.0018739735696546695
817
+ },
818
+ "parameters": 210903360,
819
+ "peak_cuda_allocated_bytes": {
820
+ "mean": 2143655936.0,
821
+ "n": 3,
822
+ "sample_sd": 0.0
823
+ },
824
+ "subset_exact_match": {
825
+ "mean": 0.006060606060606061,
826
+ "n": 3,
827
+ "sample_sd": 0.01049727762162956
828
+ },
829
+ "test_articles": 55,
830
+ "test_seconds_per_article": {
831
+ "mean": 0.08543555729062946,
832
+ "n": 3,
833
+ "sample_sd": 0.0005980667826383295
834
+ },
835
+ "total_wall_seconds": {
836
+ "mean": 1643.6388439506602,
837
+ "n": 3,
838
+ "sample_sd": 9.226529522850198
839
+ }
840
+ }
841
+ ],
842
+ "schema": "qwen35-4l-cross-task-base-card-figures-v1",
843
+ "status": "complete",
844
+ "task_free_compression_ladder": [
845
+ {
846
+ "layers": 24,
847
+ "name": "24L upstream text teacher",
848
+ "parameters": 752393024,
849
+ "weight_bytes": 1504825632,
850
+ "weight_mib": 1435.1135559082031
851
+ },
852
+ {
853
+ "layers": 8,
854
+ "name": "8L task-free KD",
855
+ "parameters": 420318144,
856
+ "weight_bytes": 840647880,
857
+ "weight_mib": 801.7042922973633
858
+ },
859
+ {
860
+ "layers": 6,
861
+ "name": "6L task-free KD",
862
+ "parameters": 377207424,
863
+ "weight_bytes": 754423424,
864
+ "weight_mib": 719.4742431640625
865
+ },
866
+ {
867
+ "layers": 4,
868
+ "name": "4L task-free KD root",
869
+ "parameters": 334096704,
870
+ "weight_bytes": 668198976,
871
+ "weight_mib": 637.2442016601562
872
+ }
873
+ ],
874
+ "uno_q_controlled_depth": {
875
+ "boundary": "Matched V128k/F16 runtime comparison. The 4L arm is the historical early-path proxy, not the final common-head quality model.",
876
+ "claim_boundaries": [
877
+ "Only layer_effect_primary is used for same-vocabulary F16 depth-effect estimates.",
878
+ "Only vocabulary_effect_primary is used for exact-remap vocabulary-effect estimates.",
879
+ "Operational and cross-generation groups are descriptive because precision, heads, layer maps, training lineages, or architectures differ.",
880
+ "llama-bench embedding-mode latency measures backbone/classifier compute but does not validate evidence-span quality.",
881
+ "CPU occupancy is instantaneous saturation; core-seconds is a compute-duration proxy, not measured energy.",
882
+ "No power meter was used, so energy consumption is not reported.",
883
+ "Temperature is device- and ambient-specific and is descriptive, not a universal hardware constant."
884
+ ],
885
+ "rows": [
886
+ {
887
+ "layers": 8,
888
+ "metric": "artifact_mib",
889
+ "metric_label": "Artifact size",
890
+ "model": "8L",
891
+ "normalized_8l_percent": 100.0
892
+ },
893
+ {
894
+ "layers": 6,
895
+ "metric": "artifact_mib",
896
+ "metric_label": "Artifact size",
897
+ "model": "6L",
898
+ "normalized_8l_percent": 85.59176355424236
899
+ },
900
+ {
901
+ "layers": 4,
902
+ "metric": "artifact_mib",
903
+ "metric_label": "Artifact size",
904
+ "model": "4L proxy",
905
+ "normalized_8l_percent": 72.2607217304372
906
+ },
907
+ {
908
+ "layers": 8,
909
+ "metric": "latency_pp128_ms",
910
+ "metric_label": "PP128 latency",
911
+ "model": "8L",
912
+ "normalized_8l_percent": 100.0
913
+ },
914
+ {
915
+ "layers": 6,
916
+ "metric": "latency_pp128_ms",
917
+ "metric_label": "PP128 latency",
918
+ "model": "6L",
919
+ "normalized_8l_percent": 76.36565126547553
920
+ },
921
+ {
922
+ "layers": 4,
923
+ "metric": "latency_pp128_ms",
924
+ "metric_label": "PP128 latency",
925
+ "model": "4L proxy",
926
+ "normalized_8l_percent": 50.16216836129361
927
+ },
928
+ {
929
+ "layers": 8,
930
+ "metric": "peak_rss_mib",
931
+ "metric_label": "Peak RSS",
932
+ "model": "8L",
933
+ "normalized_8l_percent": 100.0
934
+ },
935
+ {
936
+ "layers": 6,
937
+ "metric": "peak_rss_mib",
938
+ "metric_label": "Peak RSS",
939
+ "model": "6L",
940
+ "normalized_8l_percent": 87.50983221571457
941
+ },
942
+ {
943
+ "layers": 4,
944
+ "metric": "peak_rss_mib",
945
+ "metric_label": "Peak RSS",
946
+ "model": "4L proxy",
947
+ "normalized_8l_percent": 78.19200407435702
948
+ },
949
+ {
950
+ "layers": 8,
951
+ "metric": "mean_core_seconds_per_round",
952
+ "metric_label": "Core-seconds",
953
+ "model": "8L",
954
+ "normalized_8l_percent": 100.0
955
+ },
956
+ {
957
+ "layers": 6,
958
+ "metric": "mean_core_seconds_per_round",
959
+ "metric_label": "Core-seconds",
960
+ "model": "6L",
961
+ "normalized_8l_percent": 76.16702179915283
962
+ },
963
+ {
964
+ "layers": 4,
965
+ "metric": "mean_core_seconds_per_round",
966
+ "metric_label": "Core-seconds",
967
+ "model": "4L proxy",
968
+ "normalized_8l_percent": 51.03561371028788
969
+ }
970
+ ]
971
+ }
972
+ }