Add the sealed final test and expand the 65-arm field with seven figures

New section 'The final test': the 2026-09-23 fixed-protocol evaluation of the SemEval-2020 Task 11 v2 re-split, reported in full including the two arms that fell. Prior test exposure is disclosed; this is not a never-seen holdout.

'Beside other models' gains four figures on the whole 65-arm field: the Pareto frontier on latency and energy, the metric spans (macro F1 1.2x against 64.4x/42.9x/40.3x on file/energy/latency), depth within each encoder family (7 of 12 peak below full depth), and all 65 arms with seed SDs against the all-positive floor.

README_KO.md mirrors both. SHA256SUMS regenerated: 343 -> 351 entries.

.gitattributes

@@ -19,3 +19,10 @@ docker-train/vendor/runtime_source.tar.gz filter=lfs diff=lfs merge=lfs -text
 models/semeval-propaganda/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 models/semeval-propaganda/seeds/seed42/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 models/semeval-propaganda/seeds/seed43/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/11_final_test_calibration_to_test.png filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/12_final_test_per_label.png filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/13_final_test_seed_spread.png filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/14_field_quality_vs_cost.png filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/15_field_metric_spans.png filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/16_field_depth_effect.png filter=lfs diff=lfs merge=lfs -text
+benchmark/figures/17_field_resolution_floor.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -95,6 +95,10 @@ architecture is what costs the time, not the vocabulary cut or the depth — but
 that, so it stays a guess. What the cut and the depth did move is the file and the energy, and
 those are visible in the panels above.
 
+Everything in this table is the opened calibration split. One fixed-protocol test evaluation was
+run on 2026-09-23 against a re-split this project had used before — not a never-seen holdout — and
+it is reported in full, decreases included, under [**The final test**](#the-final-test).
+
 ---
 
 ## Run it in a container
@@ -438,8 +442,120 @@ Full table: [`benchmark/full/61_arm_bf16.csv`](benchmark/full/61_arm_bf16.csv).
 ![the top arms overlap](benchmark/figures/02_top_arms_overlap.png)
 ![depth](benchmark/figures/03_depth_quality_and_cost.png)
 
+### What the whole field looks like
+
+![quality against cost across all 65 arms](benchmark/figures/14_field_quality_vs_cost.png)
+
+Both panels hold the same 65 arms; only the cost axis changes. The dashed line is the Pareto
+frontier — the arms that no other arm beats on quality and cost at the same time. Seven arms hold
+it on latency, eight on energy, and the RoBERTa family holds the cheap half of both. This root
+sits at the top of the quality axis and pays about 2.7x the latency of `roberta-base` to be there.
+
+![what actually varies across the field](benchmark/figures/15_field_metric_spans.png)
+
+Laid beside each other, the spans are the finding. Across the whole field macro F1 moves **1.2x**
+— 0.4756 to 0.5781 — while the weight file moves **64.4x**, GPU energy **42.9x** and document
+latency **40.3x**. The quality span is narrower than even that suggests: its bottom is exactly the
+all-positive predictor, so the entire field fits inside 0.093 of usable range above a model that
+answers yes to everything. Peak RSS barely moves, because it is dominated by the harness rather
+than by the weights.
+
+![depth within each encoder family](benchmark/figures/16_field_depth_effect.png)
+
+Every encoder family was cut to 4, 6 and 8 layers under the same protocol, and its own full-depth
+checkpoint was kept beside them. **In 7 of the 12 families the best arm is not the deepest one.**
+`bert-base`, `mbert` and `electra-base` peak at 4 layers, `minilm-multilingual` at 6, `mpnet-base`
+and `mdeberta-v3` at 8. The five families where depth does pay — roberta, xlmr, deberta-v3,
+albert, modernbert — pay for it in latency and file size. Three arms did not train at all under
+the shared protocol and sit exactly on the floor.
+
+This is an observation about this task at this resolution. It is not a claim that depth does not
+matter, and it is not why this root has four layers — section 3 is.
+
+![all 65 arms on one axis](benchmark/figures/17_field_resolution_floor.png)
+
+The last figure is the honest summary of the ordering. The median three-seed SD across the field
+is 0.0116, and six arms sit within one such SD of the best. Several SD bars are wider than the
+whole gap between the top ten arms. The ranking is reported because it was measured, not because
+the measurement resolves it.
+
 ---
 
+---
+
+## The final test
+
+On 2026-09-23 one fixed-protocol evaluation was run against the SemEval-2020 Task 11 **v2 test
+re-split**, under a preregistration frozen before any test byte was read.
+
+**It is not a never-seen holdout.** This project has used the v2 split before, and the frozen
+document records prior test exposure as `true`. It is not the official leaderboard test, there is
+no secure-harness attestation, and nothing in it establishes Korean quality or general
+intent-classification behaviour. An earlier draft of the preregistration wrongly claimed the split
+had never been opened; that draft is kept beside the corrected one rather than deleted.
+
+What the freeze fixed in advance: the roster of eleven arms, the decision threshold for every
+arm/seed/precision — each read from the arm's existing calibration report and never recomputed on
+test — the aggregation rule, the metric set, and the rule that every result would be published
+whatever it said. The archived payload had to match SHA-256 `ed224269dc…` before extraction. It
+did. 31 of 31 jobs completed, 55 documents, 14 labels, zero failures, no training and no threshold
+search. All 31 per-run metrics were then recomputed from the saved probabilities by a separate
+standard-library implementation and agreed with the evaluator.
+
+![calibration to sealed test](benchmark/figures/11_final_test_calibration_to_test.png)
+
+| Arm | Precision | Calibration | Test | Δ |
+|---|---|---:|---:|---:|
+| **task-blind 4L N=32,768 — this root's classifier** | BF16 | 0.5719 ± 0.0092 | **0.5988 ± 0.0103** | **+0.0269** |
+| task-blind 4L N=32,768 | Q8_0 | 0.5849 (seed 41) | 0.5862 (seed 41) | +0.0013 |
+| task-blind 4L N=65,536 | BF16 | 0.5775 ± 0.0076 | 0.5832 ± 0.0047 | +0.0057 |
+| task-blind 4L N=16,384 | BF16 | 0.5787 ± 0.0021 | 0.5379 ± 0.0496 | −0.0408 |
+| task-blind 4L N=8,192 | BF16 | 0.5699 ± 0.0160 | 0.5555 ± 0.0070 | −0.0144 |
+| historical 4L v128k | BF16 | 0.5637 ± 0.0321 | 0.5747 ± 0.0183 | +0.0110 |
+| historical 8L v248k | BF16 | 0.5686 ± 0.0212 | 0.5718 ± 0.0213 | +0.0032 |
+| historical 4L v248k | BF16 | 0.5602 ± 0.0391 | 0.5707 ± 0.0081 | +0.0105 |
+| historical 24L v248k | BF16 | 0.5487 ± 0.0228 | 0.5612 ± 0.0102 | +0.0125 |
+| roberta-base 12L | BF16 | 0.5524 ± 0.0143 | 0.5883 ± 0.0109 | +0.0359 |
+| roberta-base 6L | BF16 | 0.5427 ± 0.0139 | 0.5839 ± 0.0152 | +0.0411 |
+| all-positive predictor | — | 0.4756 | **0.4866** | — |
+
+Quantization cost 0.0013 macro F1 on test, which is the same "nothing measurable" that section 4
+saw on calibration, now seen on a split the thresholds were not fitted to.
+
+**The +0.0269 is not a result about this model.** Nine of the eleven arms moved up, the eleven-arm
+mean move is +0.0085, and the two RoBERTa references moved up the *most* of all. Whatever makes
+this test split score higher than the calibration split applies to the whole board, so the move is
+a property of the split pair rather than of any arm on it.
+
+Neither does any arm separate from another. This root's mean ± SD interval overlaps `roberta-base`
+(0.5883 ± 0.0109), `roberta-base-6l` (0.5839 ± 0.0152) and both historical 4L arms. Where it does
+clear an interval it clears N=65,536 by 0.0006 and N=16,384 by 0.0010 — gaps one to two orders of
+magnitude below the spread of the design that produced them.
+
+Two arms moved down. N=8,192 fell 0.0144, and N=16,384 fell 0.0408 while carrying the widest
+spread in the table.
+
+![seed spread on the sealed test](benchmark/figures/13_final_test_seed_spread.png)
+
+Every arm and every seed lands inside a 0.103-wide band on 55 documents, and for most arms the
+distance between their own three seeds is larger than the distance to their neighbours. N=16,384
+is the extreme: one seed at 0.5951 against two near 0.51.
+
+![per-label results for the published root](benchmark/figures/12_final_test_per_label.png)
+
+Per label, F1 tracks support almost monotonically — `Loaded_Language` at 45 supporting documents
+scores 0.918, and the six labels with 14 or fewer land between 0.446 and 0.481. Recall exceeds
+precision on all fourteen labels: at the frozen thresholds, which run from 0.05 to 0.3, the head
+over-predicts everywhere. A head trained for a deployment would be calibrated differently.
+
+The v2 split is now retired from tuning and model selection under this protocol. v1 was not
+opened, and its never-used status is not asserted.
+
+Evidence: `REPORT.md` lines 1–89, `RESULTS.json` lines 1–742 (SHA-256
+`0d2da5e2e8b250671bf3b19be0e7bce49f14fe492f9c71ce75717b0eee02dec7`), `PER_LABEL.csv`, and the
+frozen preregistration (SHA-256 `85ec92a0be…`). All six are registered in this project's research
+store, and the run is recorded there as `artifact_final_test_20260923_v1`.
+
 ## Two things we checked because they would have changed the reading
 
 ### Does the host matter?
@@ -637,8 +753,10 @@ described at the top of this card.
 
 ## What this does not tell you
 
-- **No held-out evaluation.** Every number here is the opened 56-document calibration split.
-  There is no `test.jsonl` in this repository.
+- **The one test evaluation is not a holdout.** The 2026-09-23 run above used the v2 re-split,
+  which this project had used before; its frozen preregistration records that prior exposure.
+  Every other number on this card is the opened 56-document calibration split, and there is no
+  `test.jsonl` in this repository.
 - **The evaluation is small.** 56 documents, 14 labels, and the rarest label appears in 6 of
   them, so one fourteenth of macro F1 rides on six samples. This is the most likely source of the
   0.005–0.039 seed SDs.
@@ -661,7 +779,7 @@ described at the top of this card.
 | root | the 39,866-entry headless backbone |
 | [`root_manifest.json`](root_manifest.json) | its vocabulary rule, layer maps, hashes, why this grid point |
 | [`benchmark/full/`](benchmark/full/) | the 61-arm table, the grid, the same-host cuts |
-| [`benchmark/figures/`](benchmark/figures/) | the ten figures above |
+| [`benchmark/figures/`](benchmark/figures/) | the seventeen figures above |
 | [`runtime/`](runtime/) | the two relaxed sources of the dedicated runtime, and its own README |
 | [`docker/`](docker/) | a container that runs the backbone with a head on it, CPU only |
 | [`docker-train/`](docker-train/) | train a classifier on your own data, convert, serve |

README_KO.md

@@ -71,6 +71,10 @@ Qwen3.5-0.8B에서 증류한 **헤드 없는 4계층 텍스트 백본**이며,
 분리해 내지는 않았으므로, 그것은 짐작으로 남습니다. 절단과 깊이가 실제로 움직인 것은 파일과
 에너지이며, 그것은 위의 패널에 보입니다.
 
+이 표의 모든 값은 열어 본 캘리브레이션 분할입니다. 고정 프로토콜 테스트 평가는 2026-09-23에 한 번,
+이 프로젝트가 전에 사용한 적 있는 재분할을 대상으로 수행했으며 — 한 번도 본 적 없는 홀드아웃이
+아닙니다 — 내려간 값까지 포함해 [**최종 테스트**](#최종-테스트)에 전부 보고했습니다.
+
 ---
 
 ## 컨테이너로 실행하기
@@ -415,8 +419,118 @@ p50은 40x(5.7 ms에서 228.9 ms), 가중치 파일은 64x(22 MiB에서 1,435 Mi
 ![상위 arm들이 겹친다](benchmark/figures/02_top_arms_overlap.png)
 ![깊이](benchmark/figures/03_depth_quality_and_cost.png)
 
+### 전체 판이 어떻게 생겼는가
+
+![65개 arm의 품질 대 비용](benchmark/figures/14_field_quality_vs_cost.png)
+
+두 패널은 같은 65개 arm을 담고 있으며, 비용 축만 다릅니다. 점선은 파레토 경계 — 품질과 비용을
+동시에 이기는 다른 arm이 없는 arm들 — 입니다. 지연 기준으로는 일곱 개가, 에너지 기준으로는 여덟
+개가 경계를 이루고, 두 경계의 싼 쪽 절반은 모두 RoBERTa 계열이 차지합니다. 이 루트는 품질 축의
+위쪽에 있고, 그 자리에 있기 위해 `roberta-base`의 약 2.7배 지연을 지불합니다.
+
+![판에서 실제로 변하는 것](benchmark/figures/15_field_metric_spans.png)
+
+나란히 놓으면 이 폭 자체가 결과입니다. 판 전체에서 macro F1은 **1.2배** — 0.4756에서 0.5781까지 —
+움직이는 동안, 가중치 파일은 **64.4배**, GPU 에너지는 **42.9배**, 문서 지연은 **40.3배**
+움직입니다. 품질의 폭은 그 숫자가 보여 주는 것보다도 좁습니다. 그 아래 끝이 정확히 전부 양성으로
+답하는 예측기이므로, 판 전체가 "무엇이든 예"라고 답하는 모델 위 0.093의 유효 범위 안에 들어갑니다.
+peak RSS는 거의 움직이지 않는데, 이는 가중치가 아니라 하네스가 그 값을 지배하기 때문입니다.
+
+![인코더 계열별 깊이 효과](benchmark/figures/16_field_depth_effect.png)
+
+모든 인코더 계열을 같은 프로토콜로 4, 6, 8 계층으로 잘랐고, 각 계열의 전층 체크포인트를 그 옆에
+그대로 두었습니다. **12개 계열 중 7개에서 가장 좋은 arm은 가장 깊은 것이 아닙니다.** `bert-base`,
+`mbert`, `electra-base`는 4계층에서, `minilm-multilingual`은 6계층에서, `mpnet-base`와
+`mdeberta-v3`는 8계층에서 정점을 찍습니다. 깊이가 값을 하는 다섯 계열 — roberta, xlmr, deberta-v3,
+albert, modernbert — 은 그 대가를 지연과 파일 크기로 치릅니다. 세 개의 arm은 공유 프로토콜 아래에서
+아예 학습되지 않았고 정확히 바닥에 앉아 있습니다.
+
+이것은 이 과제를, 이 해상도에서 관찰한 결과입니다. 깊이가 중요하지 않다는 주장이 아니며, 이 루트가
+네 계층인 이유도 아닙니다. 그 이유는 3절에 있습니다.
+
+![65개 arm 전부를 한 축 위에](benchmark/figures/17_field_resolution_floor.png)
+
+마지막 그림이 이 순위에 대한 정직한 요약입니다. 판 전체의 3시드 SD 중앙값은 0.0116이고, 여섯 개의
+arm이 최고점으로부터 그 한 개 SD 안에 들어 있습니다. 몇몇 SD 막대는 상위 열 개 arm 사이의 간격
+전체보다도 넓습니다. 이 순위는 측정했기 때문에 보고하는 것이지, 측정이 그것을 가려냈기 때문이
+아닙니다.
+
 ---
 
+---
+
+## 최종 테스트
+
+2026-09-23에 SemEval-2020 Task 11 **v2 테스트 재분할**을 대상으로 고정 프로토콜 평가를 한 번
+수행했습니다. 사전등록 문서는 테스트 바이트를 한 개도 읽기 전에 동결되었습니다.
+
+**이것은 한 번도 본 적 없는 홀드아웃이 아닙니다.** 이 프로젝트는 v2 분할을 전에 사용한 적이 있고,
+동결된 문서는 이전 테스트 노출을 `true`로 기록하고 있습니다. 공식 리더보드 테스트가 아니며, 보안
+하네스 증명도 없고, 여기서 한국어 품질이나 일반적인 의도 분류 성능이 확립되지도 않습니다.
+사전등록 초안은 이 분할이 한 번도 열린 적 없다고 잘못 적었는데, 그 초안은 삭제하지 않고 수정본
+옆에 그대로 보관하고 있습니다.
+
+동결이 미리 고정한 것은 다음과 같습니다. 열한 개 arm의 명단, arm·시드·정밀도별 결정 임계값 — 각각
+그 arm의 기존 캘리브레이션 보고서에서 읽어 온 값이며 테스트에서 다시 계산하지 않았습니다 — 집계
+규칙, 지표 집합, 그리고 어떤 결과가 나오든 전부 공개한다는 규칙입니다. 보관된 페이로드는 추출
+전에 SHA-256 `ed224269dc…`와 일치해야 했고, 일치했습니다. 31개 작업 중 31개가 완료되었으며, 문서
+55개, 레이블 14개, 실패 0건, 학습 없음, 임계값 탐색 없음입니다. 이후 31개 실행의 지표 전부를 저장된
+확률로부터 별도의 표준 라이브러리 구현으로 다시 계산했고, 평가기와 일치했습니다.
+
+![캘리브레이션에서 봉인 테스트로](benchmark/figures/11_final_test_calibration_to_test.png)
+
+| arm | 정밀도 | 캘리브레이션 | 테스트 | Δ |
+|---|---|---:|---:|---:|
+| **task-blind 4L N=32,768 — 이 루트의 분류기** | BF16 | 0.5719 ± 0.0092 | **0.5988 ± 0.0103** | **+0.0269** |
+| task-blind 4L N=32,768 | Q8_0 | 0.5849 (시드 41) | 0.5862 (시드 41) | +0.0013 |
+| task-blind 4L N=65,536 | BF16 | 0.5775 ± 0.0076 | 0.5832 ± 0.0047 | +0.0057 |
+| task-blind 4L N=16,384 | BF16 | 0.5787 ± 0.0021 | 0.5379 ± 0.0496 | −0.0408 |
+| task-blind 4L N=8,192 | BF16 | 0.5699 ± 0.0160 | 0.5555 ± 0.0070 | −0.0144 |
+| 과거 4L v128k | BF16 | 0.5637 ± 0.0321 | 0.5747 ± 0.0183 | +0.0110 |
+| 과거 8L v248k | BF16 | 0.5686 ± 0.0212 | 0.5718 ± 0.0213 | +0.0032 |
+| 과거 4L v248k | BF16 | 0.5602 ± 0.0391 | 0.5707 ± 0.0081 | +0.0105 |
+| 과거 24L v248k | BF16 | 0.5487 ± 0.0228 | 0.5612 ± 0.0102 | +0.0125 |
+| roberta-base 12L | BF16 | 0.5524 ± 0.0143 | 0.5883 ± 0.0109 | +0.0359 |
+| roberta-base 6L | BF16 | 0.5427 ± 0.0139 | 0.5839 ± 0.0152 | +0.0411 |
+| 전부 양성 예측기 | — | 0.4756 | **0.4866** | — |
+
+양자화는 테스트에서 macro F1 0.0013을 치렀습니다. 4절이 캘리브레이션에서 본 "측정되지 않는 정도"와
+같은 값이며, 이번에는 임계값을 맞추지 않은 분할에서 본 것입니다.
+
+**+0.0269은 이 모델에 관한 결과가 아닙니다.** 열한 개 arm 중 아홉 개가 올랐고, 열한 개의 평균
+이동은 +0.0085이며, 가장 많이 오른 것은 RoBERTa 레퍼런스 *둘*입니다. 이 테스트 분할이 캘리브레이션
+분할보다 높게 나오게 만드는 무언가는 판 전체에 적용되므로, 이 이동은 어느 arm의 성질이 아니라
+분할 쌍의 성질입니다.
+
+어느 arm도 다른 arm과 분리되지 않습니다. 이 루트의 평균 ± SD 구간은 `roberta-base`(0.5883 ±
+0.0109), `roberta-base-6l`(0.5839 ± 0.0152), 그리고 과거 4L arm 둘과 겹칩니다. 구간을 벗어나는
+경우에도 N=65,536과는 0.0006, N=16,384와는 0.0010 차이인데, 이는 그 값을 만들어 낸 설계의 산포보다
+한두 자릿수 작은 간격입니다.
+
+내려간 arm도 둘 있습니다. N=8,192는 0.0144, N=16,384는 0.0408 떨어졌고, 후자는 표에서 가장 넓은
+산포를 함께 지고 있습니다.
+
+![봉인 테스트의 시드 산포](benchmark/figures/13_final_test_seed_spread.png)
+
+모든 arm의 모든 시드가 문서 55개 위에서 폭 0.103의 띠 안에 들어옵니다. 그리고 대부분의 arm에서는
+자기 시드 세 개 사이의 거리가 이웃 arm까지의 거리보다 큽니다. N=16,384가 극단적인 사례로, 한 시드는
+0.5951인데 나머지 둘은 0.51 부근입니다.
+
+![게시된 루트의 레이블별 결과](benchmark/figures/12_final_test_per_label.png)
+
+레이블별로 보면 F1은 거의 단조롭게 support를 따라갑니다. 지지 문서가 45개인 `Loaded_Language`는
+0.918이고, 14개 이하인 여섯 레이블은 0.446에서 0.481 사이에 놓입니다. 그리고 열네 개 레이블 전부에서
+recall이 precision을 넘습니다. 0.05에서 0.3에 걸친 동결 임계값 아래에서 이 헤드는 어디서나
+과다예측합니다. 실제 배포를 위해 학습하는 헤드라면 다르게 캘리브레이션하실 것입니다.
+
+v2 분할은 이 프로토콜 아래에서 튜닝과 모델 선택 용도로 이제 은퇴했습니다. v1은 열지 않았으며,
+v1이 한 번도 쓰이지 않았다고 주장하지는 않습니다.
+
+증거: `REPORT.md` 1–89행, `RESULTS.json` 1–742행(SHA-256
+`0d2da5e2e8b250671bf3b19be0e7bce49f14fe492f9c71ce75717b0eee02dec7`), `PER_LABEL.csv`, 그리고
+동결된 사전등록 문서(SHA-256 `85ec92a0be…`). 여섯 건 모두 이 프로젝트의 리서치 스토어에 등록되어
+있고, 이 실행은 그곳에 `artifact_final_test_20260923_v1`로 기록되어 있습니다.
+
 ## 해석을 바꿀 수 있었기에 확인한 두 가지
 
 ### 호스트가 영향을 주는가?
@@ -609,8 +723,10 @@ A/B/A, 스레드 수 1/3/4, 라이브 서비스 경로도 모두 일치했습니
 
 ## 이것이 말해 주지 않는 것
 
-- **홀드아웃 평가가 없습니다.** 여기의 모든 수치는 열어 본 56개 문서 캘리브레이션 분할입니다. 이
-  저장소에는 `test.jsonl`이 없습니다.
+- **단 한 번의 테스트 평가는 홀드아웃이 아닙니다.** 위 2026-09-23 실행은 v2 재분할을 사용했고,
+  이 프로젝트는 그 분할을 전에 사용한 적이 있으며, 동결된 사전등록 문서가 그 이전 노출을 기록하고
+  있습니다. 이 카드의 다른 모든 수치는 열어 본 56개 문서 캘리브레이션 분할이고, 이 저장소에는
+  `test.jsonl`이 없습니다.
 - **평가가 작습니다.** 문서 56개, 레이블 14개, 그리고 가장 드문 레이블은 그중 6개에 나타나므로,
   macro F1의 14분의 1이 표본 여섯 개에 걸려 있습니다. 이것이 0.005–0.039 시드 SD의 가장 유력한
   원인입니다.
@@ -632,7 +748,7 @@ A/B/A, 스레드 수 1/3/4, 라이브 서비스 경로도 모두 일치했습니
 | root | 39,866항목 헤드 없는 백본 |
 | [`root_manifest.json`](root_manifest.json) | 그 어휘 규칙, 계층 맵, 해시, 왜 이 그리드 포인트인지 |
 | [`benchmark/full/`](benchmark/full/) | 61개 arm 표, 그리드, 동일 호스트 절단 |
-| [`benchmark/figures/`](benchmark/figures/) | 위의 열 개 그림 |
+| [`benchmark/figures/`](benchmark/figures/) | 위의 열일곱 개 그림 |
 | [`runtime/`](runtime/) | 전용 런타임의 완화된 두 소스와 그 자신의 README |
 | [`docker/`](docker/) | 헤드를 얹은 백본을 돌리는 컨테이너, CPU 전용 |
 | [`docker-train/`](docker-train/) | 자기 데이터로 분류기를 학습시키고, 변환하고, 서빙 |

SHA256SUMS

@@ -1,6 +1,6 @@
 77fd4710def9ec3c0f6225800e0235f15a425abd4a8b03559127fcd782612049  LICENSE
-b901b5be1af24ece943d174566a9c22e55019eef403081f9fc21672100456f50  README.md
-9d99d92cf91bcffa39d794ee8f15b7517695f8f2f67311ae081010e37c153dee  README_KO.md
+95f3404c3d41a691e4de347b7cf5b510769a23763808dc9a9a713b72340e765c  README.md
+d3aa51ff96c38b1bfb6fa2391591f7a4cf95357418c43a95f88ef9545d372608  README_KO.md
 bf4073de38c3905effd2dcce6f38894f325c8a0a79bedfb2779f0545c86b9c9f  benchmark/BENCHMARK_CARD.md
 e01bd7ac59a3212e44b77e5fcfbc7c41ec91d74c998306d2438209d6ef1a3aa7  benchmark/arm_summary.csv
 2c544c3907c4f592d724f658850b8c6b9634845cb5b50a66ebd2ea6c5d48ce0b  benchmark/compression_ladder.csv
@@ -26,6 +26,13 @@ c9c09f7f91a79d992434262ba50e0f2cb92d34058da6be7e3468a3eef60fe131  benchmark/figu
 9519e737b99e44271434e9ea45c5fac583dc0ef16ecf3bc3deb3f0c71a39618e  benchmark/figures/08_long_context_window.png
 c28ed833aef0025646d3d6c9163fbafeec84712e6941fb1c4590b9a47d55c84b  benchmark/figures/09_tokenizer_cost.png
 56a7353a71dd727077a420fd39a97b21742b8af87fabcbd4d4cdbfe09f5a0b40  benchmark/figures/10_root_on_every_metric.png
+71aa42d84f71dfc975a6906fc84e6f39ed934bf3ed3038aadbc1ad924ca2bf21  benchmark/figures/11_final_test_calibration_to_test.png
+cfb3071db4077bed7aee06c141f5f12873779a732516242580c889fb223c14fa  benchmark/figures/12_final_test_per_label.png
+4a9704a2f64b58f43b7feab7b845c0fd860f45fc3a2a2ea6ae996473c9eafff4  benchmark/figures/13_final_test_seed_spread.png
+5d2963b051a74a71c63ed0d226352b207ca501d3a356d99c32e832217fc51f05  benchmark/figures/14_field_quality_vs_cost.png
+4639fc800d53bec745bc9f1d5df37a6177f06116689798738c3677b2f05d303a  benchmark/figures/15_field_metric_spans.png
+5da006b4d603cc858c480a3cff94d9584327170933268fbac8ce8f131db87fcf  benchmark/figures/16_field_depth_effect.png
+dc621e266a694496ba30d9d5c03e89e4d6c65616f5c25df0216d012102cd3f53  benchmark/figures/17_field_resolution_floor.png
 7c238ed969ed64ac7c63084c663ecc531b618622199a77b09d28f461db11a657  benchmark/figures/README.md
 afd203646513c49140a25ef2a562a064286a4fa5efb495e6a73f83f0106e7933  benchmark/figures/cross_task/05_why_4l_edge_tradeoff.csv
 22f531ea586f9c79ac8f114345cec56e74ea5ed97aa3726f7310018666ae6312  benchmark/figures/cross_task/05_why_4l_edge_tradeoff.pdf
@@ -39,6 +46,7 @@ aac7d3400cf8dcb3ac8b6fc317c27ded5343231c93476997011a3bb56b5e4d6c  benchmark/figu
 f3e79965f14ec0fd20d11c361d25c3a3abc4c9a20b5dbce9fac0660bb12789f0  benchmark/figures/cross_task/README.md
 c3312e98b8abbbdc488e5dfde77c4110732e8d08abe9ed931bd985ad799ecad8  benchmark/figures/fig10.py
 3f6b1b82af1619afab8c753cd836040a7353a3226646b8b99fb641617e4ecec7  benchmark/figures/make_more_figures.py
+5f6b1979dcb7ff4ee897659b2f902dd3cc36339987f42f681bff7be5ae06d304  benchmark/figures/make_readout_figures.py
 078b4f34897e0a1b9bcff66ce21ba0feb28d72c86eaa8cef608f110c48f52f2e  benchmark/figures/v128k_compression.csv
 014da12a435e3404b31470e2642d180dc111eb608fb8513fd993097019698b7a  benchmark/figures/v128k_quality_efficiency.csv
 644e5b7469c0b4e0b63d5ecaa1c0f2fc8e07af8bb5d9e4d3ae5a88166de374f3  benchmark/figures/v128k_quality_storage.png

benchmark/figures/make_readout_figures.py

@@ -0,0 +1,517 @@
+"""Figures for the model card: the sealed final test, and the 65-arm field.
+
+Run from benchmark/figures/ . Figures 14-17 need only benchmark/full/*.csv, which ships here.
+
+Sources, all read-only:
+  final test   agents/sessions/2026-09-23/S-20260923-final-test-preflight-v1/{RESULTS,PER_LABEL}.csv
+               ... BASELINE.json
+  65-arm field benchmark/full/61_arm_bf16.csv + benchmark/full/taskblind_grid.csv
+
+Writes PNGs next to this file.
+"""
+import csv
+import json
+import os
+import re
+import collections
+import math
+import statistics as st
+
+import matplotlib
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+from matplotlib.lines import Line2D
+from matplotlib.patches import Patch
+from matplotlib.colors import LinearSegmentedColormap, Normalize
+from matplotlib.cm import ScalarMappable
+
+# Paths are resolved relative to this file's repository, or overridden by env vars.
+HERE = os.path.dirname(os.path.abspath(__file__))
+REPO = os.path.abspath(os.path.join(HERE, "..", ".."))
+# Final-test inputs are not redistributed in this repository; point SESS at a local copy of
+# agents/sessions/2026-09-23/S-20260923-final-test-preflight-v1/ to rebuild figures 11-13.
+SESS = os.environ.get("FINAL_TEST_DIR", os.path.join(REPO, "final_test")) + os.sep
+BENCH = os.path.join(REPO, "benchmark", "full") + os.sep
+OUT = os.path.join(HERE, "") 
+
+FLOOR_TEST = json.load(open(SESS + "BASELINE.json"))["always_positive"]["macro_f1"]
+FLOOR_CAL = 0.475627  # all-positive predictor on the opened calibration split
+
+INK, MUTED, FAINT = "#1a1a1a", "#6a6a72", "#9a9aa2"
+GRID = "#e6e6ea"
+# Series hues, checked pairwise in OKLab and under deuter/prot/trit simulation:
+# minimum normal-vision dE 18.3, minimum CVD dE 8.6, all >= 30 against white.
+ROOTC, GRIDC, QWENC, ENCC = "#1a5496", "#0d7a52", "#b05512", "#9a9aa2"
+# Status, never a series colour: always carries a dash pattern, an arrow direction,
+# a marker shape or a text label as well.
+DOWN = "#c0392b"
+ACC = QWENC
+
+
+def style(ax, xgrid=True, ygrid=False):
+    ax.grid(axis="both" if (xgrid and ygrid) else ("x" if xgrid else "y"),
+            color=GRID, lw=.8)
+    ax.set_axisbelow(True)
+    for s in ("top", "right"):
+        ax.spines[s].set_visible(False)
+    for s in ("bottom", "left"):
+        ax.spines[s].set_color("#c9c9cf")
+    ax.tick_params(length=0, colors="#4a4a52")
+
+
+def footer(fig, text):
+    fig.text(.006, .012, text, fontsize=8.6, color=MUTED, ha="left")
+
+
+# ---------------------------------------------------------------- final test
+FT_NAMES = {
+    "qwen35-taskblind-base4l-N32768-commonhead": "task-blind 4L  N=32,768",
+    "qwen35-taskblind-base4l-N65536-commonhead": "task-blind 4L  N=65,536",
+    "qwen35-taskblind-base4l-N16384-commonhead": "task-blind 4L  N=16,384",
+    "qwen35-taskblind-base4l-N8192-commonhead":  "task-blind 4L  N=8,192",
+    "qwen35-taskfree-base24l-v248k-commonhead":  "historical 24L  v248k",
+    "qwen35-taskfree-base8l-v248k-commonhead":   "historical 8L  v248k",
+    "qwen35-taskfree-base4l-v248k-commonhead":   "historical 4L  v248k",
+    "qwen35-taskfree-base4l-v128k-commonhead":   "historical 4L  v128k",
+    "roberta-base-commonhead":                   "RoBERTa-base  12L",
+    "roberta-base-6l-commonhead":                "RoBERTa-base  6L",
+}
+FT_FAM = {"published": (ROOTC, "published root (N=32,768)"),
+          "grid": (GRIDC, "other task-blind grid points"),
+          "historical": (ENCC, "historical qwen arms"),
+          "roberta": (QWENC, "RoBERTa references")}
+
+
+def ft_family(arm):
+    if arm == "qwen35-taskblind-base4l-N32768-commonhead":
+        return "published"
+    if arm.startswith("qwen35-taskblind"):
+        return "grid"
+    if arm.startswith("roberta"):
+        return "roberta"
+    return "historical"
+
+
+def ft_arms():
+    rows = list(csv.DictReader(open(SESS + "RESULTS.csv")))
+    g = collections.defaultdict(list)
+    for r in rows:
+        g[(r["arm_id"], r["precision"])].append(r)
+    arms = []
+    for (a, p), rs in g.items():
+        cal = [float(x["calibration_macro_f1"]) for x in rs]
+        tst = [float(x["test_macro_f1"]) for x in rs]
+        arms.append({"label": FT_NAMES[a] + ("   Q8_0" if p != "BF16" else ""),
+                     "fam": ft_family(a), "n": len(rs),
+                     "cal": st.mean(cal), "test": st.mean(tst),
+                     "seeds": sorted(tst),
+                     "sd": st.stdev(tst) if len(tst) > 1 else 0.0})
+    arms.sort(key=lambda d: d["test"])
+    return arms
+
+
+def fig_calibration_to_test():
+    arms = ft_arms()
+    fig, ax = plt.subplots(figsize=(10.6, 6.4))
+    fig.patch.set_facecolor("white")
+    XMIN, XMAX, XVAL = .468, .646, .624
+    for i, d in enumerate(arms):
+        c = FT_FAM[d["fam"]][0]
+        up = d["test"] >= d["cal"]
+        if d["sd"]:
+            ax.plot([d["test"] - d["sd"], d["test"] + d["sd"]], [i, i],
+                    color=c, lw=7, alpha=.20, solid_capstyle="butt", zorder=2)
+        ax.annotate("", xy=(d["test"], i), xytext=(d["cal"], i),
+                    arrowprops=dict(arrowstyle="-|>", lw=2.0,
+                                    color=c if up else DOWN, shrinkA=0, shrinkB=0, alpha=.9))
+        ax.plot([d["cal"]], [i], "o", ms=7, mfc="white", mec=c, mew=2.0, zorder=3)
+        ax.plot([d["test"]], [i], "o", ms=9.5, color=c, zorder=4,
+                mec="white", mew=1.6)
+        bold = "bold" if d["fam"] == "published" else "normal"
+        ax.text(XVAL, i, f"{d['test']:.4f}", va="center", ha="right",
+                fontsize=9.5, color=INK, fontweight=bold)
+        dl = d["test"] - d["cal"]
+        ax.text(XMAX - .002, i, f"{dl:+.4f}", va="center", ha="right", fontsize=9,
+                color=DOWN if dl < 0 else "#4a7a4a", fontweight=bold)
+    ax.axvline(FLOOR_TEST, color=DOWN, ls=(0, (5, 3)), lw=1.6, zorder=1)
+    ax.text(FLOOR_TEST + .003, -.62, f"all-positive floor {FLOOR_TEST:.4f}",
+            va="bottom", ha="left", fontsize=8.8, color=DOWN)
+    ax.text(XVAL, len(arms) - .34, "test", ha="right", va="bottom", fontsize=8.6, color=FAINT)
+    ax.text(XMAX - .002, len(arms) - .34, "Δ", ha="right", va="bottom",
+            fontsize=8.6, color=FAINT)
+    ax.set_yticks(range(len(arms)))
+    ax.set_yticklabels([d["label"] for d in arms], fontsize=9.5)
+    for t, d in zip(ax.get_yticklabels(), arms):
+        if d["fam"] == "published":
+            t.set_fontweight("bold")
+    ax.set_xlabel("macro F1  (14-label fixed denominator)", fontsize=10)
+    ax.set_xlim(XMIN, XMAX)
+    ax.set_ylim(-.95, len(arms) - .15)
+    ax.set_xticks([.48, .50, .52, .54, .56, .58, .60])
+    style(ax, xgrid=True)
+    ax.spines["left"].set_visible(False)
+    handles = [Line2D([], [], marker="o", ls="", color=c, ms=8, label=l)
+               for c, l in FT_FAM.values()]
+    handles.append(Line2D([], [], color=DOWN, lw=2.2, label="moved down on test"))
+    ax.legend(handles=handles, fontsize=8.6, frameon=False, ncol=1,
+              loc="center left", bbox_to_anchor=(.005, .30), labelspacing=.55)
+    ax.set_title("Calibration → sealed test\n"
+                 "hollow = calibration mean, filled = test mean, band = sample SD over 3 seeds",
+                 fontsize=11.5, loc="left", color=INK, pad=12)
+    footer(fig, "Nine of eleven arms moved up and the two RoBERTa references moved up the most, "
+                "so the direction belongs to the split pair, not to any model.")
+    fig.tight_layout(rect=(0, .035, 1, 1))
+    fig.savefig(OUT + "11_final_test_calibration_to_test.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+def fig_per_label():
+    pl = [r for r in csv.DictReader(open(SESS + "PER_LABEL.csv"))
+          if r["job_id"].startswith("qwen35-taskblind-base4l-N32768-commonhead")
+          and r["job_id"].endswith("BF16")]
+    by = collections.defaultdict(list)
+    for r in pl:
+        by[r["label"]].append(r)
+    num = lambda v: float(v) if v not in ("", "null", None) else None
+    pts = []
+    for lab, rs in by.items():
+        pts.append({"lab": lab.replace("_", " ").replace(",", " / "),
+                    "sup": int(rs[0]["support"]),
+                    "f1": st.mean([num(r["f1"]) for r in rs]),
+                    "p": st.mean([num(r["precision"]) for r in rs]),
+                    "r": st.mean([num(r["recall"]) for r in rs])})
+    pts.sort(key=lambda d: d["f1"])
+    ramp = LinearSegmentedColormap.from_list("sup", ["#cfe0f0", "#14406f"])
+    norm = Normalize(vmin=min(p["sup"] for p in pts), vmax=max(p["sup"] for p in pts))
+    fig, ax = plt.subplots(figsize=(10.2, 6.2))
+    fig.patch.set_facecolor("white")
+    ax.barh(range(len(pts)), [p["f1"] for p in pts], height=.64,
+            color=[ramp(norm(p["sup"])) for p in pts], zorder=3)
+    for i, p in enumerate(pts):
+        ax.text(p["f1"] - .012, i, f"{p['f1']:.3f}", va="center", ha="right",
+                fontsize=8.8, color="white", fontweight="bold", zorder=4)
+        ax.text(p["f1"] + .014, i, f"n={p['sup']}    P {p['p']:.2f}  /  R {p['r']:.2f}",
+                va="center", ha="left", fontsize=8.4, color="#5a5a62", zorder=4)
+    macro = st.mean([p["f1"] for p in pts])
+    ax.axvline(macro, color="#14406f", ls=(0, (5, 3)), lw=1.5, zorder=2)
+    ax.text(macro + .006, len(pts) - .42, f"macro F1 {macro:.4f}",
+            fontsize=9, color="#14406f", ha="left", va="bottom")
+    ax.set_yticks(range(len(pts)))
+    ax.set_yticklabels([p["lab"] for p in pts], fontsize=9)
+    ax.set_xlabel("per-label F1  (3-seed mean)", fontsize=10)
+    ax.set_xlim(0, 1.16)
+    ax.set_ylim(-.7, len(pts) - .15)
+    ax.set_xticks([0, .2, .4, .6, .8, 1.0])
+    style(ax, xgrid=True)
+    ax.spines["left"].set_visible(False)
+    ax.set_title("Published root on the sealed test, per label\n"
+                 "bar shade = support in the 55 test documents",
+                 fontsize=11.5, loc="left", color=INK, pad=12)
+    cb = fig.colorbar(ScalarMappable(norm=norm, cmap=ramp), ax=ax,
+                      orientation="horizontal", fraction=.030, pad=.12, aspect=44)
+    cb.set_label("label support (documents)", fontsize=8.6, color=MUTED)
+    cb.ax.tick_params(labelsize=8, length=0, colors=MUTED)
+    cb.outline.set_visible(False)
+    footer(fig, "Recall exceeds precision on all fourteen labels: at these thresholds the head "
+                "over-predicts. F1 tracks support almost monotonically.")
+    fig.tight_layout(rect=(0, .035, 1, 1))
+    fig.savefig(OUT + "12_final_test_per_label.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+def fig_seed_spread():
+    arms = [a for a in ft_arms() if a["n"] == 3]
+    arms.sort(key=lambda d: d["test"])
+    fig, ax = plt.subplots(figsize=(10.6, 5.8))
+    fig.patch.set_facecolor("white")
+    for i, d in enumerate(arms):
+        c = FT_FAM[d["fam"]][0]
+        ax.plot([min(d["seeds"]), max(d["seeds"])], [i, i], color=c, lw=2, alpha=.35,
+                solid_capstyle="round", zorder=2)
+        for s in d["seeds"]:
+            ax.plot([s], [i], "o", ms=7, color=c, alpha=.85, zorder=3, mec="white", mew=1.2)
+        ax.plot([d["test"]], [i], "|", ms=17, color=INK, mew=2.0, zorder=4)
+        ax.text(.638, i, f"spread {max(d['seeds']) - min(d['seeds']):.4f}",
+                va="center", ha="right", fontsize=8.8,
+                color=DOWN if max(d["seeds"]) - min(d["seeds"]) > .05 else MUTED)
+    ax.set_yticks(range(len(arms)))
+    ax.set_yticklabels([d["label"] for d in arms], fontsize=9.5)
+    ax.set_xlabel("test macro F1, one dot per seed (41 / 42 / 43); tick = mean", fontsize=10)
+    ax.set_xlim(.478, .642)
+    ax.set_ylim(-.7, len(arms) - .3)
+    style(ax, xgrid=True)
+    ax.spines["left"].set_visible(False)
+    ax.set_title("Seed spread swamps the gaps between arms\n"
+                 "every arm and seed lands inside a 0.103-wide band on 55 documents",
+                 fontsize=11.5, loc="left", color=INK, pad=12)
+    footer(fig, "N=16,384 is the extreme case: one seed at 0.5951 and two near 0.51. "
+                "Three seeds give a sample SD, not a confidence interval.")
+    fig.tight_layout(rect=(0, .038, 1, 1))
+    fig.savefig(OUT + "13_final_test_seed_spread.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+# ------------------------------------------------------------- the 65 arms
+BASE_LAYERS = {"bert-base": 12, "roberta-base": 12, "xlmr-base": 12, "deberta-v3-base": 12,
+               "mdeberta-v3": 12, "albert-base-v2": 12, "electra-base": 12, "mpnet-base": 12,
+               "modernbert-base": 22, "distilbert-base": 6, "mbert": 12,
+               "minilm-multilingual": 12}
+
+
+def encoder_family(arm_id):
+    s = arm_id.replace("-commonhead", "")
+    m = re.match(r"^(.*?)-(\d+)l$", s)
+    if m and m.group(1) in BASE_LAYERS:
+        return m.group(1), int(m.group(2))
+    if s in BASE_LAYERS:
+        return s, BASE_LAYERS[s]
+    return None, None
+
+
+def field():
+    """61 measured arms plus the 4 task-blind grid points = 65."""
+    out = []
+    for r in csv.DictReader(open(BENCH + "61_arm_bf16.csv")):
+        fam, _ = encoder_family(r["arm_id"])
+        out.append({"id": r["arm_id"], "q": float(r["macro_f1_mean"]),
+                    "sd": float(r["macro_f1_sd"]), "p50": float(r["p50_ms"]),
+                    "mib": float(r["weight_file_mib"]), "j": float(r["gpu_j_per_doc"]),
+                    "rss": float(r["peak_rss_mib"]),
+                    "floor": r["degenerate_at_floor"] == "True",
+                    "grp": "encoder" if fam else "qwen"})
+    for r in csv.DictReader(open(BENCH + "taskblind_grid.csv")):
+        n = int(r["grid_point_N"])
+        out.append({"id": f"qwen35-taskblind-base4l-N{n}", "q": float(r["macro_f1_mean"]),
+                    "sd": float(r["macro_f1_sd"]), "p50": float(r["bf16_p50_ms"]),
+                    "mib": float(r["root_mib"]), "j": float(r["bf16_gpu_j_per_doc"]),
+                    "rss": float("nan"), "floor": False,
+                    "grp": "root" if n == 32768 else "grid"})
+    return out
+
+
+GRP = {"root": (ROOTC, "this root (N=32,768)"),
+       "grid": (GRIDC, "task-blind grid"),
+       "qwen": (QWENC, "other Qwen (3.5 and 2.5)"),
+       "encoder": (ENCC, "encoder families (10)")}
+
+
+def pareto(rows, key):
+    keep = []
+    for r in rows:
+        if not any(o["q"] >= r["q"] and o[key] <= r[key] and
+                   (o["q"] > r["q"] or o[key] < r[key]) for o in rows):
+            keep.append(r)
+    return sorted(keep, key=lambda r: r[key])
+
+
+def fig_quality_vs_cost():
+    F = field()
+    fig, axes = plt.subplots(1, 2, figsize=(14.6, 6.2), sharey=True)
+    fig.patch.set_facecolor("white")
+    for ax, key, xlab, unit in (
+            (axes[0], "p50", "document p50 latency (ms, log scale)", "ms"),
+            (axes[1], "j", "GPU energy per document (J, log scale)", "J")):
+        front = pareto(F, key)
+        ax.plot([r[key] for r in front], [r["q"] for r in front],
+                color=INK, lw=1.3, alpha=.45, zorder=2, ls=(0, (4, 3)))
+        for grp in ("encoder", "qwen", "grid", "root"):
+            pts = [r for r in F if r["grp"] == grp]
+            if not pts:
+                continue
+            big = grp == "root"
+            ax.scatter([r[key] for r in pts], [r["q"] for r in pts],
+                       s=150 if big else 46, color=GRP[grp][0], zorder=5 if big else 3,
+                       edgecolor="white", linewidth=1.6 if big else .9,
+                       marker="D" if big else "o", alpha=1 if big else .92)
+        ax.axhline(FLOOR_CAL, color=DOWN, ls=(0, (5, 3)), lw=1.4, zorder=1)
+        ax.set_xscale("log")
+        ax.set_xlabel(xlab, fontsize=10)
+        style(ax, xgrid=True, ygrid=True)
+        labelled = []
+        for j, r in enumerate(front):
+            if r["grp"] in ("root", "grid"):
+                continue
+            nm = r["id"].replace("-commonhead", "").replace("qwen35-", "")
+            # skip a label that would land on top of one already placed
+            if any(abs(math.log10(r[key]) - math.log10(o)) < .17 and abs(r["q"] - q) < .012
+                   for o, q in labelled):
+                continue
+            labelled.append((r[key], r["q"]))
+            ax.annotate(nm, (r[key], r["q"]), textcoords="offset points",
+                        xytext=(9, -13 if j % 2 == 0 else 9), fontsize=7.6, color=MUTED)
+        rt = [r for r in F if r["grp"] == "root"][0]
+        ax.annotate("this root", (rt[key], rt["q"]), textcoords="offset points",
+                    xytext=(13, 9), fontsize=8.8, color=ROOTC, fontweight="bold")
+    axes[0].set_ylabel("macro F1  (3-seed mean)", fontsize=10)
+    axes[0].set_ylim(.466, .582)
+    axes[0].text(6.1, FLOOR_CAL + .0018, f"all-positive floor {FLOOR_CAL:.4f}",
+                 fontsize=8.5, color=DOWN, va="bottom")
+    handles = [Line2D([], [], marker="D" if k == "root" else "o", ls="", color=c,
+                      ms=9 if k == "root" else 7, label=l) for k, (c, l) in GRP.items()]
+    handles.append(Line2D([], [], color=INK, lw=1.3, ls=(0, (4, 3)), alpha=.45,
+                          label="Pareto frontier"))
+    handles.append(Line2D([], [], color=DOWN, lw=1.4, ls=(0, (5, 3)),
+                          label="all-positive floor"))
+    fig.legend(handles=handles, fontsize=8.8, frameon=False, ncol=6,
+               loc="upper left", bbox_to_anchor=(.006, .945), columnspacing=1.5)
+    fig.suptitle("65 arms, one frozen protocol: quality barely moves, cost moves by orders of magnitude",
+                 fontsize=12.5, x=.006, ha="left", y=.975, color=INK)
+    footer(fig, "Dashed line is the Pareto frontier. The RoBERTa family owns the cheap half of it; "
+                "the Qwen3.5 arms own the top. Every arm lives within 0.093 of a trivial all-positive predictor.")
+    fig.tight_layout(rect=(0, .038, 1, .895))
+    fig.savefig(OUT + "14_field_quality_vs_cost.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+def fig_metric_spans():
+    F = field()
+    specs = [("macro F1", [r["q"] for r in F], "", 4),
+             ("peak RSS", [r["rss"] for r in F if r["rss"] == r["rss"]], "MiB", 0),
+             ("document p50", [r["p50"] for r in F], "ms", 0),
+             ("GPU energy / doc", [r["j"] for r in F], "J", 2),
+             ("weight file", [r["mib"] for r in F], "MiB", 0)]
+    rows = []
+    for name, vals, unit, dec in specs:
+        lo, hi = min(vals), max(vals)
+        rows.append((name, lo, hi, hi / lo if lo > 0 else float("inf"), unit, dec))
+    rows.sort(key=lambda t: t[3])
+    fig, ax = plt.subplots(figsize=(11.0, 4.4))
+    fig.patch.set_facecolor("white")
+    for i, (name, lo, hi, ratio, unit, dec) in enumerate(rows):
+        c = ROOTC if ratio > 5 else QWENC
+        ax.plot([1, ratio], [i, i], color=c, lw=9, solid_capstyle="round",
+                alpha=.85, zorder=3)
+        ax.text(ratio * 1.09, i, f"{ratio:.1f}×", va="center", ha="left",
+                fontsize=11, color=c, fontweight="bold")
+        ax.text(ratio * 1.09, i - .30, f"{lo:,.{dec}f} → {hi:,.{dec}f} {unit}".strip(),
+                va="center", ha="left", fontsize=8.3, color=MUTED)
+    ax.set_yticks(range(len(rows)))
+    ax.set_yticklabels([r[0] for r in rows], fontsize=10)
+    ax.set_xscale("log")
+    ax.set_xlim(.92, 260)
+    ax.set_ylim(-.7, len(rows) - .3)
+    ax.set_xlabel("ratio of the largest arm to the smallest, across all 65 arms (log scale)",
+                  fontsize=10)
+    style(ax, xgrid=True)
+    ax.spines["left"].set_visible(False)
+    ax.set_title("What actually varies across the field\n"
+                 "the answers are nearly identical; what they cost to produce is not",
+                 fontsize=11.5, loc="left", color=INK, pad=12)
+    footer(fig, f"The best arm beats a trivial all-positive predictor ({FLOOR_CAL:.4f}) by 0.0930, "
+                "and the worst three tie it exactly. Peak RSS is dominated by the harness, not the model.")
+    fig.tight_layout(rect=(0, .05, 1, 1))
+    fig.savefig(OUT + "15_field_metric_spans.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+def fig_depth_effect():
+    rows = list(csv.DictReader(open(BENCH + "61_arm_bf16.csv")))
+    fams = collections.defaultdict(list)
+    for r in rows:
+        fam, L = encoder_family(r["arm_id"])
+        if fam:
+            fams[fam].append((L, float(r["macro_f1_mean"]),
+                              r["degenerate_at_floor"] == "True"))
+    order = sorted(fams, key=lambda k: -max(q for _, q, _ in fams[k]))
+    fig, axes = plt.subplots(3, 4, figsize=(14.2, 7.4), sharex=True, sharey=True)
+    fig.patch.set_facecolor("white")
+    shallow_wins = 0
+    for ax, fam in zip(axes.ravel(), order):
+        pts = sorted(fams[fam])
+        xs = [p[0] for p in pts]
+        ys = [p[1] for p in pts]
+        deepest = max(xs)
+        best_L = max(pts, key=lambda t: t[1])[0]
+        win = best_L != deepest
+        shallow_wins += win
+        c = ROOTC if win else QWENC
+        ax.plot(xs, ys, "-", color=c, lw=2, zorder=3, alpha=.9)
+        for L, q, fl in pts:
+            ax.plot([L], [q], "X" if fl else "o", ms=8.5 if fl else (8 if L == best_L else 6),
+                    color=DOWN if fl else c, zorder=4, mec="white", mew=1.3)
+        ax.axhline(FLOOR_CAL, color=DOWN, ls=(0, (4, 3)), lw=1.1, zorder=1)
+        ax.set_title(fam, fontsize=9.5, loc="left", color=INK, pad=4)
+        ax.text(.97, .18, "best at %dL" % best_L, transform=ax.transAxes,
+                ha="right", va="bottom", fontsize=8.2,
+                color=ROOTC if win else MUTED, fontweight="bold" if win else "normal")
+        ax.set_xticks([4, 6, 8, 12, 22])
+        ax.set_ylim(.468, .566)
+        style(ax, xgrid=False, ygrid=True)
+    for ax in axes[-1]:
+        ax.set_xlabel("layers", fontsize=9)
+    for ax in axes[:, 0]:
+        ax.set_ylabel("macro F1", fontsize=9)
+    fig.suptitle(f"Depth does not reliably buy quality: in {shallow_wins} of {len(order)} encoder "
+                 f"families the best arm is not the deepest one",
+                 fontsize=12.5, x=.006, ha="left", y=.978, color=INK)
+    handles = [Line2D([], [], color=ROOTC, lw=2.4, label="a shallower variant wins"),
+               Line2D([], [], color=QWENC, lw=2.4, label="the deepest variant wins"),
+               Line2D([], [], marker="X", ls="", color=DOWN, ms=8,
+                      label="failed to train: sits exactly on the all-positive floor")]
+    fig.legend(handles=handles, fontsize=9, frameon=False, ncol=3,
+               loc="lower left", bbox_to_anchor=(.006, .035))
+    footer(fig, "Same frozen protocol, three seeds, same split. Each family's own full-depth "
+                "checkpoint is the rightmost point.")
+    fig.tight_layout(rect=(0, .10, 1, .948))
+    fig.savefig(OUT + "16_field_depth_effect.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+def fig_resolution_floor():
+    F = sorted(field(), key=lambda r: r["q"])
+    med_sd = st.median([r["sd"] for r in F if r["sd"] > 0])
+    top = max(r["q"] for r in F)
+    band = top - med_sd
+    fig, ax = plt.subplots(figsize=(10.4, 9.4))
+    fig.patch.set_facecolor("white")
+    ax.axvspan(band, top + .004, color=ROOTC, alpha=.08, zorder=1)
+    for i, r in enumerate(F):
+        c = GRP[r["grp"]][0]
+        if r["sd"]:
+            ax.plot([r["q"] - r["sd"], r["q"] + r["sd"]], [i, i], color=c, lw=3.4,
+                    alpha=.30, solid_capstyle="butt", zorder=2)
+        if r["floor"]:
+            ax.plot([r["q"]], [i], "X", ms=7.5, color=DOWN, zorder=4, mec="white", mew=1.0)
+        else:
+            ax.plot([r["q"]], [i], "D" if r["grp"] == "root" else "o",
+                    ms=8 if r["grp"] == "root" else 5.4, color=c,
+                    zorder=4, mec="white", mew=1.0)
+    ax.axvline(FLOOR_CAL, color=DOWN, ls=(0, (5, 3)), lw=1.5, zorder=3)
+    ax.text(FLOOR_CAL - .0015, len(F) * .55, f"all-positive floor {FLOOR_CAL:.4f}",
+            rotation=90, va="center", ha="right", fontsize=8.6, color=DOWN)
+    n_in = sum(1 for r in F if r["q"] >= band)
+    ax.set_yticks(range(len(F)))
+    ax.set_yticklabels([r["id"].replace("-commonhead", "") for r in F], fontsize=6.9)
+    for t, r in zip(ax.get_yticklabels(), F):
+        if r["grp"] == "root":
+            t.set_fontweight("bold")
+            t.set_color(ROOTC)
+    ax.set_xlabel("macro F1  (3-seed mean, bar = sample SD)", fontsize=10)
+    ax.set_xlim(.466, .607)
+    ax.set_ylim(-1.2, len(F) - .3)
+    style(ax, xgrid=True)
+    ax.spines["left"].set_visible(False)
+    ax.set_title("All 65 arms on one axis\n"
+                 f"shaded band = one median seed-SD ({med_sd:.4f}) below the best arm; "
+                 f"{n_in} arms fall inside it",
+                 fontsize=11.5, loc="left", color=INK, pad=12)
+    handles = [Line2D([], [], marker="D" if k == "root" else "o", ls="", color=c,
+                      ms=8 if k == "root" else 6, label=l) for k, (c, l) in GRP.items()]
+    handles.append(Line2D([], [], marker="X", ls="", color=DOWN, ms=7.5,
+                          label="failed to train (on the floor)"))
+    ax.legend(handles=handles, fontsize=8.6, frameon=False, loc="upper left",
+              bbox_to_anchor=(.012, .995), labelspacing=.5)
+    footer(fig, "Three arms — mdeberta-v3-4l, deberta-v3-base-6l, deberta-v3-base-8l — score exactly "
+                "the floor. They are reported rather than dropped.")
+    fig.tight_layout(rect=(0, .026, 1, 1))
+    fig.savefig(OUT + "17_field_resolution_floor.png", dpi=170, facecolor="white")
+    plt.close(fig)
+
+
+if __name__ == "__main__":
+    os.makedirs(OUT, exist_ok=True)
+    for fn in (fig_calibration_to_test, fig_per_label, fig_seed_spread,
+               fig_quality_vs_cost, fig_metric_spans, fig_depth_effect,
+               fig_resolution_floor):
+        fn()
+        print("ok", fn.__name__)