evijit HF Staff commited on
Commit
350e866
·
1 Parent(s): d52d9e0

Consolidate hierarchy terminology + handle v2 hierarchy shape

Browse files

Two threads in one change.

(1) v2 backend snapshot ships families[].composites[].benchmarks[]
instead of families[].leaves[]. The family-table only walked .leaves,
so every aggregator family rendered as a single non-expandable row
and clicking it navigated to one arbitrary child (e.g. clicking BFCL
landed on bfcl-live).

(2) The codebase mixed "Suite/Subtask" labels with "Composite/Slice"
data fields. Consolidate to one term per level so the UI and the
schema agree.

Hierarchy: 5 levels. Family -> Composite -> Benchmark -> Slice ->
Metric. "Suite" and "Subtask" are gone everywhere a human reads them.
Backend-shipped JSON keys (subtask_key, subtask_name,
metric.scope === "subtask", composite_benchmark_*) are kept since
they are wire contracts.

Family-table fix
- collectLeafEntries now treats nested benchmarks as the primary
source: walks family.standalone_benchmarks, family.benchmarks, and
family.composites[].benchmarks[] before falling back to the legacy
family.leaves[]. Each nested benchmark's summary_eval_ids[0] becomes
the navigation target, with `${family_key}_${benchmark_key}` as a
fallback when summary_eval_ids is missing.
- Verified end-to-end against the v2 snapshot
(j-chim/temp_evalcard_backend/warehouse/latest): bfcl now renders
"4 benchmarks" with the chevron, and clicking expands to BFCL Live /
BFCL Multi-turn / BFCL Non-live / BFCL Web Search.

Domain extraction in /evals
- The familyDomains and domainCounts memos walk the same nested
benchmarks before falling back to leaves, so the "Filter by
domain" pill list populates correctly under v2.

Hierarchy view
- Home page hierarchy strip collapses Split + Subtask into a single
"Slice" tier (5 levels instead of 6). Lede, kicker, OG image,
layout metadata description all updated to "five-level" to match.

Terminology rename (UI labels + local code identifiers)
- "Suite" / "Suites" -> "Composite" / "Composites"
- "Subtask" / "subtask" -> "Slice" / "slice"
- Renamed local identifiers in benchmark-detail.tsx and
eval-detail.tsx: SuiteGroup -> CompositeGroup,
getSuiteBadgeMeta -> getCompositeBadgeMeta,
normalizeSuiteKey -> normalizeCompositeKey,
groupBySuite -> groupByComposite,
getSuiteKey/Name/DisplayName -> getComposite*,
suiteGroups -> compositeGroups,
categorySuiteSections -> categoryCompositeSections,
suiteBenchmarkScoreRanges -> compositeBenchmarkScoreRanges,
activeSubtaskTab -> activeSliceTab,
subtaskMetricCounts -> sliceMetricCounts,
singleMetricSubtaskTabs -> singleMetricSliceTabs,
hasSubtaskTabs -> hasSliceTabs,
showSubtaskTopline -> showSliceTopline,
subtaskLabel/Count/DrivenBenchmarkCount -> slice* equivalents,
BenchmarkVariant discriminant strings "subtask" / "setup+subtask"
-> "slice" / "setup+slice",
subtasksOpen -> slicesOpen.
- Reworded prose in app/page.tsx, app/about/page.tsx,
app/evals/page.tsx, app/evals/[id]/page.tsx, app/layout.tsx,
app/opengraph-image.tsx, components/eval-card.tsx,
components/eval-detail.tsx, components/benchmark-evaluation-card.tsx,
components/policy-overview.tsx, components/model-compare-dialog.tsx.

Backend wire fields kept untouched (would require coordinated backend
change): metric.scope === "subtask", metric.subtask_key,
metric.subtask_name, summary.subtasks, slice.subtask_*,
composite_benchmark_name, composite_benchmark_key. A spec for the v2
maintainer covering the remaining family-grouping work
(gaia / gpqa / caparena variants) was sent separately.

app/about/page.tsx CHANGED
@@ -65,8 +65,8 @@ export default function AboutPage() {
65
  "Risk-domain annotations on benchmarks (capability, robustness, safety, agentic risk, fairness) so policy readers can locate which deployment-relevant property a number speaks to.",
66
  ],
67
  [
68
- "Six-level hierarchy",
69
- "Family → Suite → Single benchmark → SplitSubtask → Metric. Every score resolves to an explicit path, so aggregate claims drill down to the evidence supporting them.",
70
  ],
71
  ].map(([h, p]) => (
72
  <div
@@ -146,7 +146,7 @@ export default function AboutPage() {
146
  ],
147
  [
148
  "Comparability",
149
- "Can two scores under the same benchmark be put side-by-side? Tracks split, subtask, metric variant, and unit harmonisation; flags rows that cannot be ranked together.",
150
  ],
151
  ].map(([h, p], i) => (
152
  <li
@@ -183,7 +183,7 @@ export default function AboutPage() {
183
  {[
184
  [
185
  "Canonicalisation",
186
- "Heterogeneous score reports — papers, model cards, leaderboards, blog posts — are normalised to (model, benchmark, split, subtask, metric, value, source) tuples. Model name aliases and benchmark version aliases are resolved against a curated mapping.",
187
  ],
188
  [
189
  "Source attribution",
 
65
  "Risk-domain annotations on benchmarks (capability, robustness, safety, agentic risk, fairness) so policy readers can locate which deployment-relevant property a number speaks to.",
66
  ],
67
  [
68
+ "Five-level hierarchy",
69
+ "Family → Composite → Single benchmark → Slice → Metric. Every score resolves to an explicit path, so aggregate claims drill down to the evidence supporting them.",
70
  ],
71
  ].map(([h, p]) => (
72
  <div
 
146
  ],
147
  [
148
  "Comparability",
149
+ "Can two scores under the same benchmark be put side-by-side? Tracks slice, metric variant, and unit harmonisation; flags rows that cannot be ranked together.",
150
  ],
151
  ].map(([h, p], i) => (
152
  <li
 
183
  {[
184
  [
185
  "Canonicalisation",
186
+ "Heterogeneous score reports — papers, model cards, leaderboards, blog posts — are normalised to (model, benchmark, slice, metric, value, source) tuples. Model name aliases and benchmark version aliases are resolved against a curated mapping.",
187
  ],
188
  [
189
  "Source attribution",
app/evals/[id]/page.tsx CHANGED
@@ -135,7 +135,7 @@ export default function EvalDetailPage() {
135
  }
136
 
137
  // ---------------------------------------------------------------------------
138
- // Composite (suite) view — paper §3.2 "composite reporting unit"
139
  // Surfaces sub-benchmarks as a hairline grid and a per-model × per-metric
140
  // matrix table. Both modes (research / policy) share the same chrome; the
141
  // policy-note panel changes per benchmark, surfaced from the sub-summary card.
@@ -169,8 +169,8 @@ function CompositeEvalView({
169
  const audience = card?.purpose_and_intended_users?.audience
170
  const audienceText = Array.isArray(audience) ? audience.join("; ") : audience
171
  const lede = isPolicy
172
- ? overview || goal || `Suite aggregating ${subBenchmarkCount} component benchmarks across ${summary.models_count.toLocaleString()} models.`
173
- : goal || overview || `Suite aggregating ${subBenchmarkCount} component benchmarks across ${summary.models_count.toLocaleString()} models.`
174
 
175
  return (
176
  <div className="space-y-10">
@@ -281,7 +281,7 @@ function CompositeEvalView({
281
  style={{ color: "var(--fg-muted)", maxWidth: 720 }}
282
  >
283
  {tab === "metrics"
284
- ? "Each card is one component benchmark inside this suite. Click a card to inspect its leaderboard, sub-tasks and benchmark card."
285
  : "Per-model scores across every component metric. Each column is a separately reported measure — distinct measures stay separate instead of collapsing into one number."}
286
  </p>
287
 
 
135
  }
136
 
137
  // ---------------------------------------------------------------------------
138
+ // Composite view — paper §3.2 "composite reporting unit"
139
  // Surfaces sub-benchmarks as a hairline grid and a per-model × per-metric
140
  // matrix table. Both modes (research / policy) share the same chrome; the
141
  // policy-note panel changes per benchmark, surfaced from the sub-summary card.
 
169
  const audience = card?.purpose_and_intended_users?.audience
170
  const audienceText = Array.isArray(audience) ? audience.join("; ") : audience
171
  const lede = isPolicy
172
+ ? overview || goal || `Composite aggregating ${subBenchmarkCount} component benchmarks across ${summary.models_count.toLocaleString()} models.`
173
+ : goal || overview || `Composite aggregating ${subBenchmarkCount} component benchmarks across ${summary.models_count.toLocaleString()} models.`
174
 
175
  return (
176
  <div className="space-y-10">
 
281
  style={{ color: "var(--fg-muted)", maxWidth: 720 }}
282
  >
283
  {tab === "metrics"
284
+ ? "Each card is one component benchmark inside this composite. Click a card to inspect its leaderboard, slices and benchmark card."
285
  : "Per-model scores across every component metric. Each column is a separately reported measure — distinct measures stay separate instead of collapsing into one number."}
286
  </p>
287
 
app/evals/page.tsx CHANGED
@@ -98,11 +98,31 @@ export default function EvalsPage() {
98
  }
99
  for (const fam of families) {
100
  const seen = new Set<string>()
 
101
  for (const d of lookupDomains(fam.key)) seen.add(d.trim().toLowerCase())
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
102
  for (const leaf of fam.leaves ?? []) {
103
  for (const d of leaf.tags?.domains ?? []) seen.add(d.trim().toLowerCase())
104
  for (const d of lookupDomains(leaf.key)) seen.add(d.trim().toLowerCase())
105
  }
 
 
106
  for (const id of fam.eval_summary_ids ?? []) {
107
  for (const d of lookupDomains(id)) seen.add(d.trim().toLowerCase())
108
  }
@@ -126,6 +146,14 @@ export default function EvalsPage() {
126
  for (const d of card?.benchmark_details?.domains ?? []) recordLabel(d)
127
  }
128
  for (const fam of families) {
 
 
 
 
 
 
 
 
129
  for (const leaf of fam.leaves ?? []) {
130
  for (const d of leaf.tags?.domains ?? []) recordLabel(d)
131
  }
@@ -244,7 +272,7 @@ export default function EvalsPage() {
244
  </span>
245
  </div>
246
  <div className="ec-page-meta-item">
247
- <span className="ec-page-meta-item-l">Suites</span>
248
  <span className="ec-page-meta-item-v">
249
  {stats.composite_count.toLocaleString()}
250
  </span>
 
98
  }
99
  for (const fam of families) {
100
  const seen = new Set<string>()
101
+ // Family-level fallback first (cards keyed by family slug).
102
  for (const d of lookupDomains(fam.key)) seen.add(d.trim().toLowerCase())
103
+
104
+ // v2 primary path: walk every nested benchmark across composites,
105
+ // standalone benchmarks, and any family-level benchmarks array.
106
+ const nestedBenchmarks = [
107
+ ...(fam.standalone_benchmarks ?? []),
108
+ ...(fam.benchmarks ?? []),
109
+ ...(fam.composites ?? []).flatMap((c) => c.benchmarks ?? []),
110
+ ]
111
+ for (const benchmark of nestedBenchmarks) {
112
+ for (const d of benchmark.tags?.domains ?? []) seen.add(d.trim().toLowerCase())
113
+ for (const d of lookupDomains(benchmark.key)) seen.add(d.trim().toLowerCase())
114
+ for (const id of benchmark.summary_eval_ids ?? []) {
115
+ for (const d of lookupDomains(id)) seen.add(d.trim().toLowerCase())
116
+ }
117
+ }
118
+
119
+ // Legacy fallback: per-leaf tags + cards keyed by leaf slug.
120
  for (const leaf of fam.leaves ?? []) {
121
  for (const d of leaf.tags?.domains ?? []) seen.add(d.trim().toLowerCase())
122
  for (const d of lookupDomains(leaf.key)) seen.add(d.trim().toLowerCase())
123
  }
124
+
125
+ // Family-level eval_summary_ids cover both shapes.
126
  for (const id of fam.eval_summary_ids ?? []) {
127
  for (const d of lookupDomains(id)) seen.add(d.trim().toLowerCase())
128
  }
 
146
  for (const d of card?.benchmark_details?.domains ?? []) recordLabel(d)
147
  }
148
  for (const fam of families) {
149
+ const nestedBenchmarks = [
150
+ ...(fam.standalone_benchmarks ?? []),
151
+ ...(fam.benchmarks ?? []),
152
+ ...(fam.composites ?? []).flatMap((c) => c.benchmarks ?? []),
153
+ ]
154
+ for (const benchmark of nestedBenchmarks) {
155
+ for (const d of benchmark.tags?.domains ?? []) recordLabel(d)
156
+ }
157
  for (const leaf of fam.leaves ?? []) {
158
  for (const d of leaf.tags?.domains ?? []) recordLabel(d)
159
  }
 
272
  </span>
273
  </div>
274
  <div className="ec-page-meta-item">
275
+ <span className="ec-page-meta-item-l">Composites</span>
276
  <span className="ec-page-meta-item-v">
277
  {stats.composite_count.toLocaleString()}
278
  </span>
app/layout.tsx CHANGED
@@ -9,7 +9,7 @@ const SITE_URL = "https://evalcards.evalevalai.com"
9
  const SITE_NAME = "Eval Cards"
10
  const SITE_TITLE = "Eval Cards — a reporting layer for AI evaluations"
11
  const SITE_DESCRIPTION =
12
- "A public registry of reported model–benchmark results, organised under a six-level rollout hierarchy and four interpretive signals: reproducibility, completeness, provenance, and comparability."
13
 
14
  export const metadata: Metadata = {
15
  metadataBase: new URL(SITE_URL),
 
9
  const SITE_NAME = "Eval Cards"
10
  const SITE_TITLE = "Eval Cards — a reporting layer for AI evaluations"
11
  const SITE_DESCRIPTION =
12
+ "A public registry of reported model–benchmark results, organised under a five-level rollout hierarchy and four interpretive signals: reproducibility, completeness, provenance, and comparability."
13
 
14
  export const metadata: Metadata = {
15
  metadataBase: new URL(SITE_URL),
app/opengraph-image.tsx CHANGED
@@ -110,7 +110,7 @@ export default function OpenGraphImage() {
110
  maxWidth: "920px",
111
  }}
112
  >
113
- A registry of reported model–benchmark results, organised under a six-level
114
  rollout hierarchy and four interpretive signals.
115
  </div>
116
  </div>
 
110
  maxWidth: "920px",
111
  }}
112
  >
113
+ A registry of reported model–benchmark results, organised under a five-level
114
  rollout hierarchy and four interpretive signals.
115
  </div>
116
  </div>
app/page.tsx CHANGED
@@ -106,7 +106,7 @@ export default async function HomePage() {
106
  </h1>
107
  <p className="home-hero-lede">
108
  <strong>Eval Cards</strong> is a registry of reported model–benchmark results,
109
- organised under a six-level rollout hierarchy and four interpretive signals
110
  computed over the joined record.
111
  </p>
112
  <div className="flex flex-wrap items-center gap-3">
@@ -164,9 +164,9 @@ export default async function HomePage() {
164
  </div>
165
  </section>
166
 
167
- {/* SIX-LEVEL HIERARCHY STRIP -------------------------------------- */}
168
  <section className="hierarchy-strip">
169
- <div className="kicker">Six-level rollout hierarchy</div>
170
  <div className="hierarchy-row">
171
  {[
172
  {
@@ -175,7 +175,7 @@ export default async function HomePage() {
175
  ex: "SWE-bench family, MMLU family",
176
  },
177
  {
178
- name: "Suite",
179
  count: formatNumber(compositeCount),
180
  ex: "Open LLM Leaderboard v2, HELM Instruct",
181
  },
@@ -185,14 +185,9 @@ export default async function HomePage() {
185
  ex: "GSM8K, IFEval, MMLU-Pro",
186
  },
187
  {
188
- name: "Split",
189
  count: formatNumber(sliceCount),
190
- ex: "algebra (within MATH), Python (Multi-SWE-Bench)",
191
- },
192
- {
193
- name: "Subtask",
194
- count: "—",
195
- ex: "level-5, multi-turn",
196
  },
197
  {
198
  name: "Metric",
 
106
  </h1>
107
  <p className="home-hero-lede">
108
  <strong>Eval Cards</strong> is a registry of reported model–benchmark results,
109
+ organised under a five-level rollout hierarchy and four interpretive signals
110
  computed over the joined record.
111
  </p>
112
  <div className="flex flex-wrap items-center gap-3">
 
164
  </div>
165
  </section>
166
 
167
+ {/* FIVE-LEVEL HIERARCHY STRIP -------------------------------------- */}
168
  <section className="hierarchy-strip">
169
+ <div className="kicker">Five-level rollout hierarchy</div>
170
  <div className="hierarchy-row">
171
  {[
172
  {
 
175
  ex: "SWE-bench family, MMLU family",
176
  },
177
  {
178
+ name: "Composite",
179
  count: formatNumber(compositeCount),
180
  ex: "Open LLM Leaderboard v2, HELM Instruct",
181
  },
 
185
  ex: "GSM8K, IFEval, MMLU-Pro",
186
  },
187
  {
188
+ name: "Slice",
189
  count: formatNumber(sliceCount),
190
+ ex: "algebra (within MATH), level-5, multi-turn",
 
 
 
 
 
191
  },
192
  {
193
  name: "Metric",
components/benchmark-detail.tsx CHANGED
@@ -62,10 +62,10 @@ interface BenchmarkVariant {
62
  evaluation: BenchmarkEvaluation
63
  result: EvaluationResult
64
  label: string
65
- variantType: "setup" | "subtask" | "setup+subtask" | "default"
66
  metricLabel: string
67
  setupLabel: string | null
68
- subtaskLabel: string | null
69
  displayScore: string
70
  normalizedScore: number
71
  rankPosition: number | null
@@ -92,9 +92,9 @@ interface BenchmarkGroup {
92
  variants: BenchmarkVariant[]
93
  }
94
 
95
- interface SuiteGroup {
96
- suiteKey: string
97
- suiteName: string
98
  benchmarks: BenchmarkGroup[]
99
  avgRawScore: number
100
  avgNormalizedScore: number
@@ -269,77 +269,77 @@ function getSourceTypeDisplayName(value: string | null | undefined) {
269
  return normalizeDisplayLabel(value?.replace(/_/g, " ")) || "Unknown"
270
  }
271
 
272
- function normalizeSuiteKey(key: string): string {
273
  const k = key.toLowerCase().replace(/[-.\s]+/g, "_").replace(/^_+|_+$/g, "")
274
  if (/^fibble\d*_arena$/.test(k)) return "fibble_arena"
275
  if (/^arc_agi_v\d+/.test(k)) return "arc_agi"
276
  return k
277
  }
278
 
279
- function doesLabelMatchSuiteKey(label: string | null | undefined, suiteKey: string) {
280
  if (!label) {
281
  return false
282
  }
283
 
284
- return normalizeSuiteKey(normalizeDisplayKey(label)) === normalizeSuiteKey(suiteKey)
285
  }
286
 
287
- function getSuiteKey(group: BenchmarkGroup): string {
288
  const evaluation = group.variants[0]?.evaluation
289
  const backendSuiteKey =
290
  evaluation?.benchmark_parent_key ||
291
  evaluation?.benchmark_family_key ||
292
  evaluation?.benchmark
293
 
294
- return normalizeSuiteKey(backendSuiteKey ?? group.key)
295
  }
296
 
297
- function getSuiteDisplayName(key: string): string {
298
- const normalizedKey = normalizeSuiteKey(key)
299
  return SUITE_DISPLAY_NAMES[normalizedKey] ?? normalizeDisplayLabel(key)
300
  }
301
 
302
- function getSuiteName(group: BenchmarkGroup, suiteKey: string): string {
303
  const evaluation = group.variants[0]?.evaluation
304
  const benchmarkCardName = group.benchmarkCard?.benchmark_details?.name
305
  const backendParentName = evaluation?.benchmark_parent_name
306
  const backendFamilyName = evaluation?.benchmark_family_name
307
 
308
- if (doesLabelMatchSuiteKey(backendParentName, suiteKey)) {
309
  return normalizeDisplayLabel(backendParentName)
310
  }
311
 
312
- if (doesLabelMatchSuiteKey(backendFamilyName, suiteKey)) {
313
  return normalizeDisplayLabel(backendFamilyName)
314
  }
315
 
316
- if (doesLabelMatchSuiteKey(benchmarkCardName, suiteKey)) {
317
  return normalizeDisplayLabel(benchmarkCardName)
318
  }
319
 
320
- return getSuiteDisplayName(suiteKey)
321
  }
322
 
323
- function groupBySuite(
324
  groups: BenchmarkGroup[],
325
  modelIds: string[],
326
  peerRanks: PeerRanksMap
327
- ): SuiteGroup[] {
328
- const suites = new Map<string, BenchmarkGroup[]>()
329
  for (const group of groups) {
330
- const key = getSuiteKey(group)
331
- const existing = suites.get(key) ?? []
332
  existing.push(group)
333
- suites.set(key, existing)
334
  }
335
 
336
- return Array.from(suites.entries()).map(([suiteKey, benchmarks]) => {
337
  const scores = benchmarks.map(b => b.avgNormalizedScore).filter(Number.isFinite)
338
  const avgScore = scores.length > 0 ? scores.reduce((a, b) => a + b, 0) / scores.length : 0
339
  const rawScores = benchmarks.map((benchmark) => benchmark.avgRawScore).filter(Number.isFinite)
340
  const avgRawScore = rawScores.length > 0 ? rawScores.reduce((a, b) => a + b, 0) / rawScores.length : 0
341
 
342
- // Find best rank across all benchmarks in suite
343
  let bestRank: { position: number; total: number } | null = null
344
  for (const b of benchmarks) {
345
  const rank = getGroupPeerRank(b, modelIds, peerRanks)
@@ -350,8 +350,8 @@ function groupBySuite(
350
  }
351
 
352
  return {
353
- suiteKey,
354
- suiteName: benchmarks[0] ? getSuiteName(benchmarks[0], suiteKey) : getSuiteDisplayName(suiteKey),
355
  benchmarks,
356
  avgRawScore,
357
  avgNormalizedScore: avgScore,
@@ -359,7 +359,7 @@ function groupBySuite(
359
  bestRank,
360
  }
361
  }).sort((a, b) => {
362
- // Sort by best peer rank ratio (lower = better); unranked suites go to the bottom
363
  const aRatio = a.bestRank ? a.bestRank.position / (a.bestRank.total || a.bestRank.position) : Infinity
364
  const bRatio = b.bestRank ? b.bestRank.position / (b.bestRank.total || b.bestRank.position) : Infinity
365
  if (aRatio !== bRatio) return aRatio - bRatio
@@ -466,25 +466,25 @@ function getMetricDisplayLabel(result: EvaluationResult) {
466
  function getVariantDescriptor(
467
  evaluation: BenchmarkEvaluation,
468
  result: EvaluationResult
469
- ): Pick<BenchmarkVariant, "label" | "variantType" | "metricLabel" | "setupLabel" | "subtaskLabel"> {
470
  const evaluationVariantRaw = getEvaluationVariantLabel(evaluation)
471
  const evaluationVariant = evaluationVariantRaw ? formatSetupDisplayLabel(evaluationVariantRaw) : null
472
  const metricLabel = getMetricDisplayLabel(result)
473
  const metricKey = normalizeDisplayKey(metricLabel)
474
  const metricIsAmbiguous = AMBIGUOUS_GROUP_LABELS.has(metricKey)
475
- const subtaskLabel = evaluation.slice_name ? normalizeDisplayLabel(evaluation.slice_name) : null
476
  const setupLabel = evaluationVariant ? formatSetupDisplayLabel(evaluationVariant) : null
477
- const baseLabel = subtaskLabel
478
- ? (metricIsAmbiguous ? subtaskLabel : `${subtaskLabel} · ${metricLabel}`)
479
  : metricLabel
480
 
481
- if (setupLabel && subtaskLabel) {
482
  return {
483
  label: `${setupLabel} · ${baseLabel}`,
484
- variantType: "setup+subtask",
485
  metricLabel,
486
  setupLabel,
487
- subtaskLabel,
488
  }
489
  }
490
 
@@ -494,17 +494,17 @@ function getVariantDescriptor(
494
  variantType: "setup",
495
  metricLabel,
496
  setupLabel,
497
- subtaskLabel: null,
498
  }
499
  }
500
 
501
- if (subtaskLabel || !metricIsAmbiguous) {
502
  return {
503
  label: baseLabel,
504
- variantType: subtaskLabel ? "subtask" : "default",
505
  metricLabel,
506
  setupLabel: null,
507
- subtaskLabel: subtaskLabel ?? null,
508
  }
509
  }
510
 
@@ -513,7 +513,7 @@ function getVariantDescriptor(
513
  variantType: "default",
514
  metricLabel,
515
  setupLabel: null,
516
- subtaskLabel: null,
517
  }
518
  }
519
 
@@ -603,7 +603,7 @@ function getTableConfigLabel(row: VariantRowData) {
603
  return row.variant.setupLabel
604
  }
605
 
606
- if (row.variant.variantType === "subtask") {
607
  return "Default setup"
608
  }
609
 
@@ -865,9 +865,9 @@ function getVariantTypeTone(variantType: BenchmarkVariant["variantType"]) {
865
  switch (variantType) {
866
  case "setup":
867
  return "bg-sky-100 text-sky-800 dark:bg-sky-950/50 dark:text-sky-300"
868
- case "subtask":
869
  return "bg-violet-100 text-violet-800 dark:bg-violet-950/50 dark:text-violet-300"
870
- case "setup+subtask":
871
  return "bg-amber-100 text-amber-800 dark:bg-amber-950/50 dark:text-amber-300"
872
  default:
873
  return "bg-muted text-muted-foreground"
@@ -878,10 +878,10 @@ function getVariantTypeLabel(variantType: BenchmarkVariant["variantType"]) {
878
  switch (variantType) {
879
  case "setup":
880
  return "Setup change"
881
- case "subtask":
882
- return "Benchmark subtask"
883
- case "setup+subtask":
884
- return "Setup + subtask"
885
  default:
886
  return "Single run"
887
  }
@@ -911,8 +911,8 @@ function formatSetupDisplayLabel(setupLabel: string | null) {
911
  }
912
 
913
  function getVariantPrimaryLabel(variant: BenchmarkVariant, groupTitle: string) {
914
- if (variant.subtaskLabel) {
915
- return variant.subtaskLabel
916
  }
917
 
918
  if (variant.metricLabel) {
@@ -929,41 +929,41 @@ function getVariantPrimaryLabel(variant: BenchmarkVariant, groupTitle: string) {
929
  return variant.label
930
  }
931
 
932
- function getGroupSubtaskLabels(group: BenchmarkGroup) {
933
  return Array.from(
934
  new Set(
935
  group.variants
936
- .map((variant) => variant.subtaskLabel?.trim())
937
  .filter((label): label is string => Boolean(label))
938
  )
939
  )
940
  }
941
 
942
- function getGroupSubtaskCount(group: BenchmarkGroup) {
943
- return getGroupSubtaskLabels(group).length
944
  }
945
 
946
  function getBenchmarkGroupHeading(group: BenchmarkGroup) {
947
  return group.canonicalTitle
948
  }
949
 
950
- function getSuiteBadgeMeta(suite: SuiteGroup) {
951
- if (suite.benchmarks.length > 1) {
952
  return {
953
- count: suite.benchmarks.length,
954
- label: `sub-benchmark${suite.benchmarks.length === 1 ? "" : "s"}`,
955
  className:
956
  "border-sky-200/80 bg-sky-50/70 text-sky-700 dark:border-sky-900/60 dark:bg-sky-950/30 dark:text-sky-300",
957
  }
958
  }
959
 
960
- const singleGroup = suite.benchmarks[0]
961
  if (!singleGroup) {
962
  return null
963
  }
964
 
965
- const suiteMatchesBenchmark = normalizeSuiteKey(suite.suiteName) === normalizeSuiteKey(singleGroup.title)
966
- if (!suiteMatchesBenchmark) {
967
  return {
968
  count: 1,
969
  label: "sub-benchmark",
@@ -972,11 +972,11 @@ function getSuiteBadgeMeta(suite: SuiteGroup) {
972
  }
973
  }
974
 
975
- const subtaskCount = getGroupSubtaskCount(singleGroup)
976
- if (subtaskCount > 0) {
977
  return {
978
- count: subtaskCount,
979
- label: `subtask${subtaskCount === 1 ? "" : "s"}`,
980
  className:
981
  "border-emerald-200/70 bg-emerald-50 text-emerald-700 dark:border-emerald-900/60 dark:bg-emerald-950/30 dark:text-emerald-300",
982
  }
@@ -1405,7 +1405,7 @@ function getVariantDedupKey(variant: BenchmarkVariant) {
1405
  metricLabel: variant.metricLabel,
1406
  variantType: variant.variantType,
1407
  setupLabel: variant.setupLabel,
1408
- subtaskLabel: variant.subtaskLabel,
1409
  displayScore: variant.displayScore,
1410
  sourceOrganization: getOrganizationDisplayName(variant.evaluation.source_metadata.source_organization_name),
1411
  sourceName: normalizeDisplayLabel(variant.evaluation.source_metadata.source_name ?? ""),
@@ -1459,7 +1459,7 @@ function buildBenchmarkGroups(
1459
  variantType: descriptor.variantType,
1460
  metricLabel: descriptor.metricLabel,
1461
  setupLabel: descriptor.setupLabel,
1462
- subtaskLabel: descriptor.subtaskLabel,
1463
  displayScore,
1464
  normalizedScore,
1465
  rankPosition,
@@ -1570,10 +1570,10 @@ function buildBenchmarkGroups(
1570
 
1571
  group.variants = Array.from(dedupedVariants.values())
1572
  group.variants.sort((a, b) => {
1573
- const aIsSubtask = Boolean(a.evaluation.slice_key)
1574
- const bIsSubtask = Boolean(b.evaluation.slice_key)
1575
- if (aIsSubtask !== bIsSubtask) {
1576
- return aIsSubtask ? 1 : -1
1577
  }
1578
 
1579
  const aPrimaryLabel = getVariantPrimaryLabel(a, group.title)
@@ -2043,9 +2043,9 @@ export function BenchmarkDetail({
2043
  }))
2044
  }, [filteredBenchmarkGroups, comparisonIndex, summary.categories_covered])
2045
 
2046
- const suiteGroups = useMemo(() => {
2047
- const groups = groupBySuite(filteredBenchmarkGroups, modelIds, peerRanks)
2048
- // Re-sort suites by max relevance of their benchmarks
2049
  return groups.sort((a, b) => {
2050
  const aMax = Math.max(...a.benchmarks.map(getRelevanceScore))
2051
  const bMax = Math.max(...b.benchmarks.map(getRelevanceScore))
@@ -2053,48 +2053,48 @@ export function BenchmarkDetail({
2053
  })
2054
  }, [filteredBenchmarkGroups, modelIds, peerRanks, getRelevanceScore])
2055
 
2056
- const categorySuiteSections = useMemo(
2057
  () =>
2058
  groupedFilteredBenchmarkGroups
2059
  .map(({ category, groups }) => ({
2060
  category,
2061
- suites: groupBySuite(groups, modelIds, peerRanks).sort((a, b) => {
2062
  const aMax = Math.max(...a.benchmarks.map(getRelevanceScore))
2063
  const bMax = Math.max(...b.benchmarks.map(getRelevanceScore))
2064
  return bMax - aMax
2065
  }),
2066
  }))
2067
- .filter((section) => section.suites.length > 0),
2068
  [groupedFilteredBenchmarkGroups, modelIds, peerRanks, getRelevanceScore]
2069
  )
2070
 
2071
  const categoryScoreRanges = useMemo(() => {
2072
  const ranges = new Map<CategoryType, ScoreRange>()
2073
 
2074
- for (const section of categorySuiteSections) {
2075
  ranges.set(
2076
  section.category,
2077
- getScoreRange(section.suites.map((suite) => suite.avgNormalizedScore))
2078
  )
2079
  }
2080
 
2081
  return ranges
2082
- }, [categorySuiteSections])
2083
 
2084
- const suiteBenchmarkScoreRanges = useMemo(() => {
2085
  const ranges = new Map<string, ScoreRange>()
2086
 
2087
- for (const section of categorySuiteSections) {
2088
- for (const suite of section.suites) {
2089
  ranges.set(
2090
- suite.suiteKey,
2091
- getScoreRange(suite.benchmarks.map((group) => group.avgNormalizedScore))
2092
  )
2093
  }
2094
  }
2095
 
2096
  return ranges
2097
- }, [categorySuiteSections])
2098
 
2099
  const benchmarkGroupLookup = useMemo(
2100
  () => new Map(benchmarkGroups.map((group) => [group.key, group] as const)),
@@ -2104,11 +2104,11 @@ export function BenchmarkDetail({
2104
  ? benchmarkGroupLookup.get(activeBenchmarkGroupKey) ?? null
2105
  : null
2106
 
2107
- const toggleSuite = (suiteKey: string) => {
2108
  setExpandedSuites((prev) => {
2109
  const next = new Set(prev)
2110
- if (next.has(suiteKey)) next.delete(suiteKey)
2111
- else next.add(suiteKey)
2112
  return next
2113
  })
2114
  }
@@ -2150,10 +2150,10 @@ export function BenchmarkDetail({
2150
  )
2151
  const repeatedBenchmarkCount = overviewBenchmarkGroups.filter((group) => group.variants.length > 1).length
2152
  const setupDrivenBenchmarkCount = overviewBenchmarkGroups.filter((group) =>
2153
- group.variants.some((variant) => variant.variantType === "setup" || variant.variantType === "setup+subtask")
2154
  ).length
2155
- const subtaskDrivenBenchmarkCount = overviewBenchmarkGroups.filter((group) =>
2156
- group.variants.some((variant) => variant.variantType === "subtask" || variant.variantType === "setup+subtask")
2157
  ).length
2158
 
2159
  useEffect(() => {
@@ -2414,7 +2414,7 @@ export function BenchmarkDetail({
2414
  summary.model_info.name,
2415
  ])
2416
 
2417
- // A plotbox can expose a top-level "view" selector (subtasks, child
2418
  // benchmarks, components) and an optional metric tab rail beneath the chart.
2419
  // Plotbox grouping is driven entirely by comparison-index's own
2420
  // benchmark_family_key so it stays in sync with the backend.
@@ -2448,7 +2448,7 @@ export function BenchmarkDetail({
2448
  familyName: string
2449
  category: CategoryType
2450
  kind: "single-eval" | "multi-eval"
2451
- childKindLabel: "metric" | "benchmark" | "component" | "subtask" | null
2452
  views: PlotboxView[]
2453
  primaryGroup: BenchmarkGroup
2454
  }
@@ -2534,7 +2534,7 @@ export function BenchmarkDetail({
2534
  const { familyName, category, resolved } = bucket
2535
 
2536
  if (resolved.length === 1) {
2537
- // One eval in scope — subtasks/splits become the view selector while
2538
  // metrics move to a compact tab rail beneath the chart.
2539
  const { group, evalEntry } = resolved[0]
2540
  const evalDisplay =
@@ -2545,10 +2545,10 @@ export function BenchmarkDetail({
2545
  >()
2546
 
2547
  for (const variant of group.variants) {
2548
- const viewKey = variant.subtaskLabel
2549
- ? `subtask:${normalizeDisplayKey(variant.subtaskLabel)}`
2550
  : "default"
2551
- const label = variant.subtaskLabel || "Overall"
2552
  const bucketForView = singleEvalViewBuckets.get(viewKey) ?? {
2553
  viewKey,
2554
  label,
@@ -2600,7 +2600,7 @@ export function BenchmarkDetail({
2600
  familyName: evalDisplay,
2601
  category,
2602
  kind: "single-eval",
2603
- childKindLabel: views.length > 1 ? "subtask" : null,
2604
  views,
2605
  primaryGroup: group,
2606
  })
@@ -2656,7 +2656,7 @@ export function BenchmarkDetail({
2656
  if (views.length === 0) continue
2657
 
2658
  let hasComponent = false
2659
- let hasSubtask = false
2660
  let hasDistinctLeaves = false
2661
  for (const r of children) {
2662
  const leafKey = r.evalEntry.benchmark_leaf_key
@@ -2666,17 +2666,17 @@ export function BenchmarkDetail({
2666
  if (r.group.variants[0]?.evaluation.benchmark_component_key ?? null) {
2667
  hasComponent = true
2668
  } else {
2669
- hasSubtask = true
2670
  }
2671
  }
2672
  const childKindLabel: PlotboxUnit["childKindLabel"] =
2673
- hasComponent && hasSubtask
2674
  ? "component"
2675
  : hasComponent
2676
  ? "metric"
2677
  : hasDistinctLeaves
2678
  ? "benchmark"
2679
- : "subtask"
2680
 
2681
  units.push({
2682
  unitKey: `family:${famKey}`,
@@ -2909,8 +2909,8 @@ export function BenchmarkDetail({
2909
  const childKindPlural =
2910
  unit.childKindLabel === "metric"
2911
  ? childKindCount === 1 ? "metric" : "metrics"
2912
- : unit.childKindLabel === "subtask"
2913
- ? childKindCount === 1 ? "subtask" : "subtasks"
2914
  : unit.childKindLabel === "benchmark"
2915
  ? childKindCount === 1 ? "benchmark" : "benchmarks"
2916
  : childKindCount === 1 ? "component" : "components"
@@ -3395,10 +3395,10 @@ export function BenchmarkDetail({
3395
  ? `${reportingStats.missingGenerationConfigs} entries are missing generation config, limiting cross-slice comparability.`
3396
  : "Generation configuration is present across the result set."}
3397
  </p>
3398
- {(setupDrivenBenchmarkCount > 0 || subtaskDrivenBenchmarkCount > 0) && (
3399
  <p className="text-[13px] leading-[1.7] text-[color:var(--fg-muted)]">
3400
  Decomposition: <span className="text-[color:var(--fg)]">{setupDrivenBenchmarkCount}</span> setup-aware ·{" "}
3401
- <span className="text-[color:var(--fg)]">{subtaskDrivenBenchmarkCount}</span> subtask-aware.
3402
  {reportingStats.libraryList.length > 0 && (
3403
  <>
3404
  {" "}Eval libraries: <span className="text-[color:var(--fg)]">{reportingStats.libraryList.join(", ")}</span>.
@@ -3531,7 +3531,7 @@ export function BenchmarkDetail({
3531
 
3532
  <p className="text-[14px] leading-[1.7] text-[color:var(--fg-muted)] max-w-[64rem] mb-6">
3533
  {isResearchView
3534
- ? "Benchmark-first view of this model's reported results, grouped by category. Setup spread and subtask-vs-setup differences surface up-front."
3535
  : "The public evidence behind this model, grouped by category. The strongest and most variable signals are listed first."}
3536
  {policyHighlights.length > 0 && !isResearchView && (
3537
  <>
@@ -3606,9 +3606,9 @@ export function BenchmarkDetail({
3606
  )}
3607
  {repeatedBenchmarkCount > 0 && (
3608
  <>
3609
- <dt>Subtask spread</dt>
3610
  <dd>
3611
- {repeatedBenchmarkCount} benchmark{repeatedBenchmarkCount === 1 ? "" : "s"} include multiple subtasks or setups.
3612
  </dd>
3613
  </>
3614
  )}
@@ -3701,7 +3701,7 @@ export function BenchmarkDetail({
3701
  <option value="rank">Sort · Best rank</option>
3702
  <option value="score">Sort · Highest score</option>
3703
  <option value="name">Sort · Name (A–Z)</option>
3704
- <option value="variants">Sort · Most subtasks</option>
3705
  <option value="spread">Sort · Largest spread</option>
3706
  </select>
3707
  </div>
@@ -3856,7 +3856,7 @@ export function BenchmarkDetail({
3856
  <span>{row.variant.result.generation_config.num_few_shot}-shot</span>
3857
  )}
3858
  {row.variant.setupLabel && <span>· {row.variant.setupLabel}</span>}
3859
- {row.variant.subtaskLabel && <span>· {row.variant.subtaskLabel}</span>}
3860
  </div>
3861
  )}
3862
  </div>
@@ -4527,22 +4527,22 @@ function BenchmarkResultCard({
4527
  <Separator className="my-4" />
4528
  <div className="mb-2">
4529
  <div className="text-xs font-semibold text-muted-foreground uppercase tracking-wider">Detailed Breakdown</div>
4530
- <div className="text-xs text-muted-foreground mt-1">Scores and structured metadata for individual subtasks or metrics</div>
4531
  </div>
4532
 
4533
  {numericBreakdown.length > 0 && (
4534
  <div className="grid grid-cols-1 sm:grid-cols-2 xl:grid-cols-3 gap-3">
4535
  {numericBreakdown.map(([key, value]) => {
4536
  let valDisplay = typeof value === 'number' ? value.toFixed(2) : value;
4537
- let normalized_subtask = 0;
4538
 
4539
  if (typeof value === 'number') {
4540
  if (unit === 'accuracy' || !unit || unit === 'pass@1') {
4541
  valDisplay = formatRawScoreValue(value);
4542
- normalized_subtask = value;
4543
  } else {
4544
  valDisplay = value.toFixed(2);
4545
- normalized_subtask = (value - min_score) / (max_score - min_score);
4546
  }
4547
  }
4548
 
@@ -4556,7 +4556,7 @@ function BenchmarkResultCard({
4556
  {valDisplay}
4557
  </div>
4558
  {typeof value === 'number' && (
4559
- <Progress value={normalized_subtask * 100} className="h-1 mt-2" />
4560
  )}
4561
  </div>
4562
  )})}
@@ -4800,7 +4800,7 @@ function AggregatedBenchmarkCard({
4800
  Number.isFinite(latestTimestamp) ? formatCompactDate(String(latestTimestamp)) : formatCompactDate(group.variants[0]?.evaluation.retrieved_timestamp ?? "")
4801
  const compactDomains = group.domains.slice(0, 2)
4802
  const progressWidth = Math.max(4, Math.min(100, group.avgNormalizedScore * 100))
4803
- const subtaskCount = getGroupSubtaskCount(group)
4804
 
4805
  const toggleRow = (rowKey: string) => {
4806
  setExpandedRows((current) => ({
@@ -4851,9 +4851,9 @@ function AggregatedBenchmarkCard({
4851
  card
4852
  </span>
4853
  )}
4854
- {subtaskCount > 0 && (
4855
  <span className="shrink-0 rounded-full border border-emerald-200/70 bg-emerald-50 px-1.5 py-0.5 text-[9px] font-semibold uppercase tracking-[0.12em] text-emerald-700 dark:border-emerald-900/60 dark:bg-emerald-950/30 dark:text-emerald-300">
4856
- {subtaskCount} subtask{subtaskCount === 1 ? "" : "s"}
4857
  </span>
4858
  )}
4859
  {compactDomains.map((domain) => (
@@ -4881,7 +4881,7 @@ function AggregatedBenchmarkCard({
4881
  )}
4882
  </div>
4883
 
4884
- {/* Subtask count */}
4885
  <span className="shrink-0 text-[11px] text-muted-foreground w-16 text-right hidden sm:block">
4886
  {group.variants.length} {group.variants.length === 1 ? "row" : "rows"}
4887
  </span>
@@ -5077,9 +5077,9 @@ function AggregatedBenchmarkCard({
5077
  </div>
5078
  <div className="text-xs text-muted-foreground">
5079
  {variant.setupLabel && <span>Setup: {variant.setupLabel}</span>}
5080
- {variant.setupLabel && variant.subtaskLabel && <span> • </span>}
5081
- {variant.subtaskLabel && <span>Subtask: {variant.subtaskLabel}</span>}
5082
- {!variant.setupLabel && !variant.subtaskLabel && <span>{group.title}</span>}
5083
  </div>
5084
  </div>
5085
  </div>
@@ -5193,8 +5193,8 @@ function BenchmarkDeepDiveDialogPanel({
5193
  const [resolvedRanks, setResolvedRanks] = useState<Record<string, { position: number; total: number | null }>>({})
5194
  const [isResolvingRanks, setIsResolvingRanks] = useState(false)
5195
  const compactDomains = group.domains.slice(0, 2)
5196
- const subtaskCount = getGroupSubtaskCount(group)
5197
- const hasSubtaskMatrix = subtaskCount > 0
5198
  const sourceOrganizations = useMemo(
5199
  () => new Set(group.variants.map((variant) => getOrganizationDisplayName(variant.evaluation.source_metadata.source_organization_name))),
5200
  [group.variants]
@@ -5335,18 +5335,18 @@ function BenchmarkDeepDiveDialogPanel({
5335
  // Kept only to drive the single-setup overview: when every reported row is
5336
  // reported under the same setup, the detail table collapses into a compact
5337
  // view that drops the redundant "Reporting setup" column.
5338
- const subtaskSetups = useMemo(() => {
5339
- if (!hasSubtaskMatrix) return null
5340
  const setupOrder: string[] = []
5341
  for (const row of variantRows) {
5342
  const setupDisplayLabel = formatSetupDisplayLabel(row.variant.setupLabel)
5343
  if (!setupOrder.includes(setupDisplayLabel)) setupOrder.push(setupDisplayLabel)
5344
  }
5345
  return { setupOrder }
5346
- }, [hasSubtaskMatrix, variantRows])
5347
 
5348
- const useSingleSetupOverview = Boolean(subtaskSetups && subtaskSetups.setupOrder.length === 1)
5349
- const singleSetupDisplayLabel = useSingleSetupOverview ? subtaskSetups?.setupOrder[0] ?? null : null
5350
 
5351
  useEffect(() => {
5352
  const pendingRows = variantRows.filter(
@@ -5411,7 +5411,7 @@ function BenchmarkDeepDiveDialogPanel({
5411
  </DialogTitle>
5412
  <DialogDescription className="mt-1.5 text-[13px] leading-[1.5] text-[color:var(--fg-muted)]">
5413
  {isResearchView
5414
- ? "Inspect setup subtasks, score details, and source provenance in one focused view."
5415
  : "Inspect reporting setup and evidence details before interpreting benchmark position."}
5416
  </DialogDescription>
5417
  {(compactDomains.length > 0 || group.benchmarkCard) && (
@@ -5462,9 +5462,9 @@ function BenchmarkDeepDiveDialogPanel({
5462
  <div className="kicker">Sources</div>
5463
  <div className="mt-1 text-[18px] font-semibold tabular-nums">
5464
  {sourceOrganizations.size}
5465
- {hasSubtaskMatrix && (
5466
  <span className="ml-2 font-mono text-[10px] tracking-[0.12em] uppercase text-[color:var(--fg-subtle)]">
5467
- · {subtaskCount} subtask{subtaskCount === 1 ? "" : "s"}
5468
  </span>
5469
  )}
5470
  </div>
@@ -5482,7 +5482,7 @@ function BenchmarkDeepDiveDialogPanel({
5482
 
5483
  {/* Sources — distinct reporting orgs and dataset links for this group.
5484
  Pulled up to the top of the deep-dive so the per-row table can
5485
- stay focused on subtask / setup / score. */}
5486
  {(() => {
5487
  type SourceEntry = {
5488
  key: string
@@ -5581,7 +5581,7 @@ function BenchmarkDeepDiveDialogPanel({
5581
  {useSingleSetupOverview ? (
5582
  <section>
5583
  <div className="section-head">
5584
- <h2>{hasAmbiguousPrimaryLabels ? "Reported runs" : "Subtask overview"}</h2>
5585
  <span className="font-mono text-[10px] uppercase tracking-[0.15em] text-[color:var(--fg-subtle)]">
5586
  {singleSetupDisplayLabel ? `${singleSetupDisplayLabel} · ` : ""}
5587
  {variantRows.length} row{variantRows.length === 1 ? "" : "s"}
@@ -5594,8 +5594,8 @@ function BenchmarkDeepDiveDialogPanel({
5594
  ? "These rows share the same benchmark label, so run names or differing config fields are surfaced to show what changed across reports."
5595
  : "These rows describe the same benchmark view, so the table surfaces the reported run name or setup differences that separate them."
5596
  : isResearchView
5597
- ? "This benchmark reports one setup, so subtasks, scores, and provenance are merged into one comparison view."
5598
- : "This benchmark only reports one setup, so the subtask evidence is consolidated into a single reader-friendly view."}
5599
  </p>
5600
 
5601
  <div className="min-h-0 overflow-auto">
@@ -5603,7 +5603,7 @@ function BenchmarkDeepDiveDialogPanel({
5603
  <thead>
5604
  <tr>
5605
  <th className="w-[60%]">
5606
- {hasAmbiguousPrimaryLabels ? "Reported row" : "Subtask"}
5607
  </th>
5608
  <th className="w-[20%]">Setup detail</th>
5609
  <th className="num w-[10%]">Score</th>
@@ -5678,14 +5678,14 @@ function BenchmarkDeepDiveDialogPanel({
5678
  </span>
5679
  </div>
5680
  <p className="mb-4 max-w-[60rem] text-[13px] leading-[1.65] text-[color:var(--fg-muted)]">
5681
- Primary row labels show the benchmark slice or subtask. Setup and source details sit alongside each row.
5682
  </p>
5683
 
5684
  <div className="min-h-0 overflow-auto">
5685
  <table className="ec-htable table-fixed">
5686
  <thead>
5687
  <tr>
5688
- <th className="w-[46%]">Subtask</th>
5689
  <th className="w-[36%]">Reporting setup</th>
5690
  <th className="num w-[9%]">Score</th>
5691
  <th className="num w-[9%]">Rank</th>
@@ -5895,7 +5895,7 @@ function VariantExpandedDetail({
5895
  )}
5896
  {sourceData?.dataset_version && <InlineMeta label="Dataset Version" value={sourceData.dataset_version} />}
5897
  {sourceData?.hf_split && <InlineMeta label="Split" value={sourceData.hf_split} />}
5898
- {variant.subtaskLabel && <InlineMeta label="Subtask" value={normalizeDisplayLabel(variant.subtaskLabel)} />}
5899
  {variant.setupLabel && <InlineMeta label="Setup" value={formatSetupDisplayLabel(variant.setupLabel)} />}
5900
  {inferencePlatform && <InlineMeta label="Inference Platform" value={inferencePlatform} />}
5901
  {variant.evaluation.source_metadata.source_name && (
@@ -5952,7 +5952,7 @@ function VariantExpandedDetail({
5952
  {numericBreakdown.length > 0 && (
5953
  <div className="space-y-3">
5954
  <div className="text-xs font-semibold uppercase tracking-[0.2em] text-muted-foreground">
5955
- {isResearchView ? "Subtask Scores" : "Reported Metrics"}
5956
  </div>
5957
  <div className="grid gap-3 md:grid-cols-2 2xl:grid-cols-3">
5958
  {numericBreakdown.map(([key, value]) => {
 
62
  evaluation: BenchmarkEvaluation
63
  result: EvaluationResult
64
  label: string
65
+ variantType: "setup" | "slice" | "setup+slice" | "default"
66
  metricLabel: string
67
  setupLabel: string | null
68
+ sliceLabel: string | null
69
  displayScore: string
70
  normalizedScore: number
71
  rankPosition: number | null
 
92
  variants: BenchmarkVariant[]
93
  }
94
 
95
+ interface CompositeGroup {
96
+ compositeKey: string
97
+ compositeName: string
98
  benchmarks: BenchmarkGroup[]
99
  avgRawScore: number
100
  avgNormalizedScore: number
 
269
  return normalizeDisplayLabel(value?.replace(/_/g, " ")) || "Unknown"
270
  }
271
 
272
+ function normalizeCompositeKey(key: string): string {
273
  const k = key.toLowerCase().replace(/[-.\s]+/g, "_").replace(/^_+|_+$/g, "")
274
  if (/^fibble\d*_arena$/.test(k)) return "fibble_arena"
275
  if (/^arc_agi_v\d+/.test(k)) return "arc_agi"
276
  return k
277
  }
278
 
279
+ function doesLabelMatchSuiteKey(label: string | null | undefined, compositeKey: string) {
280
  if (!label) {
281
  return false
282
  }
283
 
284
+ return normalizeCompositeKey(normalizeDisplayKey(label)) === normalizeCompositeKey(compositeKey)
285
  }
286
 
287
+ function getCompositeKey(group: BenchmarkGroup): string {
288
  const evaluation = group.variants[0]?.evaluation
289
  const backendSuiteKey =
290
  evaluation?.benchmark_parent_key ||
291
  evaluation?.benchmark_family_key ||
292
  evaluation?.benchmark
293
 
294
+ return normalizeCompositeKey(backendSuiteKey ?? group.key)
295
  }
296
 
297
+ function getCompositeDisplayName(key: string): string {
298
+ const normalizedKey = normalizeCompositeKey(key)
299
  return SUITE_DISPLAY_NAMES[normalizedKey] ?? normalizeDisplayLabel(key)
300
  }
301
 
302
+ function getCompositeName(group: BenchmarkGroup, compositeKey: string): string {
303
  const evaluation = group.variants[0]?.evaluation
304
  const benchmarkCardName = group.benchmarkCard?.benchmark_details?.name
305
  const backendParentName = evaluation?.benchmark_parent_name
306
  const backendFamilyName = evaluation?.benchmark_family_name
307
 
308
+ if (doesLabelMatchSuiteKey(backendParentName, compositeKey)) {
309
  return normalizeDisplayLabel(backendParentName)
310
  }
311
 
312
+ if (doesLabelMatchSuiteKey(backendFamilyName, compositeKey)) {
313
  return normalizeDisplayLabel(backendFamilyName)
314
  }
315
 
316
+ if (doesLabelMatchSuiteKey(benchmarkCardName, compositeKey)) {
317
  return normalizeDisplayLabel(benchmarkCardName)
318
  }
319
 
320
+ return getCompositeDisplayName(compositeKey)
321
  }
322
 
323
+ function groupByComposite(
324
  groups: BenchmarkGroup[],
325
  modelIds: string[],
326
  peerRanks: PeerRanksMap
327
+ ): CompositeGroup[] {
328
+ const composites = new Map<string, BenchmarkGroup[]>()
329
  for (const group of groups) {
330
+ const key = getCompositeKey(group)
331
+ const existing = composites.get(key) ?? []
332
  existing.push(group)
333
+ composites.set(key, existing)
334
  }
335
 
336
+ return Array.from(composites.entries()).map(([compositeKey, benchmarks]) => {
337
  const scores = benchmarks.map(b => b.avgNormalizedScore).filter(Number.isFinite)
338
  const avgScore = scores.length > 0 ? scores.reduce((a, b) => a + b, 0) / scores.length : 0
339
  const rawScores = benchmarks.map((benchmark) => benchmark.avgRawScore).filter(Number.isFinite)
340
  const avgRawScore = rawScores.length > 0 ? rawScores.reduce((a, b) => a + b, 0) / rawScores.length : 0
341
 
342
+ // Find best rank across all benchmarks in the composite
343
  let bestRank: { position: number; total: number } | null = null
344
  for (const b of benchmarks) {
345
  const rank = getGroupPeerRank(b, modelIds, peerRanks)
 
350
  }
351
 
352
  return {
353
+ compositeKey,
354
+ compositeName: benchmarks[0] ? getCompositeName(benchmarks[0], compositeKey) : getCompositeDisplayName(compositeKey),
355
  benchmarks,
356
  avgRawScore,
357
  avgNormalizedScore: avgScore,
 
359
  bestRank,
360
  }
361
  }).sort((a, b) => {
362
+ // Sort by best peer rank ratio (lower = better); unranked composites go to the bottom
363
  const aRatio = a.bestRank ? a.bestRank.position / (a.bestRank.total || a.bestRank.position) : Infinity
364
  const bRatio = b.bestRank ? b.bestRank.position / (b.bestRank.total || b.bestRank.position) : Infinity
365
  if (aRatio !== bRatio) return aRatio - bRatio
 
466
  function getVariantDescriptor(
467
  evaluation: BenchmarkEvaluation,
468
  result: EvaluationResult
469
+ ): Pick<BenchmarkVariant, "label" | "variantType" | "metricLabel" | "setupLabel" | "sliceLabel"> {
470
  const evaluationVariantRaw = getEvaluationVariantLabel(evaluation)
471
  const evaluationVariant = evaluationVariantRaw ? formatSetupDisplayLabel(evaluationVariantRaw) : null
472
  const metricLabel = getMetricDisplayLabel(result)
473
  const metricKey = normalizeDisplayKey(metricLabel)
474
  const metricIsAmbiguous = AMBIGUOUS_GROUP_LABELS.has(metricKey)
475
+ const sliceLabel = evaluation.slice_name ? normalizeDisplayLabel(evaluation.slice_name) : null
476
  const setupLabel = evaluationVariant ? formatSetupDisplayLabel(evaluationVariant) : null
477
+ const baseLabel = sliceLabel
478
+ ? (metricIsAmbiguous ? sliceLabel : `${sliceLabel} · ${metricLabel}`)
479
  : metricLabel
480
 
481
+ if (setupLabel && sliceLabel) {
482
  return {
483
  label: `${setupLabel} · ${baseLabel}`,
484
+ variantType: "setup+slice",
485
  metricLabel,
486
  setupLabel,
487
+ sliceLabel,
488
  }
489
  }
490
 
 
494
  variantType: "setup",
495
  metricLabel,
496
  setupLabel,
497
+ sliceLabel: null,
498
  }
499
  }
500
 
501
+ if (sliceLabel || !metricIsAmbiguous) {
502
  return {
503
  label: baseLabel,
504
+ variantType: sliceLabel ? "slice" : "default",
505
  metricLabel,
506
  setupLabel: null,
507
+ sliceLabel: sliceLabel ?? null,
508
  }
509
  }
510
 
 
513
  variantType: "default",
514
  metricLabel,
515
  setupLabel: null,
516
+ sliceLabel: null,
517
  }
518
  }
519
 
 
603
  return row.variant.setupLabel
604
  }
605
 
606
+ if (row.variant.variantType === "slice") {
607
  return "Default setup"
608
  }
609
 
 
865
  switch (variantType) {
866
  case "setup":
867
  return "bg-sky-100 text-sky-800 dark:bg-sky-950/50 dark:text-sky-300"
868
+ case "slice":
869
  return "bg-violet-100 text-violet-800 dark:bg-violet-950/50 dark:text-violet-300"
870
+ case "setup+slice":
871
  return "bg-amber-100 text-amber-800 dark:bg-amber-950/50 dark:text-amber-300"
872
  default:
873
  return "bg-muted text-muted-foreground"
 
878
  switch (variantType) {
879
  case "setup":
880
  return "Setup change"
881
+ case "slice":
882
+ return "Benchmark slice"
883
+ case "setup+slice":
884
+ return "Setup + slice"
885
  default:
886
  return "Single run"
887
  }
 
911
  }
912
 
913
  function getVariantPrimaryLabel(variant: BenchmarkVariant, groupTitle: string) {
914
+ if (variant.sliceLabel) {
915
+ return variant.sliceLabel
916
  }
917
 
918
  if (variant.metricLabel) {
 
929
  return variant.label
930
  }
931
 
932
+ function getGroupSliceLabels(group: BenchmarkGroup) {
933
  return Array.from(
934
  new Set(
935
  group.variants
936
+ .map((variant) => variant.sliceLabel?.trim())
937
  .filter((label): label is string => Boolean(label))
938
  )
939
  )
940
  }
941
 
942
+ function getGroupSliceCount(group: BenchmarkGroup) {
943
+ return getGroupSliceLabels(group).length
944
  }
945
 
946
  function getBenchmarkGroupHeading(group: BenchmarkGroup) {
947
  return group.canonicalTitle
948
  }
949
 
950
+ function getCompositeBadgeMeta(composite: CompositeGroup) {
951
+ if (composite.benchmarks.length > 1) {
952
  return {
953
+ count: composite.benchmarks.length,
954
+ label: `sub-benchmark${composite.benchmarks.length === 1 ? "" : "s"}`,
955
  className:
956
  "border-sky-200/80 bg-sky-50/70 text-sky-700 dark:border-sky-900/60 dark:bg-sky-950/30 dark:text-sky-300",
957
  }
958
  }
959
 
960
+ const singleGroup = composite.benchmarks[0]
961
  if (!singleGroup) {
962
  return null
963
  }
964
 
965
+ const compositeMatchesBenchmark = normalizeCompositeKey(composite.compositeName) === normalizeCompositeKey(singleGroup.title)
966
+ if (!compositeMatchesBenchmark) {
967
  return {
968
  count: 1,
969
  label: "sub-benchmark",
 
972
  }
973
  }
974
 
975
+ const sliceCount = getGroupSliceCount(singleGroup)
976
+ if (sliceCount > 0) {
977
  return {
978
+ count: sliceCount,
979
+ label: `slice${sliceCount === 1 ? "" : "s"}`,
980
  className:
981
  "border-emerald-200/70 bg-emerald-50 text-emerald-700 dark:border-emerald-900/60 dark:bg-emerald-950/30 dark:text-emerald-300",
982
  }
 
1405
  metricLabel: variant.metricLabel,
1406
  variantType: variant.variantType,
1407
  setupLabel: variant.setupLabel,
1408
+ sliceLabel: variant.sliceLabel,
1409
  displayScore: variant.displayScore,
1410
  sourceOrganization: getOrganizationDisplayName(variant.evaluation.source_metadata.source_organization_name),
1411
  sourceName: normalizeDisplayLabel(variant.evaluation.source_metadata.source_name ?? ""),
 
1459
  variantType: descriptor.variantType,
1460
  metricLabel: descriptor.metricLabel,
1461
  setupLabel: descriptor.setupLabel,
1462
+ sliceLabel: descriptor.sliceLabel,
1463
  displayScore,
1464
  normalizedScore,
1465
  rankPosition,
 
1570
 
1571
  group.variants = Array.from(dedupedVariants.values())
1572
  group.variants.sort((a, b) => {
1573
+ const aIsSlice = Boolean(a.evaluation.slice_key)
1574
+ const bIsSlice = Boolean(b.evaluation.slice_key)
1575
+ if (aIsSlice !== bIsSlice) {
1576
+ return aIsSlice ? 1 : -1
1577
  }
1578
 
1579
  const aPrimaryLabel = getVariantPrimaryLabel(a, group.title)
 
2043
  }))
2044
  }, [filteredBenchmarkGroups, comparisonIndex, summary.categories_covered])
2045
 
2046
+ const compositeGroups = useMemo(() => {
2047
+ const groups = groupByComposite(filteredBenchmarkGroups, modelIds, peerRanks)
2048
+ // Re-sort composites by max relevance of their benchmarks
2049
  return groups.sort((a, b) => {
2050
  const aMax = Math.max(...a.benchmarks.map(getRelevanceScore))
2051
  const bMax = Math.max(...b.benchmarks.map(getRelevanceScore))
 
2053
  })
2054
  }, [filteredBenchmarkGroups, modelIds, peerRanks, getRelevanceScore])
2055
 
2056
+ const categoryCompositeSections = useMemo(
2057
  () =>
2058
  groupedFilteredBenchmarkGroups
2059
  .map(({ category, groups }) => ({
2060
  category,
2061
+ composites: groupByComposite(groups, modelIds, peerRanks).sort((a, b) => {
2062
  const aMax = Math.max(...a.benchmarks.map(getRelevanceScore))
2063
  const bMax = Math.max(...b.benchmarks.map(getRelevanceScore))
2064
  return bMax - aMax
2065
  }),
2066
  }))
2067
+ .filter((section) => section.composites.length > 0),
2068
  [groupedFilteredBenchmarkGroups, modelIds, peerRanks, getRelevanceScore]
2069
  )
2070
 
2071
  const categoryScoreRanges = useMemo(() => {
2072
  const ranges = new Map<CategoryType, ScoreRange>()
2073
 
2074
+ for (const section of categoryCompositeSections) {
2075
  ranges.set(
2076
  section.category,
2077
+ getScoreRange(section.composites.map((composite) => composite.avgNormalizedScore))
2078
  )
2079
  }
2080
 
2081
  return ranges
2082
+ }, [categoryCompositeSections])
2083
 
2084
+ const compositeBenchmarkScoreRanges = useMemo(() => {
2085
  const ranges = new Map<string, ScoreRange>()
2086
 
2087
+ for (const section of categoryCompositeSections) {
2088
+ for (const composite of section.composites) {
2089
  ranges.set(
2090
+ composite.compositeKey,
2091
+ getScoreRange(composite.benchmarks.map((group) => group.avgNormalizedScore))
2092
  )
2093
  }
2094
  }
2095
 
2096
  return ranges
2097
+ }, [categoryCompositeSections])
2098
 
2099
  const benchmarkGroupLookup = useMemo(
2100
  () => new Map(benchmarkGroups.map((group) => [group.key, group] as const)),
 
2104
  ? benchmarkGroupLookup.get(activeBenchmarkGroupKey) ?? null
2105
  : null
2106
 
2107
+ const toggleSuite = (compositeKey: string) => {
2108
  setExpandedSuites((prev) => {
2109
  const next = new Set(prev)
2110
+ if (next.has(compositeKey)) next.delete(compositeKey)
2111
+ else next.add(compositeKey)
2112
  return next
2113
  })
2114
  }
 
2150
  )
2151
  const repeatedBenchmarkCount = overviewBenchmarkGroups.filter((group) => group.variants.length > 1).length
2152
  const setupDrivenBenchmarkCount = overviewBenchmarkGroups.filter((group) =>
2153
+ group.variants.some((variant) => variant.variantType === "setup" || variant.variantType === "setup+slice")
2154
  ).length
2155
+ const sliceDrivenBenchmarkCount = overviewBenchmarkGroups.filter((group) =>
2156
+ group.variants.some((variant) => variant.variantType === "slice" || variant.variantType === "setup+slice")
2157
  ).length
2158
 
2159
  useEffect(() => {
 
2414
  summary.model_info.name,
2415
  ])
2416
 
2417
+ // A plotbox can expose a top-level "view" selector (slices, child
2418
  // benchmarks, components) and an optional metric tab rail beneath the chart.
2419
  // Plotbox grouping is driven entirely by comparison-index's own
2420
  // benchmark_family_key so it stays in sync with the backend.
 
2448
  familyName: string
2449
  category: CategoryType
2450
  kind: "single-eval" | "multi-eval"
2451
+ childKindLabel: "metric" | "benchmark" | "component" | "slice" | null
2452
  views: PlotboxView[]
2453
  primaryGroup: BenchmarkGroup
2454
  }
 
2534
  const { familyName, category, resolved } = bucket
2535
 
2536
  if (resolved.length === 1) {
2537
+ // One eval in scope — slices/splits become the view selector while
2538
  // metrics move to a compact tab rail beneath the chart.
2539
  const { group, evalEntry } = resolved[0]
2540
  const evalDisplay =
 
2545
  >()
2546
 
2547
  for (const variant of group.variants) {
2548
+ const viewKey = variant.sliceLabel
2549
+ ? `slice:${normalizeDisplayKey(variant.sliceLabel)}`
2550
  : "default"
2551
+ const label = variant.sliceLabel || "Overall"
2552
  const bucketForView = singleEvalViewBuckets.get(viewKey) ?? {
2553
  viewKey,
2554
  label,
 
2600
  familyName: evalDisplay,
2601
  category,
2602
  kind: "single-eval",
2603
+ childKindLabel: views.length > 1 ? "slice" : null,
2604
  views,
2605
  primaryGroup: group,
2606
  })
 
2656
  if (views.length === 0) continue
2657
 
2658
  let hasComponent = false
2659
+ let hasSlice = false
2660
  let hasDistinctLeaves = false
2661
  for (const r of children) {
2662
  const leafKey = r.evalEntry.benchmark_leaf_key
 
2666
  if (r.group.variants[0]?.evaluation.benchmark_component_key ?? null) {
2667
  hasComponent = true
2668
  } else {
2669
+ hasSlice = true
2670
  }
2671
  }
2672
  const childKindLabel: PlotboxUnit["childKindLabel"] =
2673
+ hasComponent && hasSlice
2674
  ? "component"
2675
  : hasComponent
2676
  ? "metric"
2677
  : hasDistinctLeaves
2678
  ? "benchmark"
2679
+ : "slice"
2680
 
2681
  units.push({
2682
  unitKey: `family:${famKey}`,
 
2909
  const childKindPlural =
2910
  unit.childKindLabel === "metric"
2911
  ? childKindCount === 1 ? "metric" : "metrics"
2912
+ : unit.childKindLabel === "slice"
2913
+ ? childKindCount === 1 ? "slice" : "slices"
2914
  : unit.childKindLabel === "benchmark"
2915
  ? childKindCount === 1 ? "benchmark" : "benchmarks"
2916
  : childKindCount === 1 ? "component" : "components"
 
3395
  ? `${reportingStats.missingGenerationConfigs} entries are missing generation config, limiting cross-slice comparability.`
3396
  : "Generation configuration is present across the result set."}
3397
  </p>
3398
+ {(setupDrivenBenchmarkCount > 0 || sliceDrivenBenchmarkCount > 0) && (
3399
  <p className="text-[13px] leading-[1.7] text-[color:var(--fg-muted)]">
3400
  Decomposition: <span className="text-[color:var(--fg)]">{setupDrivenBenchmarkCount}</span> setup-aware ·{" "}
3401
+ <span className="text-[color:var(--fg)]">{sliceDrivenBenchmarkCount}</span> slice-aware.
3402
  {reportingStats.libraryList.length > 0 && (
3403
  <>
3404
  {" "}Eval libraries: <span className="text-[color:var(--fg)]">{reportingStats.libraryList.join(", ")}</span>.
 
3531
 
3532
  <p className="text-[14px] leading-[1.7] text-[color:var(--fg-muted)] max-w-[64rem] mb-6">
3533
  {isResearchView
3534
+ ? "Benchmark-first view of this model's reported results, grouped by category. Setup spread and slice-vs-setup differences surface up-front."
3535
  : "The public evidence behind this model, grouped by category. The strongest and most variable signals are listed first."}
3536
  {policyHighlights.length > 0 && !isResearchView && (
3537
  <>
 
3606
  )}
3607
  {repeatedBenchmarkCount > 0 && (
3608
  <>
3609
+ <dt>Slice spread</dt>
3610
  <dd>
3611
+ {repeatedBenchmarkCount} benchmark{repeatedBenchmarkCount === 1 ? "" : "s"} include multiple slices or setups.
3612
  </dd>
3613
  </>
3614
  )}
 
3701
  <option value="rank">Sort · Best rank</option>
3702
  <option value="score">Sort · Highest score</option>
3703
  <option value="name">Sort · Name (A–Z)</option>
3704
+ <option value="variants">Sort · Most slices</option>
3705
  <option value="spread">Sort · Largest spread</option>
3706
  </select>
3707
  </div>
 
3856
  <span>{row.variant.result.generation_config.num_few_shot}-shot</span>
3857
  )}
3858
  {row.variant.setupLabel && <span>· {row.variant.setupLabel}</span>}
3859
+ {row.variant.sliceLabel && <span>· {row.variant.sliceLabel}</span>}
3860
  </div>
3861
  )}
3862
  </div>
 
4527
  <Separator className="my-4" />
4528
  <div className="mb-2">
4529
  <div className="text-xs font-semibold text-muted-foreground uppercase tracking-wider">Detailed Breakdown</div>
4530
+ <div className="text-xs text-muted-foreground mt-1">Scores and structured metadata for individual slices or metrics</div>
4531
  </div>
4532
 
4533
  {numericBreakdown.length > 0 && (
4534
  <div className="grid grid-cols-1 sm:grid-cols-2 xl:grid-cols-3 gap-3">
4535
  {numericBreakdown.map(([key, value]) => {
4536
  let valDisplay = typeof value === 'number' ? value.toFixed(2) : value;
4537
+ let normalized_slice = 0;
4538
 
4539
  if (typeof value === 'number') {
4540
  if (unit === 'accuracy' || !unit || unit === 'pass@1') {
4541
  valDisplay = formatRawScoreValue(value);
4542
+ normalized_slice = value;
4543
  } else {
4544
  valDisplay = value.toFixed(2);
4545
+ normalized_slice = (value - min_score) / (max_score - min_score);
4546
  }
4547
  }
4548
 
 
4556
  {valDisplay}
4557
  </div>
4558
  {typeof value === 'number' && (
4559
+ <Progress value={normalized_slice * 100} className="h-1 mt-2" />
4560
  )}
4561
  </div>
4562
  )})}
 
4800
  Number.isFinite(latestTimestamp) ? formatCompactDate(String(latestTimestamp)) : formatCompactDate(group.variants[0]?.evaluation.retrieved_timestamp ?? "")
4801
  const compactDomains = group.domains.slice(0, 2)
4802
  const progressWidth = Math.max(4, Math.min(100, group.avgNormalizedScore * 100))
4803
+ const sliceCount = getGroupSliceCount(group)
4804
 
4805
  const toggleRow = (rowKey: string) => {
4806
  setExpandedRows((current) => ({
 
4851
  card
4852
  </span>
4853
  )}
4854
+ {sliceCount > 0 && (
4855
  <span className="shrink-0 rounded-full border border-emerald-200/70 bg-emerald-50 px-1.5 py-0.5 text-[9px] font-semibold uppercase tracking-[0.12em] text-emerald-700 dark:border-emerald-900/60 dark:bg-emerald-950/30 dark:text-emerald-300">
4856
+ {sliceCount} slice{sliceCount === 1 ? "" : "s"}
4857
  </span>
4858
  )}
4859
  {compactDomains.map((domain) => (
 
4881
  )}
4882
  </div>
4883
 
4884
+ {/* Slice count */}
4885
  <span className="shrink-0 text-[11px] text-muted-foreground w-16 text-right hidden sm:block">
4886
  {group.variants.length} {group.variants.length === 1 ? "row" : "rows"}
4887
  </span>
 
5077
  </div>
5078
  <div className="text-xs text-muted-foreground">
5079
  {variant.setupLabel && <span>Setup: {variant.setupLabel}</span>}
5080
+ {variant.setupLabel && variant.sliceLabel && <span> • </span>}
5081
+ {variant.sliceLabel && <span>Slice: {variant.sliceLabel}</span>}
5082
+ {!variant.setupLabel && !variant.sliceLabel && <span>{group.title}</span>}
5083
  </div>
5084
  </div>
5085
  </div>
 
5193
  const [resolvedRanks, setResolvedRanks] = useState<Record<string, { position: number; total: number | null }>>({})
5194
  const [isResolvingRanks, setIsResolvingRanks] = useState(false)
5195
  const compactDomains = group.domains.slice(0, 2)
5196
+ const sliceCount = getGroupSliceCount(group)
5197
+ const hasSliceMatrix = sliceCount > 0
5198
  const sourceOrganizations = useMemo(
5199
  () => new Set(group.variants.map((variant) => getOrganizationDisplayName(variant.evaluation.source_metadata.source_organization_name))),
5200
  [group.variants]
 
5335
  // Kept only to drive the single-setup overview: when every reported row is
5336
  // reported under the same setup, the detail table collapses into a compact
5337
  // view that drops the redundant "Reporting setup" column.
5338
+ const sliceSetups = useMemo(() => {
5339
+ if (!hasSliceMatrix) return null
5340
  const setupOrder: string[] = []
5341
  for (const row of variantRows) {
5342
  const setupDisplayLabel = formatSetupDisplayLabel(row.variant.setupLabel)
5343
  if (!setupOrder.includes(setupDisplayLabel)) setupOrder.push(setupDisplayLabel)
5344
  }
5345
  return { setupOrder }
5346
+ }, [hasSliceMatrix, variantRows])
5347
 
5348
+ const useSingleSetupOverview = Boolean(sliceSetups && sliceSetups.setupOrder.length === 1)
5349
+ const singleSetupDisplayLabel = useSingleSetupOverview ? sliceSetups?.setupOrder[0] ?? null : null
5350
 
5351
  useEffect(() => {
5352
  const pendingRows = variantRows.filter(
 
5411
  </DialogTitle>
5412
  <DialogDescription className="mt-1.5 text-[13px] leading-[1.5] text-[color:var(--fg-muted)]">
5413
  {isResearchView
5414
+ ? "Inspect setup slices, score details, and source provenance in one focused view."
5415
  : "Inspect reporting setup and evidence details before interpreting benchmark position."}
5416
  </DialogDescription>
5417
  {(compactDomains.length > 0 || group.benchmarkCard) && (
 
5462
  <div className="kicker">Sources</div>
5463
  <div className="mt-1 text-[18px] font-semibold tabular-nums">
5464
  {sourceOrganizations.size}
5465
+ {hasSliceMatrix && (
5466
  <span className="ml-2 font-mono text-[10px] tracking-[0.12em] uppercase text-[color:var(--fg-subtle)]">
5467
+ · {sliceCount} slice{sliceCount === 1 ? "" : "s"}
5468
  </span>
5469
  )}
5470
  </div>
 
5482
 
5483
  {/* Sources — distinct reporting orgs and dataset links for this group.
5484
  Pulled up to the top of the deep-dive so the per-row table can
5485
+ stay focused on slice / setup / score. */}
5486
  {(() => {
5487
  type SourceEntry = {
5488
  key: string
 
5581
  {useSingleSetupOverview ? (
5582
  <section>
5583
  <div className="section-head">
5584
+ <h2>{hasAmbiguousPrimaryLabels ? "Reported runs" : "Slice overview"}</h2>
5585
  <span className="font-mono text-[10px] uppercase tracking-[0.15em] text-[color:var(--fg-subtle)]">
5586
  {singleSetupDisplayLabel ? `${singleSetupDisplayLabel} · ` : ""}
5587
  {variantRows.length} row{variantRows.length === 1 ? "" : "s"}
 
5594
  ? "These rows share the same benchmark label, so run names or differing config fields are surfaced to show what changed across reports."
5595
  : "These rows describe the same benchmark view, so the table surfaces the reported run name or setup differences that separate them."
5596
  : isResearchView
5597
+ ? "This benchmark reports one setup, so slices, scores, and provenance are merged into one comparison view."
5598
+ : "This benchmark only reports one setup, so the slice evidence is consolidated into a single reader-friendly view."}
5599
  </p>
5600
 
5601
  <div className="min-h-0 overflow-auto">
 
5603
  <thead>
5604
  <tr>
5605
  <th className="w-[60%]">
5606
+ {hasAmbiguousPrimaryLabels ? "Reported row" : "Slice"}
5607
  </th>
5608
  <th className="w-[20%]">Setup detail</th>
5609
  <th className="num w-[10%]">Score</th>
 
5678
  </span>
5679
  </div>
5680
  <p className="mb-4 max-w-[60rem] text-[13px] leading-[1.65] text-[color:var(--fg-muted)]">
5681
+ Primary row labels show the benchmark slice or slice. Setup and source details sit alongside each row.
5682
  </p>
5683
 
5684
  <div className="min-h-0 overflow-auto">
5685
  <table className="ec-htable table-fixed">
5686
  <thead>
5687
  <tr>
5688
+ <th className="w-[46%]">Slice</th>
5689
  <th className="w-[36%]">Reporting setup</th>
5690
  <th className="num w-[9%]">Score</th>
5691
  <th className="num w-[9%]">Rank</th>
 
5895
  )}
5896
  {sourceData?.dataset_version && <InlineMeta label="Dataset Version" value={sourceData.dataset_version} />}
5897
  {sourceData?.hf_split && <InlineMeta label="Split" value={sourceData.hf_split} />}
5898
+ {variant.sliceLabel && <InlineMeta label="Slice" value={normalizeDisplayLabel(variant.sliceLabel)} />}
5899
  {variant.setupLabel && <InlineMeta label="Setup" value={formatSetupDisplayLabel(variant.setupLabel)} />}
5900
  {inferencePlatform && <InlineMeta label="Inference Platform" value={inferencePlatform} />}
5901
  {variant.evaluation.source_metadata.source_name && (
 
5952
  {numericBreakdown.length > 0 && (
5953
  <div className="space-y-3">
5954
  <div className="text-xs font-semibold uppercase tracking-[0.2em] text-muted-foreground">
5955
+ {isResearchView ? "Slice Scores" : "Reported Metrics"}
5956
  </div>
5957
  <div className="grid gap-3 md:grid-cols-2 2xl:grid-cols-3">
5958
  {numericBreakdown.map(([key, value]) => {
components/benchmark-evaluation-card.tsx CHANGED
@@ -135,7 +135,7 @@ function formatScoreValue(value: number | null | undefined) {
135
 
136
  function getCoverageSummaryLabel(data: BenchmarkEvaluationCardData) {
137
  if (data.benchmarks_count > 0) {
138
- return `${data.benchmarks_count} benchmark suite${data.benchmarks_count === 1 ? "" : "s"} surfaced`
139
  }
140
 
141
  if (data.latest_source_name) {
@@ -302,7 +302,7 @@ export function BenchmarkEvaluationCard({
302
  <Badge variant="secondary">{data.variant_count} versions</Badge>
303
  )}
304
  {paramsBillions && <Badge variant="secondary">{paramsBillions} parameters</Badge>}
305
- <Badge variant="outline">{data.benchmarks_count} benchmark suites</Badge>
306
  <Badge variant="outline">{data.evaluations_count} reported results</Badge>
307
  {reproducibilityGapCount > 0 && (
308
  <Badge
 
135
 
136
  function getCoverageSummaryLabel(data: BenchmarkEvaluationCardData) {
137
  if (data.benchmarks_count > 0) {
138
+ return `${data.benchmarks_count} benchmark composite${data.benchmarks_count === 1 ? "" : "s"} surfaced`
139
  }
140
 
141
  if (data.latest_source_name) {
 
302
  <Badge variant="secondary">{data.variant_count} versions</Badge>
303
  )}
304
  {paramsBillions && <Badge variant="secondary">{paramsBillions} parameters</Badge>}
305
+ <Badge variant="outline">{data.benchmarks_count} benchmark composites</Badge>
306
  <Badge variant="outline">{data.evaluations_count} reported results</Badge>
307
  {reproducibilityGapCount > 0 && (
308
  <Badge
components/eval-card.tsx CHANGED
@@ -120,7 +120,7 @@ export function EvalCard({ summary, delayMs = 0 }: EvalCardProps) {
120
  <div className="min-w-0">
121
  <div className="text-xl font-bold">{summary.evaluation_name}</div>
122
  <div className="mt-1 text-xs font-medium uppercase tracking-[0.18em] text-muted-foreground">
123
- Suite: {summary.composite_benchmark_name}
124
  </div>
125
  <div className="mt-1 text-sm text-muted-foreground line-clamp-2">
126
  {/*
 
120
  <div className="min-w-0">
121
  <div className="text-xl font-bold">{summary.evaluation_name}</div>
122
  <div className="mt-1 text-xs font-medium uppercase tracking-[0.18em] text-muted-foreground">
123
+ Composite: {summary.composite_benchmark_name}
124
  </div>
125
  <div className="mt-1 text-sm text-muted-foreground line-clamp-2">
126
  {/*
components/eval-detail.tsx CHANGED
@@ -113,14 +113,14 @@ interface SliceTab {
113
  * - <= SLICE_PILL_THRESHOLD: render every slice as a pill (current familiar UX).
114
  * - > SLICE_PILL_THRESHOLD: render "All slices" + currently-selected pill +
115
  * a "Browse N slices" button that opens a searchable dialog. Hundreds of
116
- * subtasks (e.g. AIRBench's 374) fit cleanly.
117
  */
118
  function SliceSelector({
119
- activeSubtaskTab,
120
  onChange,
121
  tabs,
122
  }: {
123
- activeSubtaskTab: string
124
  onChange: (key: string) => void
125
  tabs: SliceTab[]
126
  }) {
@@ -128,7 +128,7 @@ function SliceSelector({
128
  const [search, setSearch] = useState("")
129
 
130
  const useBrowser = tabs.length > SLICE_PILL_THRESHOLD
131
- const activeTab = tabs.find((tab) => tab.key === activeSubtaskTab)
132
 
133
  const filteredTabs = useMemo(() => {
134
  const query = search.trim().toLowerCase()
@@ -143,7 +143,7 @@ function SliceSelector({
143
  <div className="flex flex-wrap gap-2">
144
  <button
145
  type="button"
146
- className={`ec-pill${activeSubtaskTab === "all" ? " on" : ""}`}
147
  onClick={() => onChange("all")}
148
  >
149
  All slices
@@ -152,7 +152,7 @@ function SliceSelector({
152
  <button
153
  key={tab.key}
154
  type="button"
155
- className={`ec-pill${activeSubtaskTab === tab.key ? " on" : ""}`}
156
  onClick={() => onChange(tab.key)}
157
  >
158
  {tab.label}
@@ -172,7 +172,7 @@ function SliceSelector({
172
  <div className="flex flex-wrap items-center gap-2">
173
  <button
174
  type="button"
175
- className={`ec-pill${activeSubtaskTab === "all" ? " on" : ""}`}
176
  onClick={() => onChange("all")}
177
  >
178
  All slices
@@ -230,11 +230,11 @@ function SliceSelector({
230
  }}
231
  className={cn(
232
  "flex w-full items-center justify-between border-b px-4 py-2.5 text-left text-sm transition-colors hover:bg-muted/40",
233
- activeSubtaskTab === "all" && "bg-muted/40 font-semibold"
234
  )}
235
  >
236
  <span>All slices (no filter)</span>
237
- {activeSubtaskTab === "all" && <span className="text-xs text-muted-foreground">selected</span>}
238
  </button>
239
  {filteredTabs.length === 0 ? (
240
  <div className="px-4 py-6 text-center text-sm text-muted-foreground">
@@ -251,11 +251,11 @@ function SliceSelector({
251
  }}
252
  className={cn(
253
  "flex w-full items-center justify-between border-b px-4 py-2 text-left text-sm transition-colors hover:bg-muted/40 last:border-b-0",
254
- activeSubtaskTab === tab.key && "bg-muted/40 font-semibold"
255
  )}
256
  >
257
  <span className="min-w-0 truncate pr-2">{tab.label}</span>
258
- {activeSubtaskTab === tab.key && (
259
  <span className="shrink-0 text-xs text-muted-foreground">selected</span>
260
  )}
261
  </button>
@@ -543,10 +543,10 @@ export function EvalDetail({ summary }: EvalDetailProps) {
543
  : "Not linked"
544
  const leaderboardDescription = isResearchView
545
  ? summary.is_aggregated
546
- ? "Models ranked by average raw score across the suite's component benchmarks."
547
  : "Models ranked by raw score for this benchmark."
548
  : summary.is_aggregated
549
- ? "Averaged model results across the suite's component benchmarks, with drill-down to each component score."
550
  : "Model results with benchmark context, source dataset detail, and optional instance-data links."
551
  const reportingCompleteness = summary.evalcards?.annotations?.reporting_completeness
552
  const documentationPopulatedCount = reportingCompleteness
@@ -656,7 +656,7 @@ export function EvalDetail({ summary }: EvalDetailProps) {
656
  className="font-mono text-[10px] uppercase tracking-[0.12em]"
657
  style={{ color: "var(--fg-subtle)" }}
658
  >
659
- metric spec · completeness · comparability{summary.subtasks?.length ? " · subtasks" : ""}
660
  </span>
661
  </div>
662
  {overviewOpen ? (
@@ -678,10 +678,10 @@ export function EvalDetail({ summary }: EvalDetailProps) {
678
  {isResearchView ? "Metric specification" : "Reading context"}
679
  </div>
680
  <dl className="ec-datalist">
681
- <dt>Suite</dt>
682
  <dd>
683
  {summary.is_aggregated
684
- ? summary.aggregate_sources?.map((source) => source.composite_benchmark_name).join(", ") || "Multiple suites"
685
  : summary.composite_benchmark_name}
686
  </dd>
687
  <dt>{isResearchView ? "Benchmark ID" : "What this covers"}</dt>
@@ -744,7 +744,7 @@ export function EvalDetail({ summary }: EvalDetailProps) {
744
  >
745
  <div className="kicker mb-2">Benchmark structure</div>
746
  <p className="text-[13px] mb-4" style={{ color: "var(--fg-muted)", maxWidth: 640 }}>
747
- Benchmark-level summary metrics and subtask slices grouped in one compact section.
748
  </p>
749
 
750
  {summary.root_metrics && summary.root_metrics.length > 0 && (
@@ -776,15 +776,15 @@ export function EvalDetail({ summary }: EvalDetailProps) {
776
  className="font-mono uppercase mb-1"
777
  style={{ fontSize: 10, letterSpacing: "0.14em", color: "var(--fg-subtle)" }}
778
  >
779
- Subtask breakdown · {summary.subtasks.length}
780
  </div>
781
  <ul
782
  className="flex flex-col"
783
  style={{ borderTop: "1px solid var(--border-soft)" }}
784
  >
785
- {summary.subtasks.map((subtask) => (
786
  <li
787
- key={subtask.subtask_key}
788
  className="grid gap-x-4 py-3"
789
  style={{
790
  gridTemplateColumns: "minmax(160px, 280px) 1fr",
@@ -793,20 +793,20 @@ export function EvalDetail({ summary }: EvalDetailProps) {
793
  >
794
  <div className="min-w-0">
795
  <div className="font-semibold text-[13px] truncate">
796
- {subtask.display_name || subtask.subtask_name}
797
  </div>
798
- {subtask.canonical_display_name && subtask.canonical_display_name !== (subtask.display_name || subtask.subtask_name) && (
799
  <div
800
  className="mt-0.5 font-mono text-[10px] uppercase tracking-[0.08em] truncate"
801
  style={{ color: "var(--fg-subtle)" }}
802
- title={subtask.canonical_display_name}
803
  >
804
- {subtask.canonical_display_name}
805
  </div>
806
  )}
807
  </div>
808
  <div className="flex flex-wrap items-center gap-1.5">
809
- {subtask.metrics.map((metric) => (
810
  <span
811
  key={metric.metric_summary_id}
812
  className="ec-tag"
@@ -919,13 +919,13 @@ export function EvalDetail({ summary }: EvalDetailProps) {
919
  <tbody>
920
  {pagedLeaderboardRows.map(({ key, rank, modelResult, normalizedScore }) => {
921
  const isExpanded = expandedRows[key] ?? false
922
- const subtasks = modelResult.score_details.details
923
  ? Object.entries(modelResult.score_details.details).filter(([, value]) => typeof value === "number")
924
  : []
925
  const hasExpandableDetails =
926
  isResearchView ||
927
  (modelResult.aggregate_components && modelResult.aggregate_components.length > 1) ||
928
- subtasks.length > 1
929
 
930
  const datasetName = Array.isArray(modelResult.source_data)
931
  ? undefined
@@ -1256,28 +1256,28 @@ export function EvalDetail({ summary }: EvalDetailProps) {
1256
  </div>
1257
  )}
1258
 
1259
- {subtasks.length > 1 && (
1260
  <div className="space-y-2">
1261
  <div
1262
  className="font-mono uppercase"
1263
  style={{ fontSize: 10, letterSpacing: "0.14em", color: "var(--fg-subtle)" }}
1264
  >
1265
- Subtask breakdown
1266
  </div>
1267
  <div className="overflow-x-auto" style={{ border: "1px solid var(--border-soft)" }}>
1268
  <table className="ec-htable">
1269
  <thead>
1270
  <tr>
1271
- <th>Subtask</th>
1272
  <th className="num">Raw</th>
1273
  </tr>
1274
  </thead>
1275
  <tbody>
1276
- {subtasks.map(([subtaskName, value]) => {
1277
  const numericValue = value as number
1278
  return (
1279
- <tr key={subtaskName}>
1280
- <td className="font-medium text-[13px] capitalize">{subtaskName.replace(/_/g, " ")}</td>
1281
  <td className="num font-mono tabular-nums text-[13px]" style={{ color: "var(--fg-muted)" }}>
1282
  {formatRawScore(numericValue, summary.metric_config.unit)}
1283
  </td>
@@ -1453,7 +1453,7 @@ function MultiMetricLeaderboard({
1453
  return root?.column_key ?? metrics[0]?.column_key ?? "model"
1454
  })
1455
  const [sortDirection, setSortDirection] = useState<"asc" | "desc">("desc")
1456
- const [activeSubtaskTab, setActiveSubtaskTab] = useState<string>("all")
1457
  const [minParamStep, setMinParamStep] = useState(0)
1458
  const [maxParamStep, setMaxParamStep] = useState(PARAM_RANGE_MAX_INDEX)
1459
  const [expandedRows, setExpandedRows] = useState<Record<string, boolean>>({})
@@ -1486,7 +1486,7 @@ function MultiMetricLeaderboard({
1486
  const leaderboardRows = summary.leaderboard_rows ?? []
1487
  const allMetricKeys = useMemo(() => leaderboardMetrics.map((metric) => metric.column_key), [leaderboardMetrics])
1488
  // Cap default visible columns to avoid hangs on benchmarks with hundreds of metrics
1489
- // (e.g. helm_air_bench has 374 subtask×metric pairs). Users can opt in to more.
1490
  const DEFAULT_VISIBLE_METRIC_CAP = 24
1491
  const defaultVisibleMetricKeys = useMemo(
1492
  () => allMetricKeys.slice(0, DEFAULT_VISIBLE_METRIC_CAP),
@@ -1498,7 +1498,7 @@ function MultiMetricLeaderboard({
1498
  [leaderboardMetrics]
1499
  )
1500
  const visibleMetricKeySet = useMemo(() => new Set(visibleMetricKeys), [visibleMetricKeys])
1501
- const subtaskMetricCounts = useMemo(() => {
1502
  const counts = new Map<string, number>()
1503
  for (const metric of leaderboardMetrics) {
1504
  if (metric.scope === "subtask" && metric.subtask_key) {
@@ -1508,16 +1508,16 @@ function MultiMetricLeaderboard({
1508
  return counts
1509
  }, [leaderboardMetrics])
1510
 
1511
- const singleMetricSubtaskTabs = useMemo(() => {
1512
  return leaderboardMetrics
1513
- .filter((metric) => metric.scope === "subtask" && metric.subtask_key && subtaskMetricCounts.get(metric.subtask_key) === 1)
1514
  .map((metric) => ({
1515
  key: metric.subtask_key as string,
1516
  label: metric.subtask_name ?? getCompactMetricLabel(metric.display_name),
1517
  }))
1518
- }, [leaderboardMetrics, subtaskMetricCounts])
1519
 
1520
- const hasSubtaskTabs = singleMetricSubtaskTabs.length > 1
1521
 
1522
  const visibleMetrics = useMemo(
1523
  () =>
@@ -1526,13 +1526,13 @@ function MultiMetricLeaderboard({
1526
  return false
1527
  }
1528
 
1529
- if (!hasSubtaskTabs || activeSubtaskTab === "all") {
1530
  return true
1531
  }
1532
 
1533
- return metric.scope === "subtask" && metric.subtask_key === activeSubtaskTab
1534
  }),
1535
- [activeSubtaskTab, hasSubtaskTabs, leaderboardMetrics, visibleMetricKeySet]
1536
  )
1537
  const visibleMetricColumnKeySet = useMemo(
1538
  () => new Set(visibleMetrics.map((metric) => metric.column_key)),
@@ -1628,7 +1628,7 @@ function MultiMetricLeaderboard({
1628
  }, [defaultVisibleMetricKeys, summary.evaluation_id])
1629
 
1630
  useEffect(() => {
1631
- setActiveSubtaskTab("all")
1632
  }, [summary.evaluation_id])
1633
 
1634
  useEffect(() => {
@@ -1648,21 +1648,21 @@ function MultiMetricLeaderboard({
1648
  }, [leaderboardMetricMap, leaderboardMetrics, sortKey, visibleMetricColumnKeySet])
1649
 
1650
  useEffect(() => {
1651
- if (!hasSubtaskTabs) {
1652
- if (activeSubtaskTab !== "all") {
1653
- setActiveSubtaskTab("all")
1654
  }
1655
  return
1656
  }
1657
 
1658
- if (activeSubtaskTab === "all") {
1659
  return
1660
  }
1661
 
1662
- if (!singleMetricSubtaskTabs.some((tab) => tab.key === activeSubtaskTab)) {
1663
- setActiveSubtaskTab("all")
1664
  }
1665
- }, [activeSubtaskTab, hasSubtaskTabs, singleMetricSubtaskTabs])
1666
 
1667
  const pagedRows = useMemo(
1668
  () => sortedRows.slice(0, page * 50),
@@ -1761,7 +1761,7 @@ function MultiMetricLeaderboard({
1761
  {leaderboardMetrics.map((metric) => {
1762
  const isVisible = visibleMetricKeySet.has(metric.column_key)
1763
  const isLastVisible = isVisible && visibleMetrics.length === 1
1764
- const visibleLabel = metric.scope === "subtask" && metric.subtask_key && subtaskMetricCounts.get(metric.subtask_key) === 1 && metric.subtask_name
1765
  ? metric.subtask_name
1766
  : getCompactMetricLabel(metric.display_name)
1767
 
@@ -1793,7 +1793,7 @@ function MultiMetricLeaderboard({
1793
  .filter((v): v is number => isNumericScore(v))
1794
  if (values.length < 3) return null
1795
  const label =
1796
- metric.scope === "subtask" && metric.subtask_key && subtaskMetricCounts.get(metric.subtask_key) === 1 && metric.subtask_name
1797
  ? metric.subtask_name
1798
  : getCompactMetricLabel(metric.display_name)
1799
  return {
@@ -1816,12 +1816,12 @@ function MultiMetricLeaderboard({
1816
  })()}
1817
 
1818
  <div className="ec-card" style={{ padding: 0, overflow: "hidden" }}>
1819
- {hasSubtaskTabs && (
1820
  <div className="border-b bg-background px-5 py-3 sm:px-6">
1821
  <SliceSelector
1822
- activeSubtaskTab={activeSubtaskTab}
1823
- onChange={setActiveSubtaskTab}
1824
- tabs={singleMetricSubtaskTabs}
1825
  />
1826
  </div>
1827
  )}
@@ -1868,13 +1868,13 @@ function MultiMetricLeaderboard({
1868
  {getSortIndicator("developer")}
1869
  </th>
1870
  {visibleMetrics.map((metric) => {
1871
- const showSubtaskTopline =
1872
- !hasSubtaskTabs &&
1873
- !(metric.scope === "subtask" && metric.subtask_key && subtaskMetricCounts.get(metric.subtask_key) === 1) &&
1874
  metric.scope === "subtask" &&
1875
  metric.subtask_name
1876
  const mainLabel =
1877
- metric.scope === "subtask" && metric.subtask_key && subtaskMetricCounts.get(metric.subtask_key) === 1 && metric.subtask_name
1878
  ? metric.subtask_name
1879
  : getCompactMetricLabel(metric.display_name)
1880
  return (
@@ -1885,7 +1885,7 @@ function MultiMetricLeaderboard({
1885
  onClick={() => handleSort(metric.column_key)}
1886
  title={describeLeaderboardMetric(metric)}
1887
  >
1888
- {showSubtaskTopline && (
1889
  <div
1890
  className="font-mono normal-case"
1891
  style={{
 
113
  * - <= SLICE_PILL_THRESHOLD: render every slice as a pill (current familiar UX).
114
  * - > SLICE_PILL_THRESHOLD: render "All slices" + currently-selected pill +
115
  * a "Browse N slices" button that opens a searchable dialog. Hundreds of
116
+ * slices (e.g. AIRBench's 374) fit cleanly.
117
  */
118
  function SliceSelector({
119
+ activeSliceTab,
120
  onChange,
121
  tabs,
122
  }: {
123
+ activeSliceTab: string
124
  onChange: (key: string) => void
125
  tabs: SliceTab[]
126
  }) {
 
128
  const [search, setSearch] = useState("")
129
 
130
  const useBrowser = tabs.length > SLICE_PILL_THRESHOLD
131
+ const activeTab = tabs.find((tab) => tab.key === activeSliceTab)
132
 
133
  const filteredTabs = useMemo(() => {
134
  const query = search.trim().toLowerCase()
 
143
  <div className="flex flex-wrap gap-2">
144
  <button
145
  type="button"
146
+ className={`ec-pill${activeSliceTab === "all" ? " on" : ""}`}
147
  onClick={() => onChange("all")}
148
  >
149
  All slices
 
152
  <button
153
  key={tab.key}
154
  type="button"
155
+ className={`ec-pill${activeSliceTab === tab.key ? " on" : ""}`}
156
  onClick={() => onChange(tab.key)}
157
  >
158
  {tab.label}
 
172
  <div className="flex flex-wrap items-center gap-2">
173
  <button
174
  type="button"
175
+ className={`ec-pill${activeSliceTab === "all" ? " on" : ""}`}
176
  onClick={() => onChange("all")}
177
  >
178
  All slices
 
230
  }}
231
  className={cn(
232
  "flex w-full items-center justify-between border-b px-4 py-2.5 text-left text-sm transition-colors hover:bg-muted/40",
233
+ activeSliceTab === "all" && "bg-muted/40 font-semibold"
234
  )}
235
  >
236
  <span>All slices (no filter)</span>
237
+ {activeSliceTab === "all" && <span className="text-xs text-muted-foreground">selected</span>}
238
  </button>
239
  {filteredTabs.length === 0 ? (
240
  <div className="px-4 py-6 text-center text-sm text-muted-foreground">
 
251
  }}
252
  className={cn(
253
  "flex w-full items-center justify-between border-b px-4 py-2 text-left text-sm transition-colors hover:bg-muted/40 last:border-b-0",
254
+ activeSliceTab === tab.key && "bg-muted/40 font-semibold"
255
  )}
256
  >
257
  <span className="min-w-0 truncate pr-2">{tab.label}</span>
258
+ {activeSliceTab === tab.key && (
259
  <span className="shrink-0 text-xs text-muted-foreground">selected</span>
260
  )}
261
  </button>
 
543
  : "Not linked"
544
  const leaderboardDescription = isResearchView
545
  ? summary.is_aggregated
546
+ ? "Models ranked by average raw score across the composite's component benchmarks."
547
  : "Models ranked by raw score for this benchmark."
548
  : summary.is_aggregated
549
+ ? "Averaged model results across the composite's component benchmarks, with drill-down to each component score."
550
  : "Model results with benchmark context, source dataset detail, and optional instance-data links."
551
  const reportingCompleteness = summary.evalcards?.annotations?.reporting_completeness
552
  const documentationPopulatedCount = reportingCompleteness
 
656
  className="font-mono text-[10px] uppercase tracking-[0.12em]"
657
  style={{ color: "var(--fg-subtle)" }}
658
  >
659
+ metric spec · completeness · comparability{summary.subtasks?.length ? " · slices" : ""}
660
  </span>
661
  </div>
662
  {overviewOpen ? (
 
678
  {isResearchView ? "Metric specification" : "Reading context"}
679
  </div>
680
  <dl className="ec-datalist">
681
+ <dt>Composite</dt>
682
  <dd>
683
  {summary.is_aggregated
684
+ ? summary.aggregate_sources?.map((source) => source.composite_benchmark_name).join(", ") || "Multiple composites"
685
  : summary.composite_benchmark_name}
686
  </dd>
687
  <dt>{isResearchView ? "Benchmark ID" : "What this covers"}</dt>
 
744
  >
745
  <div className="kicker mb-2">Benchmark structure</div>
746
  <p className="text-[13px] mb-4" style={{ color: "var(--fg-muted)", maxWidth: 640 }}>
747
+ Benchmark-level summary metrics and slices grouped in one compact section.
748
  </p>
749
 
750
  {summary.root_metrics && summary.root_metrics.length > 0 && (
 
776
  className="font-mono uppercase mb-1"
777
  style={{ fontSize: 10, letterSpacing: "0.14em", color: "var(--fg-subtle)" }}
778
  >
779
+ Slice breakdown · {summary.subtasks.length}
780
  </div>
781
  <ul
782
  className="flex flex-col"
783
  style={{ borderTop: "1px solid var(--border-soft)" }}
784
  >
785
+ {summary.subtasks.map((slice) => (
786
  <li
787
+ key={slice.subtask_key}
788
  className="grid gap-x-4 py-3"
789
  style={{
790
  gridTemplateColumns: "minmax(160px, 280px) 1fr",
 
793
  >
794
  <div className="min-w-0">
795
  <div className="font-semibold text-[13px] truncate">
796
+ {slice.display_name || slice.subtask_name}
797
  </div>
798
+ {slice.canonical_display_name && slice.canonical_display_name !== (slice.display_name || slice.subtask_name) && (
799
  <div
800
  className="mt-0.5 font-mono text-[10px] uppercase tracking-[0.08em] truncate"
801
  style={{ color: "var(--fg-subtle)" }}
802
+ title={slice.canonical_display_name}
803
  >
804
+ {slice.canonical_display_name}
805
  </div>
806
  )}
807
  </div>
808
  <div className="flex flex-wrap items-center gap-1.5">
809
+ {slice.metrics.map((metric) => (
810
  <span
811
  key={metric.metric_summary_id}
812
  className="ec-tag"
 
919
  <tbody>
920
  {pagedLeaderboardRows.map(({ key, rank, modelResult, normalizedScore }) => {
921
  const isExpanded = expandedRows[key] ?? false
922
+ const slices = modelResult.score_details.details
923
  ? Object.entries(modelResult.score_details.details).filter(([, value]) => typeof value === "number")
924
  : []
925
  const hasExpandableDetails =
926
  isResearchView ||
927
  (modelResult.aggregate_components && modelResult.aggregate_components.length > 1) ||
928
+ slices.length > 1
929
 
930
  const datasetName = Array.isArray(modelResult.source_data)
931
  ? undefined
 
1256
  </div>
1257
  )}
1258
 
1259
+ {slices.length > 1 && (
1260
  <div className="space-y-2">
1261
  <div
1262
  className="font-mono uppercase"
1263
  style={{ fontSize: 10, letterSpacing: "0.14em", color: "var(--fg-subtle)" }}
1264
  >
1265
+ Slice breakdown
1266
  </div>
1267
  <div className="overflow-x-auto" style={{ border: "1px solid var(--border-soft)" }}>
1268
  <table className="ec-htable">
1269
  <thead>
1270
  <tr>
1271
+ <th>Slice</th>
1272
  <th className="num">Raw</th>
1273
  </tr>
1274
  </thead>
1275
  <tbody>
1276
+ {slices.map(([sliceName, value]) => {
1277
  const numericValue = value as number
1278
  return (
1279
+ <tr key={sliceName}>
1280
+ <td className="font-medium text-[13px] capitalize">{sliceName.replace(/_/g, " ")}</td>
1281
  <td className="num font-mono tabular-nums text-[13px]" style={{ color: "var(--fg-muted)" }}>
1282
  {formatRawScore(numericValue, summary.metric_config.unit)}
1283
  </td>
 
1453
  return root?.column_key ?? metrics[0]?.column_key ?? "model"
1454
  })
1455
  const [sortDirection, setSortDirection] = useState<"asc" | "desc">("desc")
1456
+ const [activeSliceTab, setActiveSliceTab] = useState<string>("all")
1457
  const [minParamStep, setMinParamStep] = useState(0)
1458
  const [maxParamStep, setMaxParamStep] = useState(PARAM_RANGE_MAX_INDEX)
1459
  const [expandedRows, setExpandedRows] = useState<Record<string, boolean>>({})
 
1486
  const leaderboardRows = summary.leaderboard_rows ?? []
1487
  const allMetricKeys = useMemo(() => leaderboardMetrics.map((metric) => metric.column_key), [leaderboardMetrics])
1488
  // Cap default visible columns to avoid hangs on benchmarks with hundreds of metrics
1489
+ // (e.g. helm_air_bench has 374 slice×metric pairs). Users can opt in to more.
1490
  const DEFAULT_VISIBLE_METRIC_CAP = 24
1491
  const defaultVisibleMetricKeys = useMemo(
1492
  () => allMetricKeys.slice(0, DEFAULT_VISIBLE_METRIC_CAP),
 
1498
  [leaderboardMetrics]
1499
  )
1500
  const visibleMetricKeySet = useMemo(() => new Set(visibleMetricKeys), [visibleMetricKeys])
1501
+ const sliceMetricCounts = useMemo(() => {
1502
  const counts = new Map<string, number>()
1503
  for (const metric of leaderboardMetrics) {
1504
  if (metric.scope === "subtask" && metric.subtask_key) {
 
1508
  return counts
1509
  }, [leaderboardMetrics])
1510
 
1511
+ const singleMetricSliceTabs = useMemo(() => {
1512
  return leaderboardMetrics
1513
+ .filter((metric) => metric.scope === "subtask" && metric.subtask_key && sliceMetricCounts.get(metric.subtask_key) === 1)
1514
  .map((metric) => ({
1515
  key: metric.subtask_key as string,
1516
  label: metric.subtask_name ?? getCompactMetricLabel(metric.display_name),
1517
  }))
1518
+ }, [leaderboardMetrics, sliceMetricCounts])
1519
 
1520
+ const hasSliceTabs = singleMetricSliceTabs.length > 1
1521
 
1522
  const visibleMetrics = useMemo(
1523
  () =>
 
1526
  return false
1527
  }
1528
 
1529
+ if (!hasSliceTabs || activeSliceTab === "all") {
1530
  return true
1531
  }
1532
 
1533
+ return metric.scope === "subtask" && metric.subtask_key === activeSliceTab
1534
  }),
1535
+ [activeSliceTab, hasSliceTabs, leaderboardMetrics, visibleMetricKeySet]
1536
  )
1537
  const visibleMetricColumnKeySet = useMemo(
1538
  () => new Set(visibleMetrics.map((metric) => metric.column_key)),
 
1628
  }, [defaultVisibleMetricKeys, summary.evaluation_id])
1629
 
1630
  useEffect(() => {
1631
+ setActiveSliceTab("all")
1632
  }, [summary.evaluation_id])
1633
 
1634
  useEffect(() => {
 
1648
  }, [leaderboardMetricMap, leaderboardMetrics, sortKey, visibleMetricColumnKeySet])
1649
 
1650
  useEffect(() => {
1651
+ if (!hasSliceTabs) {
1652
+ if (activeSliceTab !== "all") {
1653
+ setActiveSliceTab("all")
1654
  }
1655
  return
1656
  }
1657
 
1658
+ if (activeSliceTab === "all") {
1659
  return
1660
  }
1661
 
1662
+ if (!singleMetricSliceTabs.some((tab) => tab.key === activeSliceTab)) {
1663
+ setActiveSliceTab("all")
1664
  }
1665
+ }, [activeSliceTab, hasSliceTabs, singleMetricSliceTabs])
1666
 
1667
  const pagedRows = useMemo(
1668
  () => sortedRows.slice(0, page * 50),
 
1761
  {leaderboardMetrics.map((metric) => {
1762
  const isVisible = visibleMetricKeySet.has(metric.column_key)
1763
  const isLastVisible = isVisible && visibleMetrics.length === 1
1764
+ const visibleLabel = metric.scope === "subtask" && metric.subtask_key && sliceMetricCounts.get(metric.subtask_key) === 1 && metric.subtask_name
1765
  ? metric.subtask_name
1766
  : getCompactMetricLabel(metric.display_name)
1767
 
 
1793
  .filter((v): v is number => isNumericScore(v))
1794
  if (values.length < 3) return null
1795
  const label =
1796
+ metric.scope === "subtask" && metric.subtask_key && sliceMetricCounts.get(metric.subtask_key) === 1 && metric.subtask_name
1797
  ? metric.subtask_name
1798
  : getCompactMetricLabel(metric.display_name)
1799
  return {
 
1816
  })()}
1817
 
1818
  <div className="ec-card" style={{ padding: 0, overflow: "hidden" }}>
1819
+ {hasSliceTabs && (
1820
  <div className="border-b bg-background px-5 py-3 sm:px-6">
1821
  <SliceSelector
1822
+ activeSliceTab={activeSliceTab}
1823
+ onChange={setActiveSliceTab}
1824
+ tabs={singleMetricSliceTabs}
1825
  />
1826
  </div>
1827
  )}
 
1868
  {getSortIndicator("developer")}
1869
  </th>
1870
  {visibleMetrics.map((metric) => {
1871
+ const showSliceTopline =
1872
+ !hasSliceTabs &&
1873
+ !(metric.scope === "subtask" && metric.subtask_key && sliceMetricCounts.get(metric.subtask_key) === 1) &&
1874
  metric.scope === "subtask" &&
1875
  metric.subtask_name
1876
  const mainLabel =
1877
+ metric.scope === "subtask" && metric.subtask_key && sliceMetricCounts.get(metric.subtask_key) === 1 && metric.subtask_name
1878
  ? metric.subtask_name
1879
  : getCompactMetricLabel(metric.display_name)
1880
  return (
 
1885
  onClick={() => handleSort(metric.column_key)}
1886
  title={describeLeaderboardMetric(metric)}
1887
  >
1888
+ {showSliceTopline && (
1889
  <div
1890
  className="font-mono normal-case"
1891
  style={{
components/family-table.tsx CHANGED
@@ -4,7 +4,7 @@ import { Fragment, useMemo, useState } from "react"
4
  import { useRouter } from "next/navigation"
5
  import { ArrowUpRight, ChevronDown, ChevronRight } from "lucide-react"
6
 
7
- import type { HierarchyFamily, HierarchyLeaf } from "@/lib/backend-artifacts"
8
  import type { BenchmarkCard, CategoryType } from "@/lib/benchmark-schema"
9
  import type { BenchmarkEvalListItem } from "@/lib/eval-processing"
10
 
@@ -67,18 +67,60 @@ interface LeafEntry {
67
  domains: string[]
68
  }
69
 
 
 
 
 
 
 
 
 
 
 
 
 
70
  function collectLeafEntries(
71
  fam: HierarchyFamily,
72
  benchmarkCards?: Record<string, BenchmarkCard>,
73
  ): LeafEntry[] {
74
  const out: LeafEntry[] = []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
  for (const leaf of fam.leaves ?? []) {
76
- // Backends differ in whether leaves carry an explicit
77
- // `eval_summary_ids` array. When absent, fall back to the
78
- // pipeline's standard `${fam.key}_${leaf.key}` naming, then to the
79
- // bare leaf key — both are stable enough for the detail page to
80
- // resolve. This stops the inline benchmarks grid from disappearing
81
- // on a backend that ships hierarchy.json without leaf eval ids.
82
  const explicit = leaf.eval_summary_ids ?? []
83
  const ids =
84
  explicit.length > 0
@@ -87,10 +129,6 @@ function collectLeafEntries(
87
  ? [`${fam.key}_${leaf.key}`, leaf.key]
88
  : []
89
  if (ids.length === 0) continue
90
- // Domain sources, in order of trust:
91
- // (1) hierarchy `leaf.tags.domains` — sometimes absent
92
- // (2) benchmark-metadata keyed by leaf.key
93
- // (3) benchmark-metadata keyed by the leaf's eval_summary_id
94
  const collected = new Set<string>()
95
  for (const d of leaf.tags?.domains ?? []) collected.add(d.toLowerCase())
96
  const cardByLeaf = benchmarkCards?.[leaf.key]
@@ -107,6 +145,7 @@ function collectLeafEntries(
107
  domains: Array.from(collected),
108
  })
109
  }
 
110
  return out
111
  }
112
 
 
4
  import { useRouter } from "next/navigation"
5
  import { ArrowUpRight, ChevronDown, ChevronRight } from "lucide-react"
6
 
7
+ import type { HierarchyBenchmark, HierarchyFamily, HierarchyLeaf } from "@/lib/backend-artifacts"
8
  import type { BenchmarkCard, CategoryType } from "@/lib/benchmark-schema"
9
  import type { BenchmarkEvalListItem } from "@/lib/eval-processing"
10
 
 
67
  domains: string[]
68
  }
69
 
70
+ /**
71
+ * Build the per-row list of expandable benchmarks under a family.
72
+ *
73
+ * Primary path: the v2 production shape, where the hierarchy nests
74
+ * benchmarks under `families[].composites[].benchmarks[]` (plus
75
+ * `standalone_benchmarks[]` and the family-level `benchmarks[]`). Each
76
+ * benchmark carries its own `summary_eval_ids` for navigation.
77
+ *
78
+ * Fallback path: the legacy `families[].leaves[]` shape used by older
79
+ * snapshots. Kept for compatibility while older caches are still in
80
+ * circulation.
81
+ */
82
  function collectLeafEntries(
83
  fam: HierarchyFamily,
84
  benchmarkCards?: Record<string, BenchmarkCard>,
85
  ): LeafEntry[] {
86
  const out: LeafEntry[] = []
87
+
88
+ // ── Primary: nested benchmarks (v2) ────────────────────────────────
89
+ const nested: HierarchyBenchmark[] = [
90
+ ...(fam.standalone_benchmarks ?? []),
91
+ ...(fam.benchmarks ?? []),
92
+ ...(fam.composites ?? []).flatMap((c) => c.benchmarks ?? []),
93
+ ]
94
+ for (const benchmark of nested) {
95
+ const summaryIds = benchmark.summary_eval_ids ?? []
96
+ const ids =
97
+ summaryIds.length > 0
98
+ ? summaryIds
99
+ : benchmark.key
100
+ ? [`${fam.key}_${benchmark.key}`, benchmark.key]
101
+ : []
102
+ if (ids.length === 0) continue
103
+ const collected = new Set<string>()
104
+ for (const d of benchmark.tags?.domains ?? []) collected.add(d.toLowerCase())
105
+ const cardByKey = benchmarkCards?.[benchmark.key]
106
+ for (const d of cardByKey?.benchmark_details?.domains ?? []) collected.add(d.toLowerCase())
107
+ for (const id of ids) {
108
+ const cardById = benchmarkCards?.[id]
109
+ for (const d of cardById?.benchmark_details?.domains ?? []) collected.add(d.toLowerCase())
110
+ }
111
+ out.push({
112
+ id: ids[0],
113
+ leafKey: benchmark.key,
114
+ leafName: benchmark.display_name || benchmark.key,
115
+ evalsCount: ids.length,
116
+ domains: Array.from(collected),
117
+ })
118
+ }
119
+
120
+ if (out.length > 0) return out
121
+
122
+ // ── Fallback: legacy `leaves` shape ────────────────────────────────
123
  for (const leaf of fam.leaves ?? []) {
 
 
 
 
 
 
124
  const explicit = leaf.eval_summary_ids ?? []
125
  const ids =
126
  explicit.length > 0
 
129
  ? [`${fam.key}_${leaf.key}`, leaf.key]
130
  : []
131
  if (ids.length === 0) continue
 
 
 
 
132
  const collected = new Set<string>()
133
  for (const d of leaf.tags?.domains ?? []) collected.add(d.toLowerCase())
134
  const cardByLeaf = benchmarkCards?.[leaf.key]
 
145
  domains: Array.from(collected),
146
  })
147
  }
148
+
149
  return out
150
  }
151
 
components/model-compare-dialog.tsx CHANGED
@@ -428,7 +428,7 @@ export function ModelCompareDialog({
428
  ) : null}
429
  {row.key === "latest" ? (
430
  <div className="flex items-center gap-2">
431
- <span>{model.latest_source_name || `${model.benchmarks_count} benchmark suites summarized`}</span>
432
  {model.source_urls[0] ? (
433
  <a
434
  href={model.source_urls[0]}
 
428
  ) : null}
429
  {row.key === "latest" ? (
430
  <div className="flex items-center gap-2">
431
+ <span>{model.latest_source_name || `${model.benchmarks_count} benchmark composites summarized`}</span>
432
  {model.source_urls[0] ? (
433
  <a
434
  href={model.source_urls[0]}
components/policy-overview.tsx CHANGED
@@ -51,7 +51,7 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
51
  const card = summary.benchmark_card
52
 
53
  // Defensive check: pipelines older than the "ancestor card leak" fix
54
- // sometimes attach the parent suite's card to a leaf benchmark (e.g.
55
  // helm_classic's card embedded under XSUM). Detect when the card's own
56
  // name is clearly not this benchmark and ignore its narrative text — the
57
  // synthesized fallback below produces something accurate instead.
@@ -81,17 +81,20 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
81
  const goal = cardMatchesEval ? card?.purpose_and_intended_users?.goal?.trim() || "" : ""
82
 
83
  // Detect "parent" benchmark pages — either an aggregated composite or a
84
- // multi-metric matrix where each column is a subtask. In both cases the
85
  // per-evaluation `metric_config.evaluation_description` belongs to whichever
86
  // component was processed first (e.g. just the "airline" subset of Tau
87
  // Bench 2) and would mislead a policy reader. Synthesize parent framing
88
- // instead and surface the subtasks separately.
 
 
 
89
  const isAggregated = summary.is_aggregated === true
90
  const aggregateNames = (summary.aggregate_sources ?? [])
91
  .map((s) => s.composite_benchmark_name)
92
  .filter((s): s is string => typeof s === "string" && s.length > 0)
93
 
94
- const subtaskLabels = useMemo(() => {
95
  const seen = new Set<string>()
96
  const labels: string[] = []
97
  const add = (raw: string | undefined | null) => {
@@ -103,8 +106,8 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
103
  seen.add(key)
104
  labels.push(trimmed)
105
  }
106
- for (const subtask of summary.subtasks ?? []) {
107
- add(subtask.display_name || subtask.subtask_name)
108
  }
109
  for (const metric of summary.leaderboard_metrics ?? []) {
110
  if (metric.scope === "subtask") {
@@ -116,14 +119,14 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
116
  }, [summary.subtasks, summary.leaderboard_metrics, aggregateNames])
117
 
118
  const isMatrix = (summary.leaderboard_metrics?.length ?? 0) > 1
119
- const isParentPage = isAggregated || (isMatrix && subtaskLabels.length > 1)
120
  const useComponentDescription = !isParentPage
121
 
122
- const parentFallback = isParentPage && subtaskLabels.length > 1
123
- ? `${summary.evaluation_name} reports results across ${subtaskLabels.length} ${
124
- isAggregated ? "component benchmarks" : "subtasks"
125
  }. Each is evaluated separately; the score shown is the ${
126
- isAggregated ? "average" : "per-subtask result"
127
  }.`
128
  : null
129
 
@@ -134,7 +137,7 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
134
  (useComponentDescription ? summary.metric_config.evaluation_description : summary.evaluation_name)
135
 
136
  const [expanded, setExpanded] = useState(false)
137
- const [subtasksOpen, setSubtasksOpen] = useState(false)
138
  const isLong = summaryText.length > SUMMARY_PREVIEW_CHARS
139
  const visibleText = expanded || !isLong
140
  ? summaryText
@@ -299,15 +302,15 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
299
  </dd>
300
  </dl>
301
 
302
- {isParentPage && subtaskLabels.length > 1 && (
303
  <div
304
  className="mt-4"
305
  style={{ border: "1px solid var(--border-soft)", background: "var(--bg)" }}
306
  >
307
  <button
308
  type="button"
309
- onClick={() => setSubtasksOpen((v) => !v)}
310
- aria-expanded={subtasksOpen}
311
  className="flex w-full items-center justify-between gap-3 px-3.5 py-2.5 text-left transition-colors hover:bg-[color:var(--bg-warm)]"
312
  >
313
  <span
@@ -315,20 +318,20 @@ export function PolicyOverview({ summary }: PolicyOverviewProps) {
315
  style={{ fontSize: 10, letterSpacing: "0.14em", color: "var(--fg-muted)" }}
316
  >
317
  <Layers className="h-3.5 w-3.5" />
318
- {isAggregated ? `Component benchmarks (${subtaskLabels.length})` : `Subtasks (${subtaskLabels.length})`}
319
  </span>
320
- {subtasksOpen ? (
321
  <ChevronUp className="h-4 w-4" style={{ color: "var(--fg-muted)" }} />
322
  ) : (
323
  <ChevronDown className="h-4 w-4" style={{ color: "var(--fg-muted)" }} />
324
  )}
325
  </button>
326
- {subtasksOpen && (
327
  <ul
328
  className="grid list-disc gap-x-6 gap-y-1 px-3.5 pb-3.5 pl-9 text-[13px] sm:grid-cols-2 lg:grid-cols-3"
329
  style={{ color: "var(--fg)" }}
330
  >
331
- {subtaskLabels.map((name) => (
332
  <li key={name} className="capitalize">
333
  {name}
334
  </li>
 
51
  const card = summary.benchmark_card
52
 
53
  // Defensive check: pipelines older than the "ancestor card leak" fix
54
+ // sometimes attach the parent composite's card to a leaf benchmark (e.g.
55
  // helm_classic's card embedded under XSUM). Detect when the card's own
56
  // name is clearly not this benchmark and ignore its narrative text — the
57
  // synthesized fallback below produces something accurate instead.
 
81
  const goal = cardMatchesEval ? card?.purpose_and_intended_users?.goal?.trim() || "" : ""
82
 
83
  // Detect "parent" benchmark pages — either an aggregated composite or a
84
+ // multi-metric matrix where each column is a slice. In both cases the
85
  // per-evaluation `metric_config.evaluation_description` belongs to whichever
86
  // component was processed first (e.g. just the "airline" subset of Tau
87
  // Bench 2) and would mislead a policy reader. Synthesize parent framing
88
+ // instead and surface the slices separately.
89
+ // Note: at the data layer the backend still ships these as `subtasks` /
90
+ // `metric.scope === "subtask"`; we read those fields directly but label
91
+ // them "slice" in the UI.
92
  const isAggregated = summary.is_aggregated === true
93
  const aggregateNames = (summary.aggregate_sources ?? [])
94
  .map((s) => s.composite_benchmark_name)
95
  .filter((s): s is string => typeof s === "string" && s.length > 0)
96
 
97
+ const sliceLabels = useMemo(() => {
98
  const seen = new Set<string>()
99
  const labels: string[] = []
100
  const add = (raw: string | undefined | null) => {
 
106
  seen.add(key)
107
  labels.push(trimmed)
108
  }
109
+ for (const slice of summary.subtasks ?? []) {
110
+ add(slice.display_name || slice.subtask_name)
111
  }
112
  for (const metric of summary.leaderboard_metrics ?? []) {
113
  if (metric.scope === "subtask") {
 
119
  }, [summary.subtasks, summary.leaderboard_metrics, aggregateNames])
120
 
121
  const isMatrix = (summary.leaderboard_metrics?.length ?? 0) > 1
122
+ const isParentPage = isAggregated || (isMatrix && sliceLabels.length > 1)
123
  const useComponentDescription = !isParentPage
124
 
125
+ const parentFallback = isParentPage && sliceLabels.length > 1
126
+ ? `${summary.evaluation_name} reports results across ${sliceLabels.length} ${
127
+ isAggregated ? "component benchmarks" : "slices"
128
  }. Each is evaluated separately; the score shown is the ${
129
+ isAggregated ? "average" : "per-slice result"
130
  }.`
131
  : null
132
 
 
137
  (useComponentDescription ? summary.metric_config.evaluation_description : summary.evaluation_name)
138
 
139
  const [expanded, setExpanded] = useState(false)
140
+ const [slicesOpen, setSlicesOpen] = useState(false)
141
  const isLong = summaryText.length > SUMMARY_PREVIEW_CHARS
142
  const visibleText = expanded || !isLong
143
  ? summaryText
 
302
  </dd>
303
  </dl>
304
 
305
+ {isParentPage && sliceLabels.length > 1 && (
306
  <div
307
  className="mt-4"
308
  style={{ border: "1px solid var(--border-soft)", background: "var(--bg)" }}
309
  >
310
  <button
311
  type="button"
312
+ onClick={() => setSlicesOpen((v) => !v)}
313
+ aria-expanded={slicesOpen}
314
  className="flex w-full items-center justify-between gap-3 px-3.5 py-2.5 text-left transition-colors hover:bg-[color:var(--bg-warm)]"
315
  >
316
  <span
 
318
  style={{ fontSize: 10, letterSpacing: "0.14em", color: "var(--fg-muted)" }}
319
  >
320
  <Layers className="h-3.5 w-3.5" />
321
+ {isAggregated ? `Component benchmarks (${sliceLabels.length})` : `Slices (${sliceLabels.length})`}
322
  </span>
323
+ {slicesOpen ? (
324
  <ChevronUp className="h-4 w-4" style={{ color: "var(--fg-muted)" }} />
325
  ) : (
326
  <ChevronDown className="h-4 w-4" style={{ color: "var(--fg-muted)" }} />
327
  )}
328
  </button>
329
+ {slicesOpen && (
330
  <ul
331
  className="grid list-disc gap-x-6 gap-y-1 px-3.5 pb-3.5 pl-9 text-[13px] sm:grid-cols-2 lg:grid-cols-3"
332
  style={{ color: "var(--fg)" }}
333
  >
334
+ {sliceLabels.map((name) => (
335
  <li key={name} className="capitalize">
336
  {name}
337
  </li>