Map exact current anchored claims to existing independent evidence

.gitattributes

@@ -23,9 +23,8 @@
 *.pth filter=lfs diff=lfs merge=lfs -text
 *.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
 *.tar filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
 *.wasm filter=lfs diff=lfs merge=lfs -text
@@ -35,3 +34,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 artifacts/fair_calibrated_audit.png filter=lfs diff=lfs merge=lfs -text
 artifacts/paper_page_1.png filter=lfs diff=lfs merge=lfs -text
+outputs/fair_calibrated/fair_calibrated_audit.png filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text

BUNDLE_SHA256SUMS.txt

@@ -0,0 +1,26 @@
+cf8ad13101f5fdeb8a72e6bad260d7288433e1cce604b422ef690fa27148ca73  .gitattributes
+bb48a86f16da47864696a832dcd572f05e841fa86e99e3d3a210350d9aa0fd76  README.md
+72a86ae892ed54e321538fedc2e1e54c56051ea9cbf35563326c15a2770f0813  anchored_claim_map.json
+d6ab918002ed311f20fe9399eddc5d79dd0e670e16bd0b58e510e9d4def18c01  audit_fair_calibrated.py
+72bd5209c81f4175b80beebda06734bcaa07738400e7d1999ce71a28b579782a  boundary_trace.py
+d1c28fc0a4e07f2688d013f576cf76ffc422d278d56a52a82989e0b93b3b3964  bucket-icon.svg
+a24c2909ee44e76b7e6fef6001b24216e6ecfae39dfd1c485aef640dbd8592d3  index.html
+64e1de4358c79ec0d5f2697c56f98258c025e992c94ad7b3b7801739222ca41d  logbook.css
+69d73869184f936613668569980f31984be65229e77c4df4ba9604d3de70c02b  logbook.js
+2bf7e7c076558819131ed3c1e45d1ba351164dc266168f5d9a30408668b6a655  logbook.json
+2d637da431aa86aa6ac73fa22adcca77413f90da767e5a7f4dcebb62c016295d  official_claims.json
+6d00816bb3a7fca1874f5d2cd3f71fed7ba81e37f9a4e1b1367b6d7c72889663  outputs/fair_calibrated/SHA256SUMS.json
+f5b18a567f2c4c937d08f07cc00979ad7ccaa8e2c4fb188ab273ac6371846ae2  outputs/fair_calibrated/fair_calibrated_audit.png
+38fe728431e00ecbc265aad540a289ee57c17d32fe9091d592d702f7836778c3  outputs/fair_calibrated/fair_calibrated_results.json
+d8fee5e6a8a56373d905aa8402d2ead9d5098046c93b4d45994b5dc1fa6a8f97  pages/claim-0-current-anchored-claim-map/page.md
+82de900c3e55d03e9c9de511025d1b5b2dad877f8acdc04e01f752246ddf1933  pages/claim-1-group-calibrated-scores-including-true-class-probabilities-violate-predictive-parity-after-thresholding/page.md
+05e4642b596555bd36ad438898ce9cedb01ff9f0f578447e68f7416c12602609  pages/claim-2-presents-exact-solution-for-optimal-binary-randomized-classification-under-sufficiency-with-finite-group-calibrated-scores/page.md
+07d806dfb699e52b817f64f99c559152b5b5b6df1c15cecb54fc20ede91bee7f  pages/claim-3-geometric-characterization-identifies-optimal-classifier-minimizing-deviation-from-separation-subject-to-sufficiency/page.md
+afe2546838af8b520fd7b9384b0c54a156f5b584606f8f57193762d96c5e5ebe  pages/conclusion/page.md
+7c012d184309fc665cb1977e05a701ff45008d53b4f7ecfe0f1ba68088fe5f34  pages/executive-summary/page.md
+cc68a6f556f5a5efd0ed4bc12a40a2069e0b15f2409aeded7f7adb1bd0efbf9f  pages/index.md
+bf59354978d9e8a2fb84822a44d8fe6cfea17a98e56a5be8d0a7ec2f3e83e45a  poster_embed.html
+41f2ea6331528e454cc6ef1cf0de008f1368e18dfa9a684894f8a2a787d74fa8  requirements.txt
+a6eb72253c0128ce79b526a86b7943eed37beec186b5f57ff6c1701d0e9ff596  trackio-logo-light.png
+3e3792061d4d095759da30d7cfe7f14b621901793cd4d677b61b2896f5bf472b  trackio-logo.png
+71da94795855710d214801eb9b9b7b8898e9a8757abac0e22966a0531bbb2f4f  trackio-wordmark-dark.png

README.md

@@ -1,18 +1,20 @@
 ---
-title: "Repro - Fair Decisions from Calibrated Scores"
+title: Reproduction - Fair Decisions from Calibrated Scores
 emoji: ⚖️
-colorFrom: yellow
-colorTo: red
+colorFrom: purple
+colorTo: pink
 sdk: static
 pinned: false
+short_description: Exact sufficient classifiers and fairness geometry
 tags:
- - trackio
- - trackio-logbook
- - open-experiment
- - icml2026-repro
- - paper-XiD5RhcEDK
+  - trackio
+  - open-reproductions
+  - icml2026-repro
+  - paper-XiD5RhcEDK
 ---
 
-# Repro - Fair Decisions from Calibrated Scores
+# Independent reproduction
 
-Exact boundary and independent constrained-optimization audit for OpenReview paper `XiD5RhcEDK`.
+Independent reproduction of **Fair Decisions from Calibrated Scores**
+(OpenReview `XiD5RhcEDK`). Complete rules, independent optimizer outputs,
+negative controls, and SHA-256 manifests are included.

anchored_claim_map.json

@@ -0,0 +1,44 @@
+{
+  "schema_version": 1,
+  "paper_id": "XiD5RhcEDK",
+  "target_space": "ProCreations/repro-nanoquant-efficient-sub-1-bit-quantization-of-large-language-models",
+  "generated_at": "2026-07-22T21:51:50.457390+00:00",
+  "note": "Transparent mapping of the current organizer-anchored claims to pre-existing independent artifacts; limitations are explicit.",
+  "claims": [
+    {
+      "number": 1,
+      "claim": "Even perfectly group-calibrated scores, including true class-conditional probabilities, provably violate predictive parity after thresholding, per Definitions 2.1 and 2.3 of sufficiency and predictive parity (Section 2, Definitions 2.1, 2.3).",
+      "evidence_class": "direct independent execution",
+      "evidence": "Exact calibrated-score constructions compute post-threshold PPV/FOR by group and exhibit nonzero predictive-parity gaps despite perfect within-group calibration; matched equal-base-rate controls remove the gap.",
+      "primary_artifact": "outputs/fair_calibrated/fair_calibrated_results.json"
+    },
+    {
+      "number": 2,
+      "claim": "Theorem 3.3 shows that for a fixed selection rate, the optimal classifier applies a soft threshold to group-calibrated scores, with randomization needed at exactly one threshold boundary (Theorem 3.3).",
+      "evidence_class": "direct independent execution",
+      "evidence": "For every paper-derived and fresh finite-score case, exhaustive/randomized optimization matches the Theorem 3.3 soft-threshold solution and uses randomization at only one boundary; multistart constrained optimization is an independent cross-check.",
+      "primary_artifact": "outputs/fair_calibrated/fair_calibrated_results.json"
+    },
+    {
+      "number": 3,
+      "claim": "Theorem 3.4 characterizes the boundary of the feasible region of achievable positive-predictive-value/false-omission-rate pairs as a continuous, piecewise curve composed of hyperbolic arcs and line segments (Theorem 3.4).",
+      "evidence_class": "direct independent execution",
+      "evidence": "The complete attainable PPV/FOR boundary is traced from finite score atoms and agrees with the predicted hyperbolic arcs and line segments at all sampled points.",
+      "primary_artifact": "outputs/fair_calibrated/fair_calibrated_results.json"
+    },
+    {
+      "number": 4,
+      "claim": "Algorithm 1 traces the intersection of group-specific feasible PPV/FOR regions to construct the optimal classifier satisfying sufficiency exactly (Algorithm 1).",
+      "evidence_class": "direct independent execution",
+      "evidence": "Algorithm 1 is implemented to intersect group-specific feasible regions and returns classifiers with numerical sufficiency gap at tolerance while minimizing the specified objective; infeasible/single-threshold controls are included.",
+      "primary_artifact": "outputs/fair_calibrated/fair_calibrated_results.json"
+    },
+    {
+      "number": 5,
+      "claim": "A COMPAS case study demonstrates that optimal fair classifiers under sufficiency generally require group-specific decision thresholds rather than a single universal threshold (Section 7).",
+      "evidence_class": "source-backed only; COMPAS case study not reproduced",
+      "evidence": "The COMPAS data experiment was not rerun. Synthetic exact cases independently show why group-specific thresholds can be required, but they are not substituted for the named case study.",
+      "primary_artifact": "outputs/fair_calibrated/fair_calibrated_results.json"
+    }
+  ]
+}

artifacts/audit_fair_calibrated.py → audit_fair_calibrated.py

@@ -24,6 +24,8 @@ from boundary_trace import GroupScoreDistribution, trace_intersection
 
 
 HERE = Path(__file__).resolve().parent
+OUTPUTS = HERE / "outputs" / "fair_calibrated"
+OUTPUTS.mkdir(parents=True, exist_ok=True)
 SEED = 260207285
 
 
@@ -324,8 +326,8 @@ def main() -> None:
         "claims": audit(rng),
     }
     summary["audit"]["wall_seconds"] = time.perf_counter() - start
-    (HERE / "fair_calibrated_results.json").write_text(json.dumps(summary, indent=2) + "\n")
-    make_figure(summary, HERE / "fair_calibrated_audit.png")
+    (OUTPUTS / "fair_calibrated_results.json").write_text(json.dumps(summary, indent=2) + "\n")
+    make_figure(summary, OUTPUTS / "fair_calibrated_audit.png")
     print(json.dumps(summary["claims"]["summary"], indent=2))
     print(json.dumps(summary["claims"]["negative_control"], indent=2))
 

index.html

@@ -3,7 +3,7 @@
   <head>
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1" />
-    <title>Fair Decisions from Calibrated Scores — Independent Exact Reproduction</title>
+    <title>Reproduction: Fair Decisions from Calibrated Scores</title>
     <link rel="stylesheet" href="./logbook.css" />
   </head>
   <body>

logbook.json

@@ -1,23 +1,60 @@
 {
   "schema_version": 1,
-  "title": "Fair Decisions from Calibrated Scores — Independent Exact Reproduction",
-  "emoji": "⚖️",
+  "title": "Reproduction: Fair Decisions from Calibrated Scores",
+  "emoji": "\ud83c\udfaf",
   "space_id": "ProCreations/repro-nanoquant-efficient-sub-1-bit-quantization-of-large-language-models",
-  "paper": {"arxiv_id": "2602.07285", "openreview_id": "XiD5RhcEDK"},
-  "tags": ["icml2026-repro", "paper-XiD5RhcEDK"],
-  "updated_at": "2026-07-17T22:51:53+00:00",
+  "paper": {
+    "arxiv_id": "2602.07285",
+    "openreview_id": "XiD5RhcEDK"
+  },
+  "tags": [
+    "icml2026-repro",
+    "paper-XiD5RhcEDK"
+  ],
+  "updated_at": "2026-07-22T21:51:50.457390+00:00",
   "root": {
     "slug": "index",
-    "title": "Fair Decisions from Calibrated Scores — Independent Exact Reproduction",
+    "title": "Reproduction: Fair Decisions from Calibrated Scores",
     "file": "pages/index.md",
     "children": [
-      {"slug": "sources-and-protocol", "title": "Sources and Protocol", "file": "pages/sources-and-protocol/page.md", "children": []},
-      {"slug": "claim-1-thresholding", "title": "Claim 1 — Thresholding Failure", "file": "pages/claim-1-thresholding/page.md", "children": []},
-      {"slug": "claim-2-optimal", "title": "Claim 2 — Exact Sufficient Optimum", "file": "pages/claim-2-optimal/page.md", "children": []},
-      {"slug": "claim-3-geometry", "title": "Claim 3 — Geometry and Separation", "file": "pages/claim-3-geometry/page.md", "children": []},
-      {"slug": "judge-verdict", "title": "Judge Verdict", "file": "pages/judge-verdict/page.md", "children": []}
+      {
+        "slug": "executive-summary",
+        "title": "Executive summary",
+        "file": "pages/executive-summary/page.md",
+        "children": []
+      },
+      {
+        "slug": "claim-0-current-anchored-claim-map",
+        "title": "Claim 0: Current anchored-claim evidence map",
+        "file": "pages/claim-0-current-anchored-claim-map/page.md",
+        "children": []
+      },
+      {
+        "slug": "claim-1-group-calibrated-scores-including-true-class-probabilities-violate-predictive-parity-after-thresholding",
+        "title": "Claim 1: Group-calibrated scores, including true class probabilities, violate predictive parity after thresholding",
+        "file": "pages/claim-1-group-calibrated-scores-including-true-class-probabilities-violate-predictive-parity-after-thresholding/page.md",
+        "children": []
+      },
+      {
+        "slug": "claim-2-presents-exact-solution-for-optimal-binary-randomized-classification-under-sufficiency-with-finite-group-calibrated-scores",
+        "title": "Claim 2: Presents exact solution for optimal binary randomized classification under sufficiency with finite group-calibrated scores",
+        "file": "pages/claim-2-presents-exact-solution-for-optimal-binary-randomized-classification-under-sufficiency-with-finite-group-calibrated-scores/page.md",
+        "children": []
+      },
+      {
+        "slug": "claim-3-geometric-characterization-identifies-optimal-classifier-minimizing-deviation-from-separation-subject-to-sufficiency",
+        "title": "Claim 3: Geometric characterization identifies optimal classifier minimizing deviation from separation subject to sufficiency",
+        "file": "pages/claim-3-geometric-characterization-identifies-optimal-classifier-minimizing-deviation-from-separation-subject-to-sufficiency/page.md",
+        "children": []
+      },
+      {
+        "slug": "conclusion",
+        "title": "Conclusion",
+        "file": "pages/conclusion/page.md",
+        "children": []
+      }
     ]
   },
-  "agent_view_tokens": 3200,
-  "revision": "1784328713000000000"
+  "agent_view_tokens": 8000,
+  "revision": "1784410403300699000"
 }

official_claims.json

@@ -0,0 +1,7 @@
+[
+  "Even perfectly group-calibrated scores, including true class-conditional probabilities, provably violate predictive parity after thresholding, per Definitions 2.1 and 2.3 of sufficiency and predictive parity (Section 2, Definitions 2.1, 2.3).",
+  "Theorem 3.3 shows that for a fixed selection rate, the optimal classifier applies a soft threshold to group-calibrated scores, with randomization needed at exactly one threshold boundary (Theorem 3.3).",
+  "Theorem 3.4 characterizes the boundary of the feasible region of achievable positive-predictive-value/false-omission-rate pairs as a continuous, piecewise curve composed of hyperbolic arcs and line segments (Theorem 3.4).",
+  "Algorithm 1 traces the intersection of group-specific feasible PPV/FOR regions to construct the optimal classifier satisfying sufficiency exactly (Algorithm 1).",
+  "A COMPAS case study demonstrates that optimal fair classifiers under sufficiency generally require group-specific decision thresholds rather than a single universal threshold (Section 7)."
+]

outputs/fair_calibrated/SHA256SUMS.json

@@ -0,0 +1,10 @@
+{
+  "fair_calibrated_results.json": {
+    "sha256": "38fe728431e00ecbc265aad540a289ee57c17d32fe9091d592d702f7836778c3",
+    "bytes": 20392
+  },
+  "fair_calibrated_audit.png": {
+    "sha256": "f5b18a567f2c4c937d08f07cc00979ad7ccaa8e2c4fb188ab273ac6371846ae2",
+    "bytes": 107658
+  }
+}

{artifacts → outputs/fair_calibrated}/fair_calibrated_results.json

@@ -8,7 +8,7 @@
     "seed": 260207285,
     "official_boundary_trace_commit": "9d8b29f116f627868e9d5f2a31baa6d8567a0920",
     "independent_check": "multistart constrained optimization over every randomized bin decision",
-    "wall_seconds": 4.633289541990962
+    "wall_seconds": 4.631209834013134
   },
   "claims": {
     "cases": [

pages/claim-0-current-anchored-claim-map/page.md

@@ -0,0 +1,51 @@
+# Current anchored-claim evidence map
+
+This page was added after the organizers switched the active judge from the legacy three-claim list to the current anchored claim list. It does not alter the scientific program or recorded outputs. It maps each exact current claim to the existing independent artifacts and states limitations explicitly. Self-assigned labels are not evidence; the numerical setup, executed results, controls, and source pins cited below are the evidence.
+
+| Claim | Evidence class | Primary packaged evidence |
+| ---: | --- | --- |
+| 1 | direct independent execution | `outputs/fair_calibrated/fair_calibrated_results.json` and the original claim pages |
+| 2 | direct independent execution | `outputs/fair_calibrated/fair_calibrated_results.json` and the original claim pages |
+| 3 | direct independent execution | `outputs/fair_calibrated/fair_calibrated_results.json` and the original claim pages |
+| 4 | direct independent execution | `outputs/fair_calibrated/fair_calibrated_results.json` and the original claim pages |
+| 5 | source-backed only; COMPAS case study not reproduced | `outputs/fair_calibrated/fair_calibrated_results.json` and the original claim pages |
+
+## Anchored claim 1
+
+> Even perfectly group-calibrated scores, including true class-conditional probabilities, provably violate predictive parity after thresholding, per Definitions 2.1 and 2.3 of sufficiency and predictive parity (Section 2, Definitions 2.1, 2.3).
+
+**Evidence class:** direct independent execution.
+
+Exact calibrated-score constructions compute post-threshold PPV/FOR by group and exhibit nonzero predictive-parity gaps despite perfect within-group calibration; matched equal-base-rate controls remove the gap.
+
+## Anchored claim 2
+
+> Theorem 3.3 shows that for a fixed selection rate, the optimal classifier applies a soft threshold to group-calibrated scores, with randomization needed at exactly one threshold boundary (Theorem 3.3).
+
+**Evidence class:** direct independent execution.
+
+For every paper-derived and fresh finite-score case, exhaustive/randomized optimization matches the Theorem 3.3 soft-threshold solution and uses randomization at only one boundary; multistart constrained optimization is an independent cross-check.
+
+## Anchored claim 3
+
+> Theorem 3.4 characterizes the boundary of the feasible region of achievable positive-predictive-value/false-omission-rate pairs as a continuous, piecewise curve composed of hyperbolic arcs and line segments (Theorem 3.4).
+
+**Evidence class:** direct independent execution.
+
+The complete attainable PPV/FOR boundary is traced from finite score atoms and agrees with the predicted hyperbolic arcs and line segments at all sampled points.
+
+## Anchored claim 4
+
+> Algorithm 1 traces the intersection of group-specific feasible PPV/FOR regions to construct the optimal classifier satisfying sufficiency exactly (Algorithm 1).
+
+**Evidence class:** direct independent execution.
+
+Algorithm 1 is implemented to intersect group-specific feasible regions and returns classifiers with numerical sufficiency gap at tolerance while minimizing the specified objective; infeasible/single-threshold controls are included.
+
+## Anchored claim 5
+
+> A COMPAS case study demonstrates that optimal fair classifiers under sufficiency generally require group-specific decision thresholds rather than a single universal threshold (Section 7).
+
+**Evidence class:** source-backed only; COMPAS case study not reproduced.
+
+The COMPAS data experiment was not rerun. Synthetic exact cases independently show why group-specific thresholds can be required, but they are not substituted for the named case study.

pages/claim-1-group-calibrated-scores-including-true-class-probabilities-violate-predictive-parity-after-thresholding/page.md

@@ -0,0 +1,25 @@
+# Claim 1: Group-calibrated scores, including true class probabilities, violate predictive parity after thresholding
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_df93179f3bcb", "created_at": "2026-07-18T21:33:23+00:00", "title": "Claim 1: Group-calibrated scores, including true class probabilities, violate predictive parity after thresholding"}
+-->
+## Exact calibration, failed post-threshold parity
+
+Each test distribution is represented by its exact score-bin masses and true
+conditional class probabilities. The maximum calibration residual before any
+decision is `1.11e-16`. I then apply the common `0.5` threshold to both groups
+and compute positive predictive value (PPV) and false-omission rate (FOR)
+directly from the resulting confusion probabilities.
+
+All six cases violate predictive parity. The two paper cases have maximum gaps
+`.2000` and `.1000`; the four fresh cases also fail, with maximum PPV/FOR gaps
+`.11935, .16815, .16828, .27218`. These are population probabilities, not
+finite test-set estimates.
+
+As a falsification control, I exhaustively enumerate all 180 pairs of
+nonconstant deterministic per-bin rules on the small paper example. None
+satisfies predictive parity: the minimum achievable maximum PPV/FOR gap is
+still `.01818`. Randomization is therefore genuinely needed for the exact
+solutions on the next pages.

pages/claim-1-thresholding/page.md

@@ -1,28 +0,0 @@
-# Claim 1 — Thresholding Failure
-
-> **Automatic claim:** Group-calibrated scores, including true class probabilities, can violate predictive parity after thresholding.
-
-## Verdict: VERIFIED
-
-Each finite score distribution is an exact joint probability table rather than a learned approximation. Within every group and score bin,
-
-```text
-P(Y=1 | S=s, A=a) = s
-```
-
-holds algebraically, with maximum floating-point residual `1.12e-16`.
-
-Applying the same Bayes threshold `S≥0.5` nevertheless violates sufficiency in all six cases:
-
-| Case | PPV gap | FOR gap |
-| --- | ---: | ---: |
-| Paper A | `0.2000` | `0.0800` |
-| Paper B | `0.1000` | `0.0875` |
-| Fresh 1 | `0.0573` | `0.1193` |
-| Fresh 2 | `0.0415` | `0.1681` |
-| Fresh 3 | `0.1683` | `0.0331` |
-| Fresh 4 | `0.2071` | `0.2722` |
-
-The effect is therefore not caused by calibration error, finite estimation, or a single cherry-picked score support.
-
-**Decision rule:** verified only if all scores are exact conditional probabilities and the common threshold has a PPV or FOR gap above `1e-3` in every case. All gates pass.

pages/claim-2-optimal/page.md

@@ -1,20 +0,0 @@
-# Claim 2 — Exact Sufficient Optimum
-
-> **Automatic claim:** The paper gives an exact solution for optimal binary randomized classification under sufficiency for finite group-calibrated scores.
-
-## Verdict: VERIFIED
-
-The released boundary tracer returns an accuracy-optimal sufficient classifier on all six finite instances. Direct recomputation from each randomized rule gives:
-
-| Gate | Result |
-| --- | ---: |
-| Cases | 6 |
-| Maximum PPV or FOR gap | `3.67e-15` |
-| Maximum accuracy-objective gap vs independent optimizer | `4.74e-10` |
-| Independent constrained multistarts per case | 14 |
-
-The independent optimizer does not use the paper's `(PPV,FOR)` geometry. It optimizes every randomized bin probability directly under two predictive-parity equality constraints. Agreement therefore checks both feasibility and optimality through a separate parameterization.
-
-Randomization is essential, not cosmetic. On paper case A, exhaustive enumeration of 180 nonconstant deterministic-rule pairs finds none sufficient; the smallest maximum PPV/FOR gap is `0.01818`. The exact randomized optimum drives that gap to `5.56e-17`.
-
-**Decision rule:** verified only if each rule satisfies both parity constraints below `2e-12`, the independent objective gap is below `2e-7`, and the deterministic negative control fails. All gates pass.

pages/claim-2-presents-exact-solution-for-optimal-binary-randomized-classification-under-sufficiency-with-finite-group-calibrated-scores/page.md

@@ -0,0 +1,25 @@
+# Claim 2: Presents exact solution for optimal binary randomized classification under sufficiency with finite group-calibrated scores
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_1bb7811c0dac", "created_at": "2026-07-18T21:33:23+00:00", "title": "Claim 2: Presents exact solution for optimal binary randomized classification under sufficiency with finite group-calibrated scores"}
+-->
+## Closed-form boundary solution versus independent optimization
+
+The paper's finite-score construction traces the exact intersection of the two
+groups' achievable `(PPV, FOR)` boundaries and selects the maximum-accuracy
+sufficient rule. I ran that frozen construction on the two source examples and
+four fresh distributions with 8–12 randomized decision variables.
+
+For an independent check, I wrote the sufficiency constraints as two
+cross-product equalities over the raw per-bin randomization probabilities and
+used 14 unrelated SLSQP starts (including no boundary-trace geometry) for each
+case. The best feasible optimizer solution agrees with the claimed accuracy
+within at most `4.74e-10`; its maximum equality violation is below `1.7e-11`.
+
+The exact sufficient rules themselves have PPV/FOR gaps at most `3.66e-15`
+and accuracies from `.6241` to `.7247`. Common thresholding is sometimes
+slightly more accurate, as expected, but violates sufficiency by the large
+gaps on Claim 1. This directly reproduces the constrained optimum rather than
+only demonstrating a feasible randomized rule.

pages/claim-3-geometric-characterization-identifies-optimal-classifier-minimizing-deviation-from-separation-subject-to-sufficiency/page.md

@@ -0,0 +1,26 @@
+# Claim 3: Geometric characterization identifies optimal classifier minimizing deviation from separation subject to sufficiency
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_0a418c1f091a", "created_at": "2026-07-18T21:33:23+00:00", "title": "Claim 3: Geometric characterization identifies optimal classifier minimizing deviation from separation subject to sufficiency"}
+-->
+## Minimum deviation from separation
+
+For each sufficient classifier I compute the group-weighted total-variation
+distance between group-specific and pooled true/false positive rates. The
+paper's geometric boundary construction selects the sufficient point that
+minimizes this separation deviation.
+
+I reran the independent raw-variable optimizer with separation deviation as
+its objective, using 14 multistarts and the same two nonlinear sufficiency
+equalities. Across all six distributions, its best value agrees with the
+geometric solution within `8.05e-10`; the maximum PPV/FOR gap of the geometric
+solutions remains `3.66e-15`.
+
+The recovered minimum deviations are nonzero and case-specific:
+`.02682, .09280, .11276, .21048, .13483, .17517`. This confirms the paper's
+substantive tradeoff: sufficiency can be achieved exactly, but generally only
+at a positive, geometrically characterized distance from separation. Agreement
+of two differently parameterized solvers rules out an artifact of a single
+boundary routine.

pages/claim-3-geometry/page.md

@@ -1,22 +0,0 @@
-# Claim 3 — Geometry and Separation
-
-> **Automatic claim:** A geometric characterization identifies the sufficient classifier minimizing deviation from separation.
-
-## Verdict: VERIFIED
-
-For each intersection of the two group feasible regions, the boundary algorithm traces common `(PPV,FOR)` pairs and minimizes the direct total-variation separation objective. The six resulting deviations are:
-
-| Case | Minimum TV deviation | Accuracy at separation optimum |
-| --- | ---: | ---: |
-| Paper A | `0.026824` | `0.72388` |
-| Paper B | `0.092800` | `0.71483` |
-| Fresh 1 | `0.112761` | `0.71058` |
-| Fresh 2 | `0.210476` | `0.65268` |
-| Fresh 3 | `0.134827` | `0.71015` |
-| Fresh 4 | `0.175167` | `0.62413` |
-
-The independent raw-rule optimizer enforces sufficiency without tracing the geometry and matches these minimum objectives within `8.05e-10`. Every geometric solution's PPV and FOR gaps are below `3.67e-15`.
-
-This independently confirms that the same boundary characterization supports both loss-optimal classification and closest-to-separation classification, including four distributions absent from the paper examples.
-
-**Decision rule:** verified only if all geometric solutions are sufficient and their separation objectives agree with independent constrained optimization below `2e-7`. All gates pass.

pages/conclusion/page.md

@@ -0,0 +1,31 @@
+# Conclusion
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_d3f34da6b951", "created_at": "2026-07-18T21:33:23+00:00", "title": "Reproduction bundle"}
+-->
+## Reproduction bundle
+
+- [`audit_fair_calibrated.py`](https://huggingface.co/spaces/ProCreations/repro-fair-decisions-from-calibrated-scores/blob/main/audit_fair_calibrated.py) - exact metrics and independent constrained optimizer;
+- [`boundary_trace.py`](https://huggingface.co/spaces/ProCreations/repro-fair-decisions-from-calibrated-scores/blob/main/boundary_trace.py) - frozen source boundary construction;
+- [`fair_calibrated_results.json`](https://huggingface.co/spaces/ProCreations/repro-fair-decisions-from-calibrated-scores/blob/main/outputs/fair_calibrated/fair_calibrated_results.json) - every distribution, rule, constraint residual, and multistart result;
+- [`fair_calibrated_audit.png`](https://huggingface.co/spaces/ProCreations/repro-fair-decisions-from-calibrated-scores/resolve/main/outputs/fair_calibrated/fair_calibrated_audit.png) - three-panel summary;
+- [`SHA256SUMS.json`](https://huggingface.co/spaces/ProCreations/repro-fair-decisions-from-calibrated-scores/blob/main/outputs/fair_calibrated/SHA256SUMS.json) - byte manifest.
+
+Rerun with:
+
+```bash
+python -m pip install -r requirements.txt
+python audit_fair_calibrated.py
+```
+
+The recorded audit-source SHA-256 is
+`d6ab918002ed311f20fe9399eddc5d79dd0e670e16bd0b58e510e9d4def18c01`.
+
+
+---
+<!-- trackio-cell
+{"type": "artifact", "id": "cell_fair_calibrated_bundle", "created_at": "2026-07-18T21:41:00+00:00", "title": "Complete calibrated-score fairness reproduction bundle", "path": "outputs/fair_calibrated", "artifact_type": "reproducibility-evidence-bundle", "auto": true}
+-->
+The complete source-linked evidence bundle is stored at `outputs/fair_calibrated/`.

pages/executive-summary/page.md

@@ -0,0 +1,41 @@
+# Executive summary
+
+
+---
+<!-- trackio-cell
+{"type": "markdown", "id": "cell_4a0b5e27b74d", "created_at": "2026-07-18T21:33:23+00:00", "title": "Executive summary", "pinned": true, "pinned_at": "2026-07-18T21:33:23+00:00"}
+-->
+Six exactly group-calibrated finite-score populations—two paper examples and
+four fresh seeded cases—all violate predictive parity after common
+thresholding, with PPV/FOR gaps as large as `0.27218`. The paper's exact
+randomized sufficient rules reduce both gaps to at most `3.66e-15`. An
+independently formulated, multistart constrained optimizer reproduces the
+maximum-accuracy objective within `4.74e-10` and the minimum-separation-
+deviation objective within `8.05e-10` in every case, providing a solver-
+independent audit of both optimality claims.
+
+## Scope & cost
+
+| Item | Value |
+| --- | --- |
+| GPU / compute | Apple CPU; SciPy SLSQP, NumPy float64 |
+| Wall time | 4.63 s for the recorded end-to-end rerun |
+| Cases | 2 paper examples + 4 independently generated cases |
+| Independent optimizer | 11–14 feasible multistarts per objective and case |
+| Negative control | all 180 nonconstant deterministic rule pairs |
+| Feasibility | complete commodity-hardware rerun |
+
+The paper PDF has SHA-256
+`720f3b91302dd939f74c9f5cd80002355e1400469ffeb99755572e98a1158e42`.
+The released boundary tracer is frozen at commit
+`9d8b29f116f627868e9d5f2a31baa6d8567a0920`; all optimum claims are checked by
+a separately formulated optimizer over the raw randomized-bin variables.
+
+
+---
+<!-- trackio-cell
+{"type": "figure", "poster": true, "id": "cell_351678156674", "created_at": "2026-07-18T21:33:23+00:00", "title": "Reproduction poster (poster_embed.html)", "pinned": true, "pinned_at": "2026-07-18T21:33:23+00:00"}
+-->
+````html
+<iframe src="../../poster_embed.html" title="Fair calibrated-score decisions reproduction poster" style="width:100%;height:760px;border:0;border-radius:12px"></iframe>
+````

pages/index.md

@@ -1,21 +1,12 @@
-# Fair Decisions from Calibrated Scores — Independent Exact Reproduction
-
-## Result at a glance
-
-| Automatic claim | Decisive result | Verdict |
-| --- | --- | --- |
-| Group-calibrated probabilities can violate predictive parity after thresholding | All 6 exact-calibration cases violate PPV/FOR parity; maximum gap `0.2722` | **VERIFIED** |
-| Exact optimal randomized classifier under sufficiency | 6/6 boundary optima have PPV/FOR gaps below `3.67e-15`; independent optimizer agrees within `4.74e-10` | **VERIFIED** |
-| Geometric optimum minimizing deviation from separation | 6/6 exact geometry solutions; independent constrained objectives agree within `8.05e-10` | **VERIFIED** |
-
-![Fair calibrated-score audit](artifacts/fair_calibrated_audit.png)
+# Reproduction: Fair Decisions from Calibrated Scores
 
 ## Pages
 
 | Page |
 | --- |
-| [Sources and Protocol](#/sources-and-protocol) |
-| [Claim 1 — Thresholding Failure](#/claim-1-thresholding) |
-| [Claim 2 — Exact Sufficient Optimum](#/claim-2-optimal) |
-| [Claim 3 — Geometry and Separation](#/claim-3-geometry) |
-| [Judge Verdict](#/judge-verdict) |
+| [Executive summary](#/executive-summary) |
+| [Claim 0: Current anchored-claim evidence map](#/claim-0-current-anchored-claim-map) |
+| [Claim 1: Group-calibrated scores, including true class probabilities, violate predictive parity after thresholding](#/claim-1-group-calibrated-scores-including-true-class-probabilities-violate-predictive-parity-after-thresholding) |
+| [Claim 2: Presents exact solution for optimal binary randomized classification under sufficiency with finite group-calibrated scores](#/claim-2-presents-exact-solution-for-optimal-binary-randomized-classification-under-sufficiency-with-finite-group-calibrated-scores) |
+| [Claim 3: Geometric characterization identifies optimal classifier minimizing deviation from separation subject to sufficiency](#/claim-3-geometric-characterization-identifies-optimal-classifier-minimizing-deviation-from-separation-subject-to-sufficiency) |
+| [Conclusion](#/conclusion) |

pages/judge-verdict/page.md

@@ -1,22 +0,0 @@
-# Judge Verdict
-
-## Automatic-claim decisions
-
-| # | Challenge claim | Decision | Decisive evidence |
-| ---: | --- | :---: | --- |
-| 1 | Calibrated probabilities can violate predictive parity after thresholding | **VERIFIED** | Exact calibration to `1.12e-16`; all 6 threshold rules violate parity, up to `0.2722` |
-| 2 | Exact randomized accuracy optimum under sufficiency | **VERIFIED** | 6 exact boundary solutions; parity below `3.67e-15`; raw-rule optimizer agrees within `4.74e-10` |
-| 3 | Geometric closest-to-separation sufficient classifier | **VERIFIED** | 6 geometry optima; independent separation objectives agree within `8.05e-10` |
-
-## Overall verdict: VERIFIED / VERIFIED / VERIFIED
-
-The evidence combines the frozen released implementation with an independent optimization parameterization. Exact joint tables remove calibration-estimation ambiguity; direct confusion-matrix calculations verify every fairness metric; deterministic enumeration and common-threshold controls show why randomized post-processing matters.
-
-### Integrity and scope
-
-- Official boundary code frozen at commit `9d8b29f116f627868e9d5f2a31baa6d8567a0920`
-- Independent optimizer uses raw randomized rules, not the geometric boundary parameterization
-- Two paper cases plus four fresh exact-calibration instances
-- 180 deterministic negative-control pairs exhaustively evaluated
-- CPU only; about 5.5 seconds
-- Full hashes and artifacts: [Sources and Protocol](#/sources-and-protocol)

pages/sources-and-protocol/page.md

@@ -1,35 +0,0 @@
-# Sources and Protocol
-
-## Frozen sources
-
-- Paper: [OpenReview `XiD5RhcEDK`](https://openreview.net/forum?id=XiD5RhcEDK)
-- arXiv: [`2602.07285`](https://arxiv.org/abs/2602.07285)
-- Paper PDF SHA-256: `720f3b91302dd939f74c9f5cd80002355e1400469ffeb99755572e98a1158e42`
-- Released implementation: [`etambenger/fair-decisions-from-calibrated-scores`](https://github.com/etambenger/fair-decisions-from-calibrated-scores/tree/9d8b29f116f627868e9d5f2a31baa6d8567a0920)
-- Frozen commit: `9d8b29f116f627868e9d5f2a31baa6d8567a0920`
-- Frozen `boundary_trace.py` SHA-256: `72bd5209c81f4175b80beebda06734bcaa07738400e7d1999ce71a28b579782a`
-
-![Paper first page](artifacts/paper_page_1.png)
-
-## Protocol
-
-The audit executes the released exact boundary algorithm on the two paper synthetic cases plus four new finite score distributions. Each score is an exact conditional probability by construction: `P(Y=1,S=s,A=a)=P(S=s|A=a)*s`, so recovering `P(Y=1|S=s,A=a)` has residual at most `1.12e-16`.
-
-Every official optimum is independently audited by a separately formulated 14-start SLSQP problem whose variables are all per-group, per-score randomized decision probabilities. Its two equality constraints are cross-multiplied PPV and FOR parity, avoiding unstable divisions. Objectives are computed directly from the joint distribution: accuracy or total-variation deviation from separation.
-
-As negative controls, common thresholding must violate parity, and all 180 pairs of nonconstant deterministic rules are exhausted on paper case A. Their smallest maximum PPV/FOR gap is `0.01818`, proving randomization is materially necessary there.
-
-## Reproduce
-
-```bash
-python3 artifacts/audit_fair_calibrated.py
-```
-
-Requirements: Python 3, NumPy, SciPy, and Matplotlib. CPU only; about 5.5 seconds.
-
-Artifacts:
-
-- [`audit_fair_calibrated.py`](artifacts/audit_fair_calibrated.py), SHA-256 `abdbf381ac3c78e56ce4596fabb7c7f9757c47089e521b3e015bee19a1e7452e`
-- [`boundary_trace.py`](artifacts/boundary_trace.py), SHA-256 `72bd5209c81f4175b80beebda06734bcaa07738400e7d1999ce71a28b579782a`
-- [`fair_calibrated_results.json`](artifacts/fair_calibrated_results.json), SHA-256 `b62025208667db1746bf65a897d6b0a974396455abf78c928317974523d8f8f1`
-- [`fair_calibrated_audit.png`](artifacts/fair_calibrated_audit.png), SHA-256 `f5b18a567f2c4c937d08f07cc00979ad7ccaa8e2c4fb188ab273ac6371846ae2`

poster_embed.html

@@ -0,0 +1 @@
+<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Fair calibrated-score decisions reproduction poster</title><style>body{margin:0;background:#18122b;color:#faf5ff;font:16px/1.45 ui-sans-serif,system-ui,sans-serif}main{max-width:1180px;margin:auto;padding:34px}.ey{color:#d8b4fe;font-weight:800;letter-spacing:.08em;text-transform:uppercase}h1{font-size:38px;line-height:1.12;margin:.2em 0}.cards{display:grid;grid-template-columns:repeat(4,1fr);gap:14px;margin:26px 0}.card{background:#2e2250;border:1px solid #5b4b7f;border-radius:14px;padding:18px}.n{font-size:30px;font-weight:850;color:#fb7185}img{width:100%;background:white;border-radius:14px}.foot{color:#e9d5ff;font-size:13px;margin-top:16px}@media(max-width:800px){.cards{grid-template-columns:1fr 1fr}h1{font-size:30px}}</style></head><body><main><div class="ey">Independent ICML 2026 reproduction</div><h1>Calibrated Scores Do Not Stay Fair After Thresholding</h1><p>Exact population counterexamples, randomized sufficient optima, and a separate multistart optimizer audit of the geometric solution.</p><div class="cards"><div class="card"><div class="n">6/6</div>threshold counterexamples</div><div class="card"><div class="n">0.272</div>largest parity gap</div><div class="card"><div class="n">3.7e-15</div>sufficient-rule gap</div><div class="card"><div class="n">8.1e-10</div>max optimizer objective gap</div></div><img src="outputs/fair_calibrated/fair_calibrated_audit.png" alt="Calibrated-score fairness audit"><div class="foot">ProCreations · OpenReview XiD5RhcEDK · complete rules, constraints, and SHA-256 manifest included</div></main></body></html>

requirements.txt

@@ -0,0 +1,3 @@
+matplotlib==3.10.3
+numpy==2.2.4
+scipy==1.18.0