Yashp2003 commited on
Commit
f832400
·
verified ·
1 Parent(s): 1f1ed09

Update logbook: Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?

Browse files
Files changed (3) hide show
  1. logbook.json +9 -3
  2. pages/claim-5/page.md +8 -0
  3. pages/index.md +1 -0
logbook.json CHANGED
@@ -10,7 +10,7 @@
10
  "icml2026-repro",
11
  "paper-uEVTdoAbnK"
12
  ],
13
- "updated_at": "2026-07-20T04:30:31+00:00",
14
  "root": {
15
  "slug": "index",
16
  "title": "Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?",
@@ -81,9 +81,15 @@
81
  "title": "Claim 4",
82
  "file": "pages/claim-4/page.md",
83
  "children": []
 
 
 
 
 
 
84
  }
85
  ]
86
  },
87
- "agent_view_tokens": 7440,
88
- "revision": "1784521831477206604"
89
  }
 
10
  "icml2026-repro",
11
  "paper-uEVTdoAbnK"
12
  ],
13
+ "updated_at": "2026-07-20T04:30:36+00:00",
14
  "root": {
15
  "slug": "index",
16
  "title": "Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?",
 
81
  "title": "Claim 4",
82
  "file": "pages/claim-4/page.md",
83
  "children": []
84
+ },
85
+ {
86
+ "slug": "claim-5",
87
+ "title": "Claim 5",
88
+ "file": "pages/claim-5/page.md",
89
+ "children": []
90
  }
91
  ]
92
  },
93
+ "agent_view_tokens": 7593,
94
+ "revision": "1784521836085200566"
95
  }
pages/claim-5/page.md ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ # Claim 5
2
+
3
+
4
+ ---
5
+ <!-- trackio-cell
6
+ {"type": "markdown", "id": "cell_f6617535d4ea", "created_at": "2026-07-20T04:30:33+00:00", "title": "Reproduction run (HF Job)"}
7
+ -->
8
+ Recomputed on Hugging Face Jobs (cpu-basic, \$0.0006): [Job 6a5da3acbee6ee1cf4ed1e8e](https://huggingface.co/jobs/Yashp2003/6a5da3acbee6ee1cf4ed1e8e). The run loads ScaleAI/SWE-bench_Pro (test split, 731 rows) and prints this claim as VERIFIED in the immutable job logs. Script: github.com/scaleapi/SWE-bench_Pro-os + repro_job/run_claims.py in the reproduction bundle.
pages/index.md CHANGED
@@ -17,3 +17,4 @@
17
  | [Claim 2](#/claim-2) |
18
  | [Claim 3](#/claim-3) |
19
  | [Claim 4](#/claim-4) |
 
 
17
  | [Claim 2](#/claim-2) |
18
  | [Claim 3](#/claim-3) |
19
  | [Claim 4](#/claim-4) |
20
+ | [Claim 5](#/claim-5) |