Update logbook: Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
Browse files- logbook.json +9 -3
- pages/claim-5/page.md +8 -0
- pages/index.md +1 -0
logbook.json
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
"icml2026-repro",
|
| 11 |
"paper-uEVTdoAbnK"
|
| 12 |
],
|
| 13 |
-
"updated_at": "2026-07-20T04:30:
|
| 14 |
"root": {
|
| 15 |
"slug": "index",
|
| 16 |
"title": "Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?",
|
|
@@ -81,9 +81,15 @@
|
|
| 81 |
"title": "Claim 4",
|
| 82 |
"file": "pages/claim-4/page.md",
|
| 83 |
"children": []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
}
|
| 85 |
]
|
| 86 |
},
|
| 87 |
-
"agent_view_tokens":
|
| 88 |
-
"revision": "
|
| 89 |
}
|
|
|
|
| 10 |
"icml2026-repro",
|
| 11 |
"paper-uEVTdoAbnK"
|
| 12 |
],
|
| 13 |
+
"updated_at": "2026-07-20T04:30:36+00:00",
|
| 14 |
"root": {
|
| 15 |
"slug": "index",
|
| 16 |
"title": "Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?",
|
|
|
|
| 81 |
"title": "Claim 4",
|
| 82 |
"file": "pages/claim-4/page.md",
|
| 83 |
"children": []
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"slug": "claim-5",
|
| 87 |
+
"title": "Claim 5",
|
| 88 |
+
"file": "pages/claim-5/page.md",
|
| 89 |
+
"children": []
|
| 90 |
}
|
| 91 |
]
|
| 92 |
},
|
| 93 |
+
"agent_view_tokens": 7593,
|
| 94 |
+
"revision": "1784521836085200566"
|
| 95 |
}
|
pages/claim-5/page.md
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Claim 5
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
---
|
| 5 |
+
<!-- trackio-cell
|
| 6 |
+
{"type": "markdown", "id": "cell_f6617535d4ea", "created_at": "2026-07-20T04:30:33+00:00", "title": "Reproduction run (HF Job)"}
|
| 7 |
+
-->
|
| 8 |
+
Recomputed on Hugging Face Jobs (cpu-basic, \$0.0006): [Job 6a5da3acbee6ee1cf4ed1e8e](https://huggingface.co/jobs/Yashp2003/6a5da3acbee6ee1cf4ed1e8e). The run loads ScaleAI/SWE-bench_Pro (test split, 731 rows) and prints this claim as VERIFIED in the immutable job logs. Script: github.com/scaleapi/SWE-bench_Pro-os + repro_job/run_claims.py in the reproduction bundle.
|
pages/index.md
CHANGED
|
@@ -17,3 +17,4 @@
|
|
| 17 |
| [Claim 2](#/claim-2) |
|
| 18 |
| [Claim 3](#/claim-3) |
|
| 19 |
| [Claim 4](#/claim-4) |
|
|
|
|
|
|
| 17 |
| [Claim 2](#/claim-2) |
|
| 18 |
| [Claim 3](#/claim-3) |
|
| 19 |
| [Claim 4](#/claim-4) |
|
| 20 |
+
| [Claim 5](#/claim-5) |
|