Update logbook: Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
Browse files
logbook.json
CHANGED
|
@@ -10,7 +10,7 @@
|
|
| 10 |
"icml2026-repro",
|
| 11 |
"paper-uEVTdoAbnK"
|
| 12 |
],
|
| 13 |
-
"updated_at": "2026-07-20T03:
|
| 14 |
"root": {
|
| 15 |
"slug": "index",
|
| 16 |
"title": "Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?",
|
|
@@ -57,9 +57,15 @@
|
|
| 57 |
"title": "Conclusion",
|
| 58 |
"file": "pages/conclusion/page.md",
|
| 59 |
"children": []
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
}
|
| 61 |
]
|
| 62 |
},
|
| 63 |
-
"agent_view_tokens":
|
| 64 |
-
"revision": "
|
| 65 |
}
|
|
|
|
| 10 |
"icml2026-repro",
|
| 11 |
"paper-uEVTdoAbnK"
|
| 12 |
],
|
| 13 |
+
"updated_at": "2026-07-20T03:53:06+00:00",
|
| 14 |
"root": {
|
| 15 |
"slug": "index",
|
| 16 |
"title": "Reproduction: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?",
|
|
|
|
| 57 |
"title": "Conclusion",
|
| 58 |
"file": "pages/conclusion/page.md",
|
| 59 |
"children": []
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"slug": "repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks",
|
| 63 |
+
"title": "repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks",
|
| 64 |
+
"file": "pages/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/page.md",
|
| 65 |
+
"children": []
|
| 66 |
}
|
| 67 |
]
|
| 68 |
},
|
| 69 |
+
"agent_view_tokens": 7045,
|
| 70 |
+
"revision": "1784519586338618226"
|
| 71 |
}
|
pages/conclusion/page.md
CHANGED
|
@@ -66,3 +66,12 @@ pip install datasets huggingface_hub pandas pyarrow
|
|
| 66 |
**馃摝 Artifact** `Yashp2003/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0` 路 dataset
|
| 67 |
|
| 68 |
trackio-artifact://Yashp2003/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
**馃摝 Artifact** `Yashp2003/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0` 路 dataset
|
| 67 |
|
| 68 |
trackio-artifact://Yashp2003/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
---
|
| 72 |
+
<!-- trackio-cell
|
| 73 |
+
{"type": "dashboard", "id": "cell_c4d330700937", "created_at": "2026-07-20T03:53:03+00:00", "title": "Dashboard: repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks", "dashboard_project": "repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks"}
|
| 74 |
+
-->
|
| 75 |
+
**馃幆 Trackio dashboard** `repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks`
|
| 76 |
+
|
| 77 |
+
trackio-local-dashboard://repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks
|
pages/index.md
CHANGED
|
@@ -13,3 +13,4 @@
|
|
| 13 |
| [Claim 4: All SWE-Bench Pro tasks underwent human verification for adequate context.](#/claim-4-all-swe-bench-pro-tasks-underwent-human-verification-for-adequate-context) |
|
| 14 |
| [Claim 5: SWE-Bench Pro is a contamination-resistant testbed spanning business apps, B2B services, and dev tools.](#/claim-5-swe-bench-pro-is-a-contamination-resistant-testbed-spanning-business-apps-b2b-services-and-dev-tools) |
|
| 15 |
| [Conclusion](#/conclusion) |
|
|
|
|
|
|
| 13 |
| [Claim 4: All SWE-Bench Pro tasks underwent human verification for adequate context.](#/claim-4-all-swe-bench-pro-tasks-underwent-human-verification-for-adequate-context) |
|
| 14 |
| [Claim 5: SWE-Bench Pro is a contamination-resistant testbed spanning business apps, B2B services, and dev tools.](#/claim-5-swe-bench-pro-is-a-contamination-resistant-testbed-spanning-business-apps-b2b-services-and-dev-tools) |
|
| 15 |
| [Conclusion](#/conclusion) |
|
| 16 |
+
| [repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks](#/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks) |
|
pages/repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/page.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
---
|
| 5 |
+
<!-- trackio-cell
|
| 6 |
+
{"type": "artifact", "id": "cell_38663e3e7079", "created_at": "2026-07-20T03:53:03+00:00", "title": "Artifact: repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0", "artifact": "repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0", "artifact_type": "dataset"}
|
| 7 |
+
-->
|
| 8 |
+
**馃摝 Artifact** `repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0` 路 dataset 路 2.0 MB
|
| 9 |
+
|
| 10 |
+
trackio-artifact://repro-swe-bench-pro-can-ai-agents-solve-long-horizon-software-engineering-tasks/repro-bundle:v0
|