File size: 1,554 Bytes
a4bbd3b a1704cb a4bbd3b a1704cb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # Claim 1: The SWE-fficiency benchmark comprises 498 real-world code optimization tasks drawn from 9 widely used Python repositories including numpy, pandas, and scipy (abstract only).
---
<!-- trackio-cell
{"type": "markdown", "id": "cell_c1_v1", "created_at": "2026-07-22T14:50:08+00:00", "title": "Verification"}
-->
## Verification
**Verdict: ✅ VERIFIED (full)**
The HuggingFace dataset `swefficiency/swefficiency` was inspected via the HuggingFace Datasets API and confirmed to contain exactly **498 instances** across **9 Python repositories**:
| Repository | Domain | Count |
|---|---|---|
| pandas-dev/pandas | Data manipulation (Python) | 279 |
| scipy/scipy | Scientific computing | 45 |
| sympy/sympy | Symbolic math | 39 |
| astropy/astropy | Astronomy | 35 |
| scikit-learn/scikit-learn | ML | 27 |
| numpy/numpy | Numerical computing | 23 |
| matplotlib/matplotlib | Plotting | 20 |
| dask/dask | Parallel computing | 19 |
| xarray/xarray | Labeled arrays | 11 |
| **Total** | | **498** |
### Dataset Features
Each instance includes:
- `repo`, `instance_id`, `base_commit` — source identification
- `patch` — expert's gold optimization patch (git diff format)
- `workload` — performance workload script (Python code)
- `speedup` — expert speedup factor (median=2.43x, up to 134x)
- `covering_tests` — repo unit tests exercising the edited code
- `PASS_TO_PASS` — test IDs that must stay passing
### Source
Dataset: [swefficiency/swefficiency](https://huggingface.co/datasets/swefficiency/swefficiency) on HuggingFace
|