pages/claim-1-the-swe-fficiency-benchmark-comprises-498-real-world-code-optimization-tasks-drawn-from-9-widely-used-python-repositories-including-numpy-pandas-and-scipy-abstract-only/page.md

Claim 1: The SWE-fficiency benchmark comprises 498 real-world code optimization tasks drawn from 9 widely used Python repositories including numpy, pandas, and scipy (abstract only).

---

Verification

Verdict: ✅ VERIFIED (full)

The HuggingFace dataset swefficiency/swefficiency was inspected via the HuggingFace Datasets API and confirmed to contain exactly 498 instances across 9 Python repositories:

Repository	Domain	Count
pandas-dev/pandas	Data manipulation (Python)	279
scipy/scipy	Scientific computing	45
sympy/sympy	Symbolic math	39
astropy/astropy	Astronomy	35
scikit-learn/scikit-learn	ML	27
numpy/numpy	Numerical computing	23
matplotlib/matplotlib	Plotting	20
dask/dask	Parallel computing	19
xarray/xarray	Labeled arrays	11
Total		498

Dataset Features

Each instance includes:

• repo, instance_id, base_commit — source identification
• patch — expert's gold optimization patch (git diff format)
• workload — performance workload script (Python code)
• speedup — expert speedup factor (median=2.43x, up to 134x)
• covering_tests — repo unit tests exercising the edited code
• PASS_TO_PASS — test IDs that must stay passing

Source

Dataset: swefficiency/swefficiency on HuggingFace