II.
TestSet overview
Reference · livetest-set:swe-bench-original
SWE-bench original test set overview
Source-backed canonical SWE-bench artifact for full real-world GitHub issue resolution evaluation.
Attributes
displayName
SWE-bench original test set
benchmarkId
caseCount
2294
releasedAt
2023-10-10
composition
Original SWE-bench issue/PR task collection from 12 popular Python
repositories. Each task asks an agent to patch a real repository at
a base commit so fail-to-pass tests succeed.
homepageUrl
description
Source-backed canonical SWE-bench artifact for full real-world
GitHub issue resolution evaluation.
Outgoing edges
belongs_to_benchmark1
- benchmark:swe-bench·BenchmarkSWE-bench
Incoming edges
None.