II.
TestSet overview
Reference · livetest-set:swe-bench-verified-2024-12
SWE-bench Verified 2024-12 overview
The December 2024 release of the SWE-bench Verified test set.
Attributes
displayName
SWE-bench Verified 2024-12
benchmarkId
caseCount
500
releasedAt
2024-12-01
description
The December 2024 release of the SWE-bench Verified test set.
Outgoing edges
belongs_to_benchmark1
- benchmark:swe-bench-verified·BenchmarkSWE-bench Verified
Incoming edges
uses_test_set11
- eval-run:swe-bench-verified.claude-haiku-4-5.2025-10·EvalRun
- eval-run:swe-bench.deepseek-v3.2024-12·EvalRun
- eval-run:swe-bench-verified.gemini-2-5-flash.2025-06·EvalRun
- eval-run:swe-bench.llama-3-1-405b.2024-07·EvalRun
- eval-run:swe-bench-verified.claude-opus-4-5.2025-09·EvalRun
- eval-run:swe-bench-verified.claude-opus-4-7.2026-01·EvalRun
- eval-run:swe-bench-verified.o3.2025-04·EvalRun
- eval-run:swe-bench-verified.gemini-2-5-pro.2025-06·EvalRun
- eval-run:swe-bench.claude-code@1.x.2025-04-29·EvalRun
- eval-run:swe-bench-verified.claude-sonnet-4-5.2025-09·EvalRun
- eval-run:swe-bench-verified.gpt-5.2025-08·EvalRun