II.
Benchmark overview
Reference · livebenchmark:mmlu
MMLU overview
Massive Multitask Language Understanding — 57-subject knowledge benchmark.
Attributes
displayName
MMLU
homepageUrl
kind
knowledge
targetsKind
ModelVersion
description
Massive Multitask Language Understanding — 57-subject knowledge benchmark.
Outgoing edges
covers1
- skill-area:general-knowledge-reasoning·SkillAreaGeneral Knowledge Reasoning
Incoming edges
bounds_subject1
- scope-boundary:mmlu.scope·ScopeBoundary
for_benchmark11
- eval-run:mmlu.qwen-2-5-72b.2024-09·EvalRun
- eval-run:mmlu.claude-sonnet-4-6.2025-11·EvalRun
- eval-run:mmlu.deepseek-v3.2024-12·EvalRun
- eval-run:mmlu.deepseek-r1.2025-01·EvalRun
- eval-run:mmlu.llama-3-1-405b.2024-07·EvalRun
- eval-run:mmlu.llama-3-3-70b.2024-12·EvalRun
- eval-run:mmlu.mistral-large-2.2024-07·EvalRun
- eval-run:mmlu.o1.2024-12·EvalRun
- eval-run:mmlu.phi-3-medium.2024-05·EvalRun
- eval-run:mmlu.gemma-2-27b.2024-06·EvalRun
- eval-run:mmlu.command-r-plus.2024-08·EvalRun
scored_against15
- eval-result:mmlu.qwen-2-5-72b.001·EvalResult
- eval-result:gpqa.claude-haiku-4-5.001·EvalResult
- eval-result:mmlu.claude-sonnet-4-6.001·EvalResult
- eval-result:mmlu.deepseek-v3.001·EvalResult
- eval-result:mmlu.deepseek-r1.001·EvalResult
- eval-result:gpqa.gemini-2-5-pro.001·EvalResult
- eval-result:mmlu.llama-3-1-405b.001·EvalResult
- eval-result:mmlu.llama-3-3-70b.001·EvalResult
- eval-result:mmlu.mistral-large-2.001·EvalResult
- eval-result:gpqa.gpt-5.001·EvalResult
- eval-result:mmlu.o1.001·EvalResult
- eval-result:mmlu.phi-3-medium.001·EvalResult
- eval-result:mmlu.gemma-2-27b.001·EvalResult
- eval-result:mmlu.command-r-plus.001·EvalResult
- eval-result:gpqa.claude-sonnet-4-5.001·EvalResult