90% of US working hours are spent on tasks not covered well by AI benchmarks
Mercor compared a sample of 29 evaluation sets with U.S. occupational tasks and estimated that most paid work hours have little or no benchmark coverage.
TL;DR
- Mercor says 90% of U.S. working hours fall on tasks not well covered by the 29 benchmarks in its map.
- The company mapped benchmark prompts to the Department of Labor’s O*NET taxonomy and combined the coverage with estimates of time spent on tasks.
- This is a measure of benchmark coverage in a limited sample, not a measure of AI performance across all work or a forecast of job displacement.
Mercor published a map comparing 29 AI benchmarks with tasks in the U.S. Department of Labor’s O*NET taxonomy. It estimates that tasks with little or no benchmark coverage account for 90% of U.S. working hours. [1] [2]
The analysis mapped benchmark prompts to occupations and tasks, then combined coverage with estimates of how much time workers spend on those tasks. Mercor says only a subset of occupations had a task meeting its threshold for meaningful coverage. [1]
The finding reflects Mercor’s selected benchmark sample and mapping method. It measures how much of the work taxonomy is represented in the benchmarks, not whether current AI systems can perform those tasks. [1] [2]
Mercor describes the map as an initial inventory. Whether adding more benchmarks changes the estimate, and how coverage corresponds to real-world productivity, remain open questions. [1] [2]
Why it matters
The map highlights that public benchmark scores cover only part of the range of work tasks. Its estimate is built from a limited sample and should not be read as a direct measure of AI capability across the labor market.
Editor's note
Attributes the 90% estimate to Mercor and describes it as a benchmark-coverage analysis, not a job-impact forecast.