Mercor, a company that connects AI labs with domain experts for training data, has released APEX-Agents, a benchmark designed to answer a practical question: can today’s AI agents actually do the long, messy work of an investment banker, management consultant or corporate lawyer? According to the company’s results, the answer for now is no. The best frontier models complete fewer than a quarter of the tasks on a single attempt.
What the benchmark measures
APEX-Agents is part of Mercor’s AI Productivity Index (APEX), a family of benchmarks aimed at measuring skills that carry real economic value rather than performance on isolated quiz questions. The agents version, described in a paper posted to arXiv, focuses on long-horizon tasks that span multiple applications: the kind of work that takes a professional hours and requires pulling information from scattered files, emails and chat threads.
Mercor argues that most existing benchmarks miss this. They test models on self-contained problems or narrow skills, while real office work involves ambiguity, incomplete information and the need to keep track of context across many steps.
How the tasks were built
The company describes a four-stage process. It first surveyed hundreds of professionals in investment banking, consulting and law, including people with experience at firms such as Goldman Sachs, McKinsey and Cravath, to understand what their work actually involves. Senior experts at Mercor then built realistic fictional scenarios inside Google Workspace, such as a multi-week consulting engagement to cut costs at a European oil and gas company.
To reproduce the file systems professionals deal with, Mercor says it worked with Box to create data rooms resembling real corporate document stores. Tasks were then written against those scenarios, each with explicit criteria for what counts as client-ready output. For the corporate law portion, Mercor says the legal AI company Harvey reviewed early tasks and confirmed they matched the standard expected of lawyers serving Fortune 500 clients.
The results
According to Mercor, leading AI models complete fewer than 25 percent of the tasks, each of which would take a human professional several hours. Allowing multiple attempts raised the ceiling, but even the strongest agents did not exceed about 40 percent.
The company’s analysis attributes the failures less to a lack of raw capability than to difficulty with the surrounding work: handling ambiguous instructions, locating the right file among many, and maintaining context across an entire workflow. In Mercor’s summary, no current model is ready to replace a professional in these roles end to end.
Open release
Mercor has open-sourced the benchmark. The task set is available on Hugging Face under a CC-BY license, the evaluation infrastructure, called Archipelago, is published on GitHub, and the full paper is on arXiv. The company also maintains a public leaderboard for the benchmark.
Benchmarks produced by companies with a commercial stake in AI evaluation should be read with that context in mind, and APEX-Agents covers only three professions. But it adds to a growing body of evidence that agent performance on tidy demonstrations does not yet translate into reliable performance on the sprawling, document-heavy tasks that fill a professional’s week.
