Open Agent Leaderboard
Enterprise Article Published on May 18, 2026
How capable are general-purpose AI agents? We built an open evaluation framework to answer that question.
Most evaluations in AI give a simple result: each model’s score on a benchmark task. But when you deploy an agent, you are choosing more than a model — you are choosing a complete system: which tools the agent can use, how it plans steps, what it remembers between actions, and how it recovers when things go wrong. Change any one of those, and the same model can produce very different outcomes at very different costs.
An AI agent’s performance depends not only on the model inside it, but also on how it is built.
Today, we are releasing the Open Agent Leaderboard, an open benchmark for comparing complete agent systems rather than just the models behind them. It reports both quality and cost, so you can see not only what works, but what is worth deploying.
The leaderboard also comes with the Exgentic framework for running and reproducing experiments, as well as a paper describing the full methodology and results. From day one, everything is open.
Can we measure “general-purpose” capability?
When AI agents are carefully tailored to a specific task, they can become extremely useful, such as coding in a familiar codebase or handling customer support requests using a known set of tools. But the harder question is whether the same agent can handle many different tasks — each with its own tools, rules, and constraints — without hand-tuning for every scenario.
A more general-purpose agent is one that you can drop into a new environment and it just works.
That is what we mean by general-purpose capability, and it is best understood as a spectrum rather than a binary label. Of course, being theoretically “general” without actually working is meaningless. What really matters is whether an agent remains capable as the range of tasks and environments expands, and whether it can do so at a reasonable cost. A system that can handle anything but costs a fortune to run is not general-purpose in any practical sense.
This leaderboard measures how general-purpose an agent really is.
It evaluates agents in diverse, unfamiliar environments, each with different tools, rules, and constraints, while reporting both quality and cost. That way, you can see not only how well a system performs, but also whether it is actually worth deploying. It does not cover every capability a truly general agent might eventually need, but it is more demanding than any previous available approach when it comes to testing performance across different settings. And because it measures complete agent systems rather than models alone, it can reveal what is actually driving the results.
What we built
We brought together six benchmarks, each testing a different kind of real-world task. Taken together, they are meant to cover a broad range of work: coding, customer support, technical support, personal assistance, and research.
SWE-Bench Verified—— Fix real bugs in real codebasesBrowseComp+—— Research complex questions across the webAppWorld—— Complete personal tasks across hundreds of apps and actionstau2-Bench Airline & Retail—— Handle customer support requests while following company policiestau2-Bench Telecom—— Handle technical support requests while following company policies
These are well-established benchmarks created and reviewed by the research community. We chose them not because any single benchmark can fully capture a general-purpose agent, but because together they test very different abilities: real code changes, open-ended research, a broad action space, and rule-constrained dialogue. It is this combination that makes the evaluation meaningful.
These benchmarks were originally designed separately, each to test a single type of task in its own way. To make them work together, we needed a unified structure. We introduced a common protocol that gives each benchmark the same format: task (what to do), context (what to know), and action space (what is allowed).
Instead of making each agent speak each benchmark’s own language, we make them all speak the same one.
That standardization is not trivial. Each benchmark has its own assumptions, instructions, and interaction patterns. Ensuring that these…