Task Success Rate
Every other agent metric answers a question about the journey. This one answers the question the user actually has: did the task get done? Task success rate is the share of attempted tasks an agent completed, judged by the state of the world at the end rather than by how good the reasoning looked along the way.
How success is decided
The metric is only as good as its verifier, so benchmarks define success programmatically:
- Tests pass — SWE-bench applies the agent's patch and runs the repository's own test suite;
- Final state matches — Tau-bench compares the resulting database against the state a correct run would have left behind;
- Exact answer — GAIA scores a short string against a known ground truth;
- Judged output — an LLM-as-judge grades against a rubric where no mechanical check exists, at some cost in reliability.
Scoring is binary per task. An agent that got nineteen steps right and corrupted the twentieth scores zero, which is the correct treatment for a system allowed to act on its own.
Why it matters
One number comparable across models is what makes agent progress measurable at all. It is also blind to cost and to luck: two agents at sixty percent are not equivalent if one burns ten times the tokens, and consistency needs repeated attempts on the same task rather than a best-of-k figure. Pair it with trajectory evaluation, which is what tells you why the other forty percent failed.
Task success rate is the fraction of attempted tasks an agent completes correctly, scored on the final state rather than on the quality of its reasoning — tests passing, a database matching the expected result, or an answer matching ground truth. It is the headline number in benchmarks such as SWE-bench and GAIA, and the only metric that reflects what the user asked for, though it says nothing about cost or about consistency across repeated runs.
If you hire someone to assemble a bookshelf, you do not grade their technique. You check whether the shelf is standing.
Task success rate is that check, applied to AI agents. Give an agent a hundred jobs, count how many it actually finished correctly, and you have the number. It is deliberately harsh: almost-finished counts as failed, because a half-built shelf is no use to anyone. That makes it the most honest measure we have of whether software can be trusted to get on with a job by itself — and the figure is usually far lower than people expect.