SemiAnalysis Launches GDPx Benchmark — Best AI Agent Scores Only 12% on Professional Tasks

SemiAnalysis released a new benchmark today that puts a hard number on AI agent capability in professional settings — and the number is 12%. The GDPx benchmark, covering 206 tasks across 11 high-value knowledge domains, found that the best-performing model scored just 12.0% on tasks drawn from real professional work, not contrived puzzles. The gap between where agents are and where autonomous deployment would require them to be has never been quantified so precisely.

Benchmarks matter because they set the terms of the debate. Until now, AI companies have largely competed on general reasoning scores like MMLU or coding benchmarks like SWE-Bench — domains where polish and curation can push numbers up. GDPx attempts something harder: simulate actual professional work in domains where errors have financial, legal, or physical consequences.

The benchmark design

SemiAnalysis, the semiconductor and AI research firm, built GDPx to measure the readiness of AI agents for economically meaningful tasks. The name is a reference to gross domestic product — the claim is that these are the domains where AI agents, if they worked, would generate measurable economic value.

The 206 tasks span 11 domains:

• Chip design — hardware description language tasks, timing closure, verification
• Biology — protein structure prediction, pathway analysis, experimental design
• Cybersecurity — vulnerability analysis, threat modeling, pen-testing scenarios
• Finance — portfolio construction, risk modeling, regulatory filings
• Legal — contract review, regulatory compliance, case research
• Plus six additional domains not specified in the initial release documentation

Tasks are designed to require multi-step reasoning, tool use, and domain-specific knowledge — characteristics of real professional workflows rather than single-turn question answering.

The leaderboard

Thirteen frontier models were evaluated. The full results:

Two sub-domains — finance and legal — returned zero scores across all 13 models. Not low scores. Zero. No model successfully completed a single evaluated task in either domain, according to the benchmark release.

Why this matters

Finance and legal are two of the domains where AI vendors have been most aggressive in claiming near-term autonomous capability. Law firms are deploying AI for contract review. Banks are running AI agents on regulatory workflows. GDPx's zero scores don't mean those deployments are worthless — they likely reflect the gap between narrow, well-defined tasks and the full professional scope the benchmark covers. But the gap is bigger than enterprise sales pitches suggest.

The 12% ceiling has practical implications:

• No AI agent can be trusted to complete a complex professional task unsupervised. A 12% completion rate means 88% of attempts fail or are incomplete. Human review is not optional at current performance levels.
• The gap to "useful autonomy" is not months away. Even doubling to 24% — a substantial improvement — would still mean three failures for every success on hard professional tasks.
• Domain expertise bottlenecks matter more than raw intelligence. The variance between domains likely reflects training data quality and task structure more than general reasoning ability. Models may improve faster in biology than legal simply because high-quality legal task data is scarcer.

For businesses evaluating AI agent investments, the GDPx results support a specific conclusion: productivity assistance is real and valuable; autonomous operation on consequential work is not ready.

What to watch

SemiAnalysis says it will update the leaderboard as new models are released and will publish task-level breakdowns showing which sub-domains drove the spread between models. The finance and legal zero scores in particular warrant closer examination — whether those tasks are representative of real-world professional work or reflect benchmark design choices that penalize current architectures will be a central debate in the coming weeks.

Source: SemiAnalysis via Unite.AI