Top Stories
All Sections
Finance
About
17 results for "benchmarks"
SemiAnalysis released GDPx, a 206-task benchmark across 11 professional domains. The best AI agent scored just 12%. Finance and legal returned zero scores across all 13 models tested.
Google's Gemini 4 Carbon scores 88.2% on SEC-bench Pro for vulnerability identification and 71% on producing working fixes — access restricted to vetted defenders through the Fairwind program.
Chinese lab Nanbeige released a 33B coding model under Apache 2.0 with a 524K-token context window, beating the current open-weight leader Qwen3.8-27B on 18 of 20 benchmarks including a 54% lead on DeepSWE.
Of 1,357 AI-powered medical tools approved by the FDA, only three were evaluated on patient outcomes. The rest cleared on technical benchmarks with no proven connection to whether patients fare better.
Google DeepMind unveiled Gemini 4 Argon, a frontier model targeting enterprise legal reasoning, finance, and cyber defense — ranking third on the Artificial Analysis Intelligence Index.
Anthropic's Fable 5.1 now leads the SWE-bench coding benchmark at 38.8% — the first time Anthropic has held the top coding position against OpenAI and Google.
Cognition's SWE-2 matches the frontier coding leaderboard at 64% lower cost — and leads all benchmarked models on Terminal-Bench 2.1 at 92.8%.
Microsoft launched MDASH, a multi-model agentic security system that coordinates AI agents to detect and respond to threats at machine speed — and claims top scores on industry cybersecurity benchmarks.
A new study finds patients consistently withhold symptoms from AI health chatbots, creating a diagnostic gap that undermines real-world accuracy even as AI outperforms doctors on benchmarks.
China installs 295,000 industrial robots annually versus 34,200 in the US — an 8.6-to-1 gap that signals the US-China AI competition is moving from model benchmarks to factory floors.
An OpenAI reasoning model outperformed two experienced ER physicians in a real-world test at Beth Israel Deaconess Medical Center — one of the first head-to-head comparisons conducted with live clinical data rather than curated research benchmarks.
A study published April 29 finds AI systems produce correct answers while fundamentally failing to understand the underlying concepts — challenging benchmarks used to certify AI for high-stakes deployment.
DeepSeek released V4 Flash and V4 Pro on April 24 with top-tier coding benchmarks and a 1-million-token context window—open-source, again, at a moment when Silicon Valley assumed the gap was widening.
Frontier AI models now solve real software engineering tasks with near-perfect accuracy — but the same report finds leading AI systems are disclosing less about how they work than ever before.
The Stanford AI Index 2026 finds top AI agents complete complex scientific research tasks at half the rate of human PhD experts — a significant check on agentic AI hype, even as frontier models exceed 50% on Humanity's Last Exam.
The 2026 Stanford AI Index found China closed the US AI performance lead from 9.26% to 1.70% in one year — while major labs slashed transparency scores by 31%.
Claude Opus 4.6 and Gemini 3.1 Pro have passed 50% on Humanity's Last Exam — a benchmark designed to be unsolvable by current AI systems.