Altman, Amodei Brief UN Security Council on AI Control Risks

For the first time in history, the CEOs of the world's two most prominent frontier AI companies addressed the UN Security Council together. Sam Altman of OpenAI and Dario Amodei of Anthropic joined AI scientist Yoshua Bengio and Hugging Face CEO Clément Delangue at the Council chamber in New York on Wednesday, September 23, in a session convened by France on artificial intelligence and international security. The stakes: whether the world's governing bodies are moving fast enough to prevent AI systems from advancing beyond meaningful human oversight.

The context: The UN Security Council has debated the risks of autonomous weapons and surveillance AI before, but Wednesday's session marked the first time the leaders of frontier model companies — organizations building the most capable AI systems on the planet — appeared together before the body. France, which has championed AI governance at the multilateral level, organized the briefing amid mounting concern that the next generation of large language models may be capable of recursive self-improvement — a term describing AI systems that can iteratively modify and enhance their own code or architecture without direct human involvement.

What they said: According to AP reporting, Altman planned to urge world leaders to adopt two specific mechanisms: capability benchmarks that define thresholds for what AI systems can do, and standardized safeguard assessments that governments and developers could apply consistently across jurisdictions. Amodei and Bengio — the latter a Turing Award–winning computer scientist and a prominent voice on existential AI risk — echoed concerns about control, particularly the risk that sufficiently advanced models could pursue goals misaligned with human intentions. Delangue, whose company Hugging Face hosts thousands of open-source AI models, represented the open-weights ecosystem in a forum that often focuses solely on large commercial labs.

Why this is different from prior UN work: Most AI governance discussions at the UN have happened in the General Assembly, in working groups, or through advisory bodies like the newly established AI Advisory Board. The Security Council, by contrast, is where the body acts on threats to international peace and security. Its five permanent members — the United States, China, Russia, the United Kingdom, and France — each hold veto power over any resolution. That France chose to bring AI before this forum signals a deliberate escalation in how some governments categorize the technology's risk profile.

No binding resolution was expected from Wednesday's session. Council sessions of this type are typically informational, designed to shape member-state positions rather than produce immediate action. But the symbolism is hard to dismiss: getting Altman and Amodei in the same room, at the Security Council table, is a different kind of signal than a joint statement or conference keynote.

What it means for the industry: AI companies have spent the past two years navigating a patchwork of national and regional regulatory frameworks — the EU AI Act, the Biden-era executive order (since amended under the Trump administration), and a wave of state-level legislation. A UN Security Council session does not create law. But it does create political pressure that can accelerate national legislation and international agreements. If member states leave Wednesday's briefing convinced that recursive self-improvement is a near-term threat rather than a hypothetical, that calculus shifts quickly.

For companies building and deploying AI, the immediate practical consequence is heightened scrutiny. Expect more government requests for model evaluations, capability disclosures, and third-party audits in the months ahead — regardless of what any resolution or communiqué says.

What to watch: Whether any permanent Security Council members — particularly the U.S. or China — signal interest in a binding international framework in the days following the briefing. A joint statement from even two major powers would represent a meaningful step. Also watch for Altman's full remarks to be released; the specifics of his proposed capability benchmark thresholds will carry far more weight than the broad call to action.

By Hector Herrera