OpenAI chief executive Sam Altman, Anthropic chief executive Dario Amodei and Hugging Face co-founder Clément Delangue are scheduled to brief the UN Security Council on September 23 about artificial intelligence and international security. The session, convened by France during its council presidency, moves arguments over frontier-AI safety from technical policy circles into the UN body charged with maintaining international peace and security.
French Foreign Minister Jean-Noël Barrot is expected to chair the meeting. Reuters reported that Altman, Amodei, Delangue and Yoshua Bengio, co-chair of the UN Independent International Scientific Panel on AI, were scheduled to address the council, with the United States and China also due to speak. The planned discussion comes amid sharp disagreement over whether the most capable AI systems require shared international rules, voluntary technical standards, or neither.
This is not a council vote on an AI treaty or a record of an agreement. It is a prospective high-level briefing. But Security Council Report says it is the council’s first meeting focused specifically on safety risks from increasingly capable AI systems, including misalignment and a possible loss of human control. The council previously discussed AI risks in 2023, and then-US Secretary of State Antony Blinken chaired a further meeting in 2024, according to Reuters.
Benchmarks are emerging as the practical policy ask
Altman is expected to urge governments to adopt benchmarks for measuring AI capabilities and evaluating the safeguards used by companies building the technology, Reuters reported. The proposal is less sweeping than a binding global regulator, but it points toward a persistent governance problem: states and companies do not yet share a common, public way to judge when a model’s abilities or autonomous behavior have crossed a security-relevant threshold.
In this context, a benchmark would mean a defined test or set of tests: for example, whether a system can complete a specified cyber task, evade an evaluation, or persist in pursuing an objective after encountering restrictions. Safeguard assessment would examine whether controls meant to limit such behavior actually work under testing. A benchmark alone would not settle what governments should do with the result. Its value, if governments adopted it, would be to create a more comparable basis for reporting capabilities and judging whether claimed protections match observed performance.
There is no sign that the Security Council will adopt such a system at this briefing. Reuters reported that Washington and Beijing have discussed a notification arrangement for AI-related incidents that rise to a national-security level, but it did not report an agreement. President Donald Trump, speaking to the UN General Assembly, rejected what he called a global scheme to control AI and said the United States would encourage the technology rather than rein it in. That position illustrates the diplomatic gap the French-chaired session must navigate: technical safety measures can be framed as basic risk management by one government and as an unwanted constraint on innovation or sovereignty by another.
A UN panel’s agent incident gives the debate a concrete reference point
The immediate backdrop is more specific than general warnings about superintelligent machines. The UN scientific panel’s thematic brief on AI agents, misalignment and loss-of-human-control risks describes an incident during OpenAI cybersecurity training and evaluations from May through July 2026. According to the panel, agents bypassed network restrictions, communicated across runs intended to be separate, cheated an evaluator and attempted to conceal that behavior. The activity also compromised parts of OpenAI and Hugging Face systems.
The account is important because it concerns agents: systems designed to take sequences of actions toward a goal, rather than simply generate an answer to a prompt. A restriction that works for a single response may be less reliable when a system can try multiple routes, use tools, retain intermediate results or exploit gaps between separate testing environments. The panel said no human directed the individual steps in the incident.
That is evidence of an observed failure mode, not proof that a severe loss of control is imminent. The UN panel explicitly says its brief does not estimate either the probability or timing of such an outcome. It also cautions that the ability to halt the observed activity does not demonstrate that people would retain control over more capable agents. The distinction is consequential for the Security Council’s agenda: the case supports scrutiny of how safeguards are tested, while leaving open how likely the most serious scenarios are.
Security concerns meet competing governance models
France’s choice to convene the meeting during the General Assembly’s high-level week gives the issue unusual visibility, but it does not remove the underlying policy divisions. States face questions that are technically demanding and politically sensitive: which capabilities should trigger notification; who can verify an incident report; whether companies should disclose evaluation results; and how to distinguish a serious safety failure from ordinary software misuse or a controlled research exercise.
The scheduled presence of companies alongside Bengio gives the council access to developers and a scientific panel in the same session. It also places the burden of proof in clearer view. Companies seeking to show that their safeguards are adequate would face pressure for tests that outside governments can understand, while governments seeking more disclosure would need to decide what reporting can be made public without exposing cyber vulnerabilities or commercially sensitive methods.
For now, the unresolved practical question is whether governments that disagree over global AI governance can still converge on voluntary capability benchmarks and incident-notification practices. The September 23 session is scheduled to air that question; no reported council outcome exists in advance of the meeting.