A Defining Moment for the United Nations and the Future of AI

The race for the next United Nations Secretary-General kicked off on April 21 with a series of live-broadcast interactive dialogues, as four candidates made the case for why they were best suited for what is often described as “the toughest job in the world.”

“The selection of the Secretary-General has widespread consequences that resonate far beyond this building,” declared Ms. Baerbock. “Our choice will shape the direction of the world’s unique multilateral organization and demonstrate whether the United Nations truly represents the more than eight billion people we serve around the world, half of whom are women and girls.”

After a decade as UN Secretary-General, Mr. António Guterres is stepping down. His departure comes at a critical moment for global AI governance.

During Guterres’ tenure, AI emerged as a transformative technological force that promised potential to reshape economies, societies, and security, and rapidly became central to governance debates worldwide. Guterres has repeatedly warned that “we must move fast, be creative, and work together to ensure adequate guardrails and ethical standards. AI must not replace human agency. It was created by humans and must always be under human control.”

In response to these disruptions, the United Nations has emerged as a leading force in shaping and advancing AI governance grounded in human rights.

The UN’s role in AI governance

Under Guterres’ leadership, the UN produced a series of landmark frameworks that laid the foundation for human rights-based AI governance. The 2021 UNESCO Recommendation on the Ethics of Artificial Intelligence became the world’s first widely adopted global standard on AI ethics. The subsequent release of the Ethical Impact Assessment and the Readiness Assessment Methodology supplement the UNESCO Recommendation by helping governments evaluate their readiness to develop AI ethically and responsibly while identifying critical institutional gaps.

The UN High Commissioner for Human Rights at the time — Michelle Bachelet, the former president of Chile — led the efforts behind the 2021 Office of the United Nations High Commissioner for Human Rights (OHCHR) report, The Right to Privacy in the Digital Age, which drew attention to the use of AI by States and businesses and its impact on the enjoyment of the right to privacy and related rights. The report recommended “expressly ban[ning] AI applications that cannot be operated in compliance with international human rights law and impos[ing] moratoriums on the sale and use of AI systems that carry a high risk for the enjoyment of human rights,” and became a foundational resource for organizations advocating for the establishment of red lines.

In an effort to foster a more cohesive, human-centered global governance framework, the UN Secretary-General’s High-Level Advisory Body on AI (HLAB-AI) released the Governing AI for Humanity report, shifting the conversation from principles to institutional design. During the development of the report, the Center for AI and Digital Policy (CAIDP) submitted a five-point plan to HLAB-AI, proposing the creation of an international panel on AI to monitor societal impacts of AI, a contribution that informed the report’s recommendations. The report helped catalyze concrete governance architecture, including the Global Dialogue on AI Governance and the Independent International Scientific Panel on AI. Together, these initiatives reflect a crucial growing international convergence around human rights-based, inclusive, and equitable approaches to AI governance.

The inaugural Global Dialogue on AI Governance, held July 6-7 in Geneva, brought together world leaders, policymakers, civil society, and academia to discuss the future of AI. During the Dialogue, the Independent International Scientific Panel on AI released its Preliminary Report, detailing both the risks and opportunities presented by AI. The report underscored the urgent need for guardrails, presenting evidence of human rights violations, increased harms to children and women, and the erosion of democratic institutions.

At the Dialogue, Secretary-General Guterres also announced several new initiatives, including the AI Child Safety Pledge, the Global Network for Exchange and Cooperation on AI Capacity Building, the Global Fund for AI, and the AI Environmental Transparency Initiative. Building on the Preliminary Report and these initiatives will be a key task for the next Secretary-General, ensuring continued momentum toward Guterres' vision of a future where AI is "built by humanity, with humanity, and for all humanity."

The UN has also played a leading role in addressing the implications of AI use in the military domain. Secretary-General Guterres has repeatedly called for legally binding international rules, declaring that “machines with the power and discretion to take lives without human involvement are politically unacceptable, morally repugnant and should be prohibited by international law.”

Building on this position, the General Assembly adopted resolutions 79/239 and 78/241 on the implications of AI for international peace and security and the impact of lethal autonomous weapons systems (LAWS), stressing that international law was applicable to military uses of AI, and highlighting the importance of human-centric, accountable, safe, secure and trustworthy AI. Moreover, the Group of Governmental Experts (GGE) on LAWS, established under the Convention on Certain Conventional Weapons, continues to advance discussions towards a legally binding instrument prohibiting LAWS that are non-compliant with international humanitarian law.

As military applications of AI become increasingly sophisticated, the next Secretary-General will play a critical role in carrying this work forward and ensuring that the governance of military AI remains grounded in human rights, international humanitarian law, and meaningful human control.

The UN Secretary-General selection process

The selection and appointment of the next UN Secretary-General is governed by a process designed to ensure transparency, inclusivity, and accountability. While previous selections often took place largely behind closed doors, recent reforms have opened the process to greater public scrutiny through live-broadcast candidate dialogues and stakeholder engagement. These reforms have made the selection of the Secretary-General more transparent than ever before, reflecting the growing expectation that the leader of the world's foremost multilateral institution should be accountable not only to governments, but also to the people the UN serves.

This process, set out in Article 97 of the Charter of the United Nations and further elaborated in General Assembly Resolution 79/327 on the Revitalization of the work of the General Assembly, as well as the Joint Letter of the President of the General Assembly and the President of the Security Council, reflects the UN’s continued efforts to strengthen openness and engagement in the selection process.

The UN has also encouraged engagement from a wide range of stakeholders, including civil society, to contribute perspectives on the qualities and priorities needed for the next Secretary-General. The interactive dialogues were structured around two key themes: (1) candidates’ leadership abilities, experience, and skills, and (2) the UN’s three pillars: peace and security, sustainable development, and human rights. All 193 Member States, alongside selected civil society organizations, were given the opportunity to directly question the candidates. CAIDP also submitted questions during this process, asking the candidates to indicate their position on how to (1) strengthen international cooperation on AI, (2) advance UN’s role in advancing equitable, inclusive, rights-based global AI norms, (3) limit weaponization of AI, and (4) support coherence across existing UN initiatives on digital cooperation and AI governance.

Opening of the 81st session of the UN General Assembly

At the opening of the General Debate of the 81st Session of the UNGeneral Assembly, and in what is expected to be his final address to the Assembly as Secretary-General, Guterres named AI as one of four tests of power, warning that "artificial intelligence moved at a speed that has blindsided even those who created it [...] humans are increasingly handing power to machines, with [AI] agents going rogue." He urged responsible pacing and an independently overseen global risk framework.

Guterres’ warning came amid growing concerns about frontier AI, following the OpenAI-Hugging Face incidents in July 2026. During the UN General Assembly’s High-Level Week, the European Commission and more than 20 nations endorsed a joint initiative launched by Finland and Norway calling for stronger control and oversight of frontier AI models. The statement highlighted several requirements for AI governance that CAIDP has long advocated for, including mandatory pre-deployment testing, independent evaluation, transparent safety protocols, international coordination, reporting of safety incidents, and strengthening oversight institutions.

The UN Independent International Scientific Panel on AI also released a thematic brief calling the OpenAI-Hugging Face incident “an early warning of one possible route to more severe future loss of control.” While panel co-chair Yoshua Bengio drew a comparison between the AI agents’ actions and criminal acts if committed by a human, CAIDP Global Academic Network member and Umeå University professor of responsible artificial intelligence Virginia Dignum and her coauthors argued that the thematic brief depicted “corporate misbehaviors as a technical mystery requiring computational fixes.”

The UN is under pressure to step up and take the necessary action that technology companies on their own won’t. Moreover, with major players such as the United States and China rejecting new global governance of AI at the UN-level and later agreeing on a bilateral channel for AI incidents, the political landscape is heating up. It is more important now than ever to elect a Secretary-General that will take charge and lead the change humanity needs.

Leading the future of AI

This is a pivotal moment in history, as the outcome of the selection will shape the future of global AI governance. The central question is which candidate for the UN Secretary-General is best positioned to uphold a future in which human rights, democracy, and the rule of law remain at the center of technological progress.

During his opening remarks at the UN Global Dialogue on AI Governance, Secretary-General Guterres identified four priorities for the road ahead: (1) safety, (2) red lines, (3) capacity, and (4) transparency. The next Secretary-General will have their work cut out for them as Guterres steps down, leaving behind a legacy rooted in advocating for a human-centered AI future. Alongside the much-anticipated reforms under the UN80 Initiative, the incoming Secretary-General will also be responsible for advancing the roadmap set out during Guterres' tenure.

This will require the leadership to guide Member States in human rights and democratic values throughout the AI lifecycle; the resolve to establish and uphold red lines, including prohibitions on lethal autonomous weapon systems and AI systems that violate international humanitarian law; the commitment to the UN’s mission of leaving no one behind by preventing the AI divide from becoming a development, security, and sovereignty gap; and the ability to promote transparency within companies and encourage alignment of their AI practices with the Sustainable Development Goals.

Recognizing that inclusive leadership strengthens institutional legitimacy, CAIDP recommends that the UN select a woman as the next Secretary-General. More than eight decades have passed since its founding, yet the UN has never been led by a woman. As an organization that “serves all of humanity, half of whom are women,” the UN has a timely opportunity to advance gender equality and inclusive governance at its highest office.

Appointing a woman Secretary-General would be a historic step toward gender equality in global leadership and would make the UN more representative of the world it serves.

CAIDP supports a vision of a future in which AI strengthens, rather than undermines, human rights, democracy, and the rule of law. As per the UN Secretary-General Guterres’ warning, “the window is closing to shape AI — for peace, for justice, for humanity.” This moment underscores the urgency of what is ultimately at stake in the selection of the next UN Secretary-General: leadership capable of ensuring that emerging technologies are governed in the service of these principles.

The selection of the next UN Secretary-General is therefore not only a question of institutional continuity, but a critical opportunity to advance a vision of global governance that is inclusive, democratic, and grounded in human rights.

The authors wish to acknowledge April Yodar, Editor-in-Chief, CAIDP AI Index; Grace S. Thomson, CAIDP AI Policy Clinics Director; Jésica Edith Tapia Reyes, CAIDP AI Policy Group Member; Tamiko Eto, CAIDP Senior Teaching fellow; Facundo Bonaura, CAIDP Policy Group Member.