Can AI Agents Be Aligned with Human Rights?

Raquel Vazquez is an AI Policy Lead at Google, Rafiya Javed is a Senior Research Engineer at Google DeepMind, and Dr. Vinodkumar Prabhakaran is a Sr. Staff Research Scientist at Google Research.

Earlier this month, the UN Secretary-General called for a global framework to govern AI and released a Preliminary Report by the Independent Scientific Panel on AI, setting the stage for the inaugural UN Global Dialogue on AI Governance held in Geneva. As the global community strives to advance a rights-respecting framework, we have an opportunity to answer a practical challenge: How do we actually embed human rights within AI training pipelines? This question becomes urgent with the rise of agentic AI.

Much of the energy is focused on understanding the impacts of AI post-deployment, after the model has already been built and shipped. While critical for AI governance, this approach does not proactively guide AI agents to operate safely when they encounter unfamiliar settings. Unlike chatbots, AI agents can reason, plan, use digital tools and act over long horizons. Because the outcomes ripple across digital ecosystems and may impact people who never prompted them, it is important to ensure they act safely—not just taking into account direct users, but society at large.

The translation gap in agentic safety

Alignment is the field of research dedicated to ensuring AI behaves in accordance with human values, intentions, and ethical principles. The conversation about what values a model should follow is not new, nor is the idea that AI should respect human rights. Historically, human rights discussions in machine learning have largely been treated as normative exercises: important in theory, but disconnected from scientific advances and too subjective for mathematical optimization. As a result, most of the discussions at the intersection of human rights and AI focus on gaining evidence about the behavior of an already-trained system and building guardrails to manage these risks (this is what one would term “backward alignment”).

Our research shifts the focus to “forward alignment”—teaching a frontier model to perform well against an objective early on, before deployment. Until recently, the idea of embedding human rights law and principles into a model during the training phase would have been premature, but today’s models can process complex, multi-step reasoning chains. This brings us closer to bridging the translation gap, and being able to convert abstract principles and the law into concrete training signals. This capability is critical for agents navigating novel areas of discretion and weighing trade-offs that will impact secondary stakeholders or non-users.

Human rights vs. AI risk taxonomies

In our just released paper, we perform an exploratory, proof-of-concept experiment translating the Universal Declaration of Human Rights (UDHR) into an alignment target. Our approach goes beyond treating human rights as a compliance checklist, instead illustrating that human rights law and principles can provide reward signals during training that can be uniquely valuable for model development. To test the feasibility of this approach without excessive compute overhead, we evaluated our framework across 100 simulated agent failure scenarios using two highly accessible and lightweight models, Gemini-2.5-Flash and GPT-5-mini, as “auto-raters” (or LLMs-as-judge).

Our research compares how the auto-raters flag errors and resolve the failure scenarios under two different frameworks: our Human Rights Taxonomy incorporating the UDHR and core international human rights principles; and the AI Risk (AIR 2024) Taxonomy, a comprehensive baseline compiling safety rules from 8 government policies and 16 major corporate guidelines worldwide.

In comparing evaluations under these two frameworks, we observe differences in how the auto-raters assess the same interaction. To take one example from our proof-of-concept experiment, consider a case where a user asks a Patent Assistant Agent to perform a prior art search for a newly developed self-locking surgical suture. The agent misses a piece of prior art described in a Japanese manual, and incorrectly concludes that the suture design is patentable. While the two taxonomies spot the mistake, their explanations diverge. Under AIR 2024, the error is flagged under the principle of not giving "advice in heavily regulated industries” and anchors the risk in terms of business liability. The resulting feedback is defensive, recommending that the agent frames its findings as preliminary and non-binding, includes legal disclaimers, and advises to consult a patent attorney.

In contrast, under the Human Rights Taxonomy, the evaluation shifts from corporate liability to downstream societal impact. It recognizes that recommending a restrictive patent on a medical device can impact the right to health by potentially blocking affordable public access to medical technology. It guides the agent to prompt the user to consult clinical, toxicological, and legal experts, and to consider other IP strategies, such as humanitarian licensing or patent pooling, to protect access in low-resource settings.

While there is still much to explore, our proof-of-concept experiment supports the thesis that human rights as an alignment target provides important insights that extend beyond capturing agentic failures. Our setup acts as a proxy for how an AI model could be steered by other training techniques, such as RLAIF or constitutional methods. These techniques reward models for producing outputs that align to a given rule or statement (this is what we call in machine learning “policy”). The human rights taxonomy offers a "normative vocabulary" that guides the agent to reason about human impacts, incorporating other stakeholders beyond the direct user. This distinction is critical for forward alignment: while other harm taxonomies may teach a model to avoid liability, a human-rights-based signal guides the model to consider broader impacts and mitigate risks to rights-holders.

Turning legal concepts into technical heuristics

Why look to human rights? While there is increasingly more research in moral fragility and robustness that identifies factors that should not change moral outcomes, there is little study of whether models are sensitive to factors that exacerbate or mitigate harm. Defining what constitutes "harm" is often highly subjective and politically contested. In this context, human rights law offers a unique advantage: rather than relying on localized, subjective, or corporate safety standards, it provides a globally recognized, legally codified framework for establishing a shared understanding of harms. Furthermore, it can help navigate agentic safety challenges because it offers a decision-making structure and heuristics that have been refined by academics, civil society and courts over decades.

To bridge the gap between disciplines, we incorporated two of these heuristics (vulnerability and irremediability) directly into the evaluation rubrics of our auto-raters. Under the vulnerability notion, disadvantaged groups require additional measures of protection. Instructing the agent to weigh vulnerability forces it to exercise heightened caution in contexts where its actions could disproportionately harm people at a disadvantage. The irremediability principle measures whether a harm can be easily reversed. A financial error can typically be remedied with compensation, whereas the loss of health or life cannot. Integrating irremediability guides the agent to pay attention to consequences that may be difficult to undo.

By translating concepts like “irremediability” and “vulnerability” into specific rubrics for the model, we can provide the agent with a path to weigh severity and determine when it must exercise heightened caution.

New ideas for research on technical AI governance

Operationalizing these concepts is just the beginning, and expanding on this work requires investing in at least three research areas. As a first step, we need safety benchmarks that capture downstream human rights impact beyond specific failure modes or mechanisms of harm. Here, international human rights law offers concepts, such as irremediability, which can help benchmark how well a model can tune its decision-making to the degree of human rights impact.

Second, we must develop methods to handle trade-offs when rights clash. No single international document can perfectly resolve these tensions in a vacuum, and different legal cultures will prioritize these trade-offs differently. An interesting technical and legal challenge is to design 'conflict-resolution rules' that prevent agents from making these balancing decisions on their own, instead guiding it to refer, for instance, to jurisprudence and context-specific or culturally-sensitive frameworks.

Third, the development of alignment instruments (such as model constitutions) should involve human rights law experts and civil society, not only employees. If we are to move beyond reactive compliance checklists, the policy community must work alongside computer scientists to define the reasoning capabilities we expect from agentic AI.

Ultimately, this leads to a fundamental question: Do we incorporate human rights law and principles into existing machine learning training pipelines, or do we need to adapt technical architectures to the structures of legal reasoning? The answer will play an important role in shaping the future of AI governance and safety.