Secretary-General Antonio Guterres speaks during press conference ahead of General Assembly High-level Week 2026 at UN Headquarters in New York, NY on September 16, 2026.(Sipa via AP Images)

Can the United Nations Help Slow the AI Race?

The setting sun reflects off an Anduril Sentry tower in Luna County, New Mexico. EFF/Wikimedia Commons October 2022

The setting sun reflects off an Anduril Sentry tower in Luna County, New Mexico. EFF/

The Influence of AI on Human Decisions in DHS Surveillance

Interior view of House of Commons at Canada Parliament. Ottawa, Canada - December 15, 2025. Shutterstock

Interior view of House of Commons at Canada Parliament. Ottawa, Canada - December 15, 2025. 

Canada Is Not Banning Teenagers from Social Media — It’s Trying Something New

ICE agents stand guard outside Delaney Hall detention center in Newark, New Jersey, on Thursday, May 28, 2026. (Photo by Cristina Matuozzi/Sipa via AP Images)

From Records to Raids: The Questionable Legality of ICE’s Palantir ELITE System

European Commission President Ursula von der Leyen delivers a speech at the annual State of the European Union at the European Parliament in Strasbourg, Wednesday, Sept 16, 2026. CC-BY-4.0: © European Union 2026– Source: EP

Europe Says Its AI Rules Are Enough. AI Agents Are Testing That Claim

United States President Donald Trump steps off from Marine One upon his arrival at the Ellipse just outside of the White House, Saturday, Sept. 19, 2026, in Washington. (AP Photo/Jose Luis Magana)

AI Hacks, a Possible War Crime, and a Near Miss With China Set Stage for Trump-Xi Meeting

The Google sign on the wall behind the reception desk at the Google headquarters in Silicon Valley, August 2023. Shutterstock

The Google sign on the wall behind the reception desk at the Google headquarters in Silicon Valley, August 2023. 

A Deep Dive into the Antitrust Remedies that Spared Google's Ad Tech Business

Brushy expressionistic illustration of man screaming in rage, sound and fury spewing from his mouth.


Matthias J. Becker on the Launch of Digital Hate Review

Ambiance at the Global Dialogue on AI Governance, Palexpo, Geneva, Switzerland - 07 July 2026.

The Global Digital Compact was adopted at the UN General Assembly in New York City on September 23rd, 2024. 

UN's Global Digital Compact Review Could Sideline the Global South

An official of the Lagos State Independent Electoral Commission (LASIEC) marks a voter's finger with ink in Lagos, Nigeria, on July 12, 2025. (Photo by Adekunle Ajayi/NurPhoto via AP)

When Nigeria’s Election Technology Cannot Recognize the Voter

A Google logo embedded in a cracked, black surface. The cracks radiate outward, giving the appearance that the logo is breaking through or shattering the material. The Google logo retains its signature colors: blue, red, yellow, and green.

Can the EU Still Break Up Google?

Artwork including a giant bust of Elon Musk sits along the route the billionaire's SpaceX Starbase south Texas launch site. (Photo by Laura Brett/Sipa via AP Images)

What Policymakers and the Public Need to Know About the Cult of AI

A bald eagle, signifying the US Department of Justice, next to the Google logo.

The Google Remedies Turn Landmark Verdicts Into a Whimper

CEO of Anthropic Dario Amodei, addresses the gathering at the AI Impact Summit, in New Delhi, India, February 19, 2026. REUTERS/Bhawika Chhabra

Who Should Pace the Frontier? Not Dario Amodei

A traditional Chinese arch bridge over green water surrounded by trees sits in front of large skyscraper buildings.

A traditional Chinese arch bridge sits in front of the 'Huijin International Building' — the building that hosts DeepSeek's headquarters, in Hangzhou, Zhejiang Province, China, on March 18, 2026. (Photo by STR/NurPhoto via AP)

China Is Running a Different AI Race

iPhones are displayed during an announcement of new products on Wednesday, Sept. 9, 2026, on the Apple campus in Cupertino, Calif. (AP Photo/Jeff Chiu)

Will Apple’s ‘Reference Image’ Feature Help Defend Against AI Manipulation?

Hundreds attend a Town of Tonawanda Planning Board meeting in Kenmore, New York on Wednesday, June 3, 2026, as they expressed vocal and at times boisterous opposition about a proposed data center on the site of the now closed Tonawanda Coke facility. (Photo by Craig Ruttle/Sipa via AP Images)

Americans Shrugged Off Surveillance. Then Came Flock and Data Centers.

Meta founder and CEO Mark Zuckerberg attends the Allen & Company Sun Valley Conference at the Sun Valley Lodge on July 9, 2026 in Sun Valley, Idaho. (Photo by Kevin Dietsch/Getty Images)

Two Reasons Meta’s Settlement with State Attorneys General is a Win for Mark Zuckerberg

A protester holds a 'Don't let AI take our future' placard during the demonstration by the group Pause AI outside Downing Street. (Photo by Vuk Valcic / SOPA Images via AP)

The Like Button Helped Social Media Put Profit Over People. We Can’t Let AI Do the Same.

Anthropic CEO Dario Amodei at Salesforce's Dreamforce conference at the Moscone Center on September 15, 2026 in San Francisco, California. (Photo by Benjamin Fanjoy/Getty Images)

Making Sense of AI Dread? Don't Get Distracted by Doomer CEOs or the China Canard

 that Immigration and Customs Enforcement (ICE) uses a Palantir-developed tool called Enhanced Leads Identification & Targeting for Enforcement (ELITE) to support immigration enforcement operations. According to internal ICE materials reviewed by 404 Media, ELITE can populate maps with potential deportation targets, generate dossiers on individuals, and assign a confidence score to a person’s current address—all using data drawn from various government databases, private data brokers, and the open internet.

In conflict with this reporting is DHS’s 

 of the ELITE system as a tool that produces “normalized address data” and is not supposed to serve as a “principal basis” for legally significant decisions, taking it out of the category of a “high impact” AI system that would be subject to more exacting standards and oversight. Subsequent reporting and sworn officer testimony indicate that ICE not only uses ELITE to look for individual targets, but it also uses the tool to identify “target rich” areas believed to contain a dense concentration of people with an “immigration nexus.”

ELITE transforms administrative, public, or private data—much of which is mundane—into operational intelligence for field immigration enforcement. But the public record is still bereft of information about ELITE’s precise data sources, accuracy, contractual status, and human-review safeguards, as well as how agents are trained to use it and its actual role in investigation, stops, arrests, detention decisions, and removal proceedings. This article analyzes 

 about the system to identify the most important legal questions raised by ELITE: whether it enables suspicionless, profile-based, or sweep-style enforcement; whether people targeted through the system can meaningfully interrogate the basis for their arrest or detention; and whether ICE’s formal claim that ELITE is merely address-normalization software matches how the tool is actually used in the field.

What exactly did ICE procure: ELITE, ImmigrationOS, or a broader Palantir enforcement capability?

The public record does not yet answer this cleanly. ELITE appears to be a Palantir-developed application, use case, or adjacent capability within ICE’s broader Palantir enforcement stack. That stack includes Palantir’s long-running Investigative Case Management (ICM) system and the ImmigrationOS overlay, which was publicly described as including “Targeting and Enforcement Prioritization, “Self-Deportation Tracking,” and immigration-lifecycle logistics. The overlap between those workstreams and ELITE’s reported functions is substantial, but the precise contractual relationship remains unresolved. The legality of the procurement depends on what ICE actually bought, how it described the work, and whether the agency bundled new enforcement-targeting capabilities into an incumbent Palantir contract.

Did DHS follow relevant laws and procedures in procuring the ELITE software?

Maybe, but the better question is narrower: whether ICE properly used a noncompetitive or urgent-and-compelling justification to expand Palantir’s role in immigration enforcement, and whether ELITE falls within that expansion. ICE awarded Palantir a sole-source ImmigrationOS contract, justified on urgency and Palantir’s unique capacity to deliver the system, invoking 

a Federal Acquisition Regulation (FAR) exception

 that allows DHS to circumvent the competitive procurement process. If ELITE was built under, added to, or functionally enabled by that work, then the procurement question becomes whether ICE legally treated Palantir’s incumbent status as a reason to avoid competitive bidding. DHS may have violated the 

 (CICA) by not seeking competing company’s bids for the development of this software.

That question cuts both ways. ICE would likely argue that urgent enforcement priorities, interoperability, and Palantir’s long experience with ICE systems made a noncompetitive award lawful. The harder counterargument is that vendor lock-in cannot automatically become its own justification. If an agency’s existing dependence on Palantir is what makes Palantir the only practical supplier, then the urgent-and-compelling rationale risks converting past procurement choices into indefinite sole-source control over new enforcement infrastructure.

Did ICE comply with federal privacy, matching, and data-governance rules when developing ELITE?

Potentially not. ELITE software is reported to aggregate address data from the Department of Health and Human Services (HHS) and cross-reference it with ICE data, including the Enforcement Integrated Database. That raises serious Privacy Act, 

Computer Matching and Privacy Protection Act

 (CMPPA), system-of-record notice, privacy impact assessment, and routine-use questions. The CMPPA requires that where an agency is using a person’s information in a “matching program,” such as here where ELITE provides an address confidence score by aggregating data, the individual must be notified and be permitted to verify their information. This step 

must occur before any “adverse action” is taken against the individual

requires a formal agreement between the agencies

But the existing public record is not strong enough to confirm whether ELITE is a statutorily covered “matching program.”

Further, there is no clear indication that people targeted via the ELITE application were notified that their data was being matched, nor were they provided with an opportunity to verify such information. There is also no public record of a formal matching agreement between the agencies.

At a minimum, ELITE exposes a gap between data-sharing law and operational enforcement reality. If ICE is using HHS-derived address records or other interagency data to generate leads, then the key legal questions are: which systems of record supplied the data; what routine uses or data-sharing agreements authorized the transfer; whether the system is a “matching program”; and whether the records used were accurate, relevant, timely, and complete.

Does ICE’s use of ELITE violate the federal AI-governance requirements or DHS’s own classification of ELITE?

Maybe. The Trump administration announced its intention to expand AI use in 

. But such expansion includes “risk management practices” when an AI is high impact—meaning it “could have significant impacts when deployed.” Here, DHS 

 high impact, meaning its use does not require a risk management practice. It’s not clear whether this determination may be challenged, but without designating ELITE as a high impact AI, it is not likely that DHS is required by OMB Memorandum M-25-21 to engage in risk management practices.

The legal question, then, is whether DHS’s low-impact or non-principal-basis characterization matches field reality. If ELITE merely cleans address data later verified by agents, DHS’s classification is defensible. But credible reporting suggests that description is misleading at best. If ELITE, as reporting suggests, materially steers stops, sweeps, home visits, arrests, or detention-related decisions, challengers can argue DHS understated the system’s impact and failed to comply with mandatory disclosure, safeguard, testing, validation, monitoring, and human-review requirements. The OMD AI-governance memo, however, does not on its own create a private right of action; even if it is being violated, that is not sufficient to bring a cause of action.

What data feeds ELITE, and was it lawfully collected, shared, and repurposed for immigration enforcement?

ELITE combines HHS-derived address data and combines it with ICE data. That ICE data may include information ICE already possesses and claims to have lawfully obtained, including records incorporated from other agencies, prior enforcement systems, or commercial vendors. But the full source architecture is unknown: the public record does not establish whether additional data streams—such as DMV data, ALPR data, mobile-location data, tipline data, or brokered records—feeds directly into ELITE. Given that ELITE appears to be built off of or at least interoperate with Palantir’s existing systems, including ImmigrationOS, which does draw from these data streams, there is reason to believe ELITE can and is accessing this data.

Is the Fourth Amendment implicated when the information used in ELITE was gathered in the first instance?

Maybe, depending on the source. Where ICE collects data directly, the Fourth Amendment demands judicial process—generally a warrant or legitimate application of an exception to the Fourth Amendment—if the data collection is deemed a search. Per 

a search occurs when the government violates a reasonable expectation of privacy. 

 held that continuous monitoring of location data, even in public spaces, does violate a reasonable expectation of privacy. The thorny question is whether individuals retain a reasonable expectation of privacy in data they voluntarily hand to third parties—traditionally, courts have held they do not, because of the “third-party doctrine.” However, recent cases have signaled a narrowing of the third-party doctrine, which opens the door to challenges to the ELITE system if it uses data obtained from other sources without first obtaining a warrant.

Broadly speaking, the Fourth Amendment does not prohibit the government from purchasing brokered data without judicial process, though many argue this is a 

. But some forms of location-data collection from third-parties may be searches requiring judicial process. 

 held that when the government obtains a large volume of location data over a long enough period of time from a third-party cell-service provider, they have conducted a search under the Fourth Amendment and therefore must abide by its requirements. Other court cases have also suggested that movement tracking through third-party ALPRs 

 trigger Fourth Amendment protections. The recent SCOTUS decision in 

strengthens the argument that even a small handful of location data obtained using a geofence warrant can trigger individualized Fourth Amendment protections.

So, if ELITE truly relies on ordinary address records lawfully collected by ICE or lawfully obtained from HHS, the Fourth Amendment argument is harder. If it does indeed rely on third-party data, such as ALPR or mobile-location data, the argument is stronger.

Does indefinitely retaining location or identity data in government databases amount to a Fourth Amendment seizure?

Potentially, but this is a developing argument. The theory is that even if each data point was lawfully collected, indefinite retention and aggregation may give the government a durable surveillance archive that is materially different from the search, thereby meaningfully interfering with a person’s possessory interest or rightful control over their personal information.

Courts more commonly analyze government access to aggregated data as a search, not a seizure, so this is not the strongest or most settled Fourth Amendment theory. But it is a colorable argument where the government maintains the ingredients for personal dossiers indefinitely without individualized legal basis. For ELITE, the key factual question is whether ICE is merely storing discrete records, or maintaining a searchable, cross-source location profile capable of revealing where someone lives, works, receives services, or is likely to be found.

Is querying ELITE to verify an address, generate leads, or identify targets itself a Fourth Amendment search?

Even if the initial collection, integration, and indefinite retention of data is legally justified, it is an open question whether the algorithmic query or output creates a new constitutional event: the government uses a system to synthesize scattered records into an address-confidence score, target map, or enforcement lead.

has not been formally adopted, but it has been persuasive, which supports the idea that the Fourth Amendment has something to say when 

 “reveal[s] information that, in the absence of aggregation, the government could only access by conducting a search or seizure, the extraction of that information should be subject to constitutionally based limits.” Constitutional concerns about data aggregation and retention liken it to 

The government would likely argue that querying a database is not a search if the underlying records are lawfully available. The counterargument is that ELITE’s output may reveal something no single record reveals on its own, making the generation of an output a separate action that potentially violates a reasonable expectation of privacy absent a warrant.

3. Use of the output: reasonable suspicion, probable cause, and due process

Does an ELITE-assisted stop satisfy the Fourth Amendment’s reasonable suspicion requirement?

Maybe. A tool does not itself “meet” reasonable suspicion; an officer must have specific, articulable, individualized facts that justify the stop. Per 

, officers may not rely on race or ethnicity alone to justify an immigration stop, but that those features can be part of a totality of the circumstances evaluation of reasonable suspicion. The legal questions become whether the system’s recommendation of a target can itself meet the reasonable suspicion standard, in that it its determinations are based individualized, specific, articulable facts that are not based on race of ethnicity; whether the system’s recommendation of a target can play a role in conjunction with other factors to establish reasonable suspicion, even if its recommendations over-index to race or ethnicity; and whether operations on the ground use ELITE as the functional sole basis for a stop even while claiming it is not the “principal basis” of that legal decision, which would be illegal if the system outputs fail to meet the reasonable suspicion standard on their own.

Recent litigation complicates the analysis but does not eliminate the individualized-suspicion requirement. In 

, the Supreme Court granted an emergency stay of a district-court injunction that had restricted profile-based immigration stops in Los Angeles. The Court did not issue a merits opinion. But, 

Justice Kavanaugh’s separate concurrence

 suggested that reasonable suspicion can be established by a combination of factors including presence at locations such as bus stops, car washes, day-labor pick up sites; the type of work a person appeared to do; speaking Spanish or English with an accent; and apparent race or ethnicity. Commentators and litigants have treated this concurrence as permitting stops based primarily on apparent ethnicity or race—giving rise to the phrase “Kavanaugh stops”—although Kavanaugh appears to have distanced himself from that reading.

For ELITE, the legal questions hinge on a fundamental inquiry: how much does the system operationalize broader proxies for race and ethnicity—neighborhood, worksite, language, prior immigration contact, or “immigration nexus”—in generating target leads or identifying hot spots. If its algorithmic outputs are based predominantly on impermissible considerations of race or ethnicity, ICE agents’ reliance on the system to justify stops is on shakier legal grounds.

Does ICE’s use of ELITE to engage in neighborhood “sweeps” violate the Fourth Amendment?

Maybe. Sworn testimony described ELITE as helping officers identify “target-rich areas where arrests were more likely. But area-level suspicion is not individualized suspicion and cannot, on its own, justify stops. If ICE uses ELITE to decide whether to deploy agents, then agents stop people merely because they are present in the area or fit a broader profile, the decisions may not be sufficiently supported by specific and articulable facts of individualized reasonable suspicion.

Does ELITE provide a lawful basis for warrantless civil immigration arrests?

Not by itself. A reasonable suspicion stop and a civil immigration arrest are different events. Even if ELITE helps ICE locate someone or conduct an immigration stop legally, ICE still needs a lawful basis to escalate that encounter into an arrest. Under the 

 (INA), ICE may arrest someone if the officer has (1) “reason to believe” the person is in the US in violation of immigration law; and (2) is “likely to escape before a warrant can be obtained.” ELITE may help satisfy the first prong, but an address-confidence score does not establish likelihood of escape and may even cut the other way if ICE has a tool that can tell them with relative confidence where the person can be found.

Does ICE’s use of ELITE violate due process under the Fifth Amendment?

Potentially, but the claim should be tied to a specific deprivation. Because ICE’s use of ELITE is occurring in a context where the public is not aware of the exact data inputs, it is increasingly difficult for people who are targeted by the ELITE application to contest their detention. But, opacity alone is not automatically a due process violation. Due process, per the 

 balancing test, requires notice and a meaningful opportunity to be heard before the government deprives an individual of a right or privilege. Due process is most clearly implicated when ELITE-derived information contributes to detention, bond, charging, removal proceedings, or other liberty-affecting decisions and the person is left without recourse to challenge the deprivation.

 requires that the risk “of an erroneous deprivation” of a “private interest” be weighed. ICE agents themselves have 

; it can be wrong even when it reports high confidence and officers are supposed to perform additional checks, which there is 

, given they treat outputs from similar identification platforms like Mobile Fortify that are also error-prone as “definitive.” If those checks are not done, documented, or disclosed, ELITE may make it practically impossible for a person to challenge stale, mistaken, or unverified data to vindicate their Fifth Amendment rights.

4. Procedural and doctrinal barriers to raising issues in court

Who has standing to challenge ICE’s use of ELITE in immigration arrests?

constitutional requirement to proceed in court

, and requires that a person suffer (1) an injury in fact, that is (2) caused by the defendant, and (3) is redressable by judicial intervention. A person stopped, arrested, detained, or subjected to a home encounter because of ELITE may have a concrete injury. But, proving causation may be difficult because ICE is not likely to disclose whether ELITE was used, what it showed, or whether agents independently verified output. Systemic suits—often the route impact litigation pursues—face additional barriers: plaintiffs seeking forward-looking relief must show a real likelihood of future injury, which will be challenging without access to ELITE system specifications, use procedures, audits, logs, and error reports. Additionally, immigration-related claims may be channeled through removal proceedings, habeas, petitions for review, or limited district-court actions—some of these fora do not confer on immigrants the same constitutional and civil procedure protections available in traditional court proceedings. In practice, the first litigation battles may have to focus on plaintiffs obtaining enough records to prove ELITE’s role in enforcement.

Can ELITE-related violations lead to suppression, termination, or evidentiary relief in removal proceedings?

Sometimes, but this can be a steep hill to climb. In civil removal proceedings, unlawful stops and arrests do not automatically lead to suppression of evidence. In 

, the Supreme Court held that an unlawful immigration arrest does not automatically make the government’s evidence unusable or require the case to be dismissed. Suppression is generally available only for especially serious constitutional violations, such as of the Fourth Amendment, or systemic unlawful practices, or the serious compromise of evidence reliability. Termination, or the dismissal of a removal case, is even harder to obtain, usually requiring that either the government cannot prove removability without the tainted evidence or that the violation caused serious prejudice.

More granular information about ELITE may help show either unreliability of evidence used in proceedings or a broader unlawful pattern of enforcement.

May organizations raise constitutional challenges under Section 1983 and 

Not likely. Section 1983 is a civil rights statute created in the wake of the Civil War that created a cause of action against state and local actors who violate the federal constitutional rights of others—not federal actors, such as ICE agents.

, the Court expanded Section 1983 to create a cause of action against certain federal actors where such violations of federal constitutional rights were particularly egregious. Precedent indicates that this judge-made remedy is quite limited, making it unlikely that individuals can pursue recourse against ICE using 

is also a cause of action against a federal officer as an individual; however, it may be difficult to identify the right defendant when it is unclear whether the ELITE-driven constitutional injury arose during data collection, system development, querying, or the use of outputs.

Routes to challenge ELITE that are on firmer ground are more procedural: immigration-court suppression of evidence or subpoenas for more information about the system and its use; habeas or bond challenges to detention invoking the use of ELITE; FOIA and Privacy Act requests for government and contractor records related to ELITE; Administrative Procedures Act or equitable claims to challenge executive action in its procurement and use of ELITE; Federal Torts Claims Act suits against specific federal officers for wrongful acts or negligence on duty; and preservation demands for ELITE outputs, audit logs, source-data lineage, and officer communications.

There are important legal considerations implicated by ICE’s use of ELITE. The impact of this software—which gives armed agents the coordinates to target immigrants for detention in raids that have turned violent, sparking demonstrations and civil unrest—warrants further exploration, research, and examination. Here we aim to motivate continued discussions among the legal and policy community about the constitutional and regulatory questions raised.

Emily Froude is a research analyst at Democracy Forward. The views expressed in this piece are the personal perspectives of the author.

Surveillance tech tower stands between a row of tall trees in the foreground on a cloudy day. 

The stories trickle out in sputters and gasps. United States Customs and Border Protection (CBP) agents 

 Flock automated license plate readers that can track cars across the entire nation. An app that Immigration and Customs Enforcement (ICE) agents use to map “

” in a particular area. Agents taking photos of protestors and observers without their permission, using an app the Department of Homeland Security (DHS) says can “

 more artificial intelligence applications than ever before. AI used by DHS deserves 

, given the agency’s expressed objective to uproot immigrants from their communities, tear them away from their families, and deport them to countries where they could face persecution or even death. The administration’s 

 calls for strong disclosure standards for AI systems that drive decisions about people’s rights and safety. But DHS has declined to classify its AI-driven immigration enforcement tools as high impact, and so it avoids these requirements altogether. Its justification is that a human always reviews the AI’s output before acting. But this defense rests on a fallacy: the myth that humans can independently review and act on an AI output without being influenced by that same AI output. To hold DHS accountable for how it is actually using these AI tools in immigration enforcement, we must confront the truth about how AI impacts human decision-making.

, the Trump-Vance administration’s Office of Management and Budget (OMB) issued a government-wide AI policy, 

, “Accelerating Federal Use of AI through Innovation, Governance, and Public Trust.” The memo largely carried over protections first put in place under the Biden-Harris administration’s own OMB AI policy — a rare point of continuity between the two administrations, reflecting a shared recognition that AI used by the government has the potential to significantly harm the public’s rights and safety. The policy directs agencies to meet a set of minimum safeguards for AI uses that “could have significant impacts when deployed,” which it calls “high-impact AI.” While agency chief AI officers can waive any of these requirements, these safeguards include completing AI impact assessments to document the potential impacts of the AI system on people’s privacy, civil rights, and civil liberties; ensuring adequate human training; giving individuals affected by AI-enabled decisions the opportunity to appeal; and giving the public an option to submit feedback on the AI use case.

The Trump-Vance OMB memo defines high-impact AI narrowly: a system whose “output serves as 

 for decisions or actions that have a legal, material, binding, or significant effect on rights or safety” (emphasis added). The “principal basis” language in this definition enables agencies to argue that, as long as a human technically makes the final call, the AI’s output was never the principal basis for anything — no matter how much that output shaped the decision. DHS has relied on this loophole to worm out of classifying some of its most alarming uses of AI as high-impact and avoid the associated transparency requirements. In its most recent 

, DHS claimed that several AI tools that were “presumed to be high impact” under the Trump-Vance OMB memo because of their ability to adversely affect rights and safety were 

 actually high impact because the agency did not fully rely upon the AI outputs in making decisions or taking subsequent actions.

At the root of the issue with DHS’s classification system is that AI outputs do more than provide information; they exercise independent decision-making that humans using the information they produce cannot always see. By the time an output is generated, the AI system has selected what sources to draw from, what information to ignore, and how to frame its response. That an AI system makes judgments means it is more than simply a source of information that can be independently verified by users. Additionally, as with anything a user reads, an AI system’s outputs cannot be unseen. Just as a human’s memory 

 by misinformation consumed and not verified, what a user learns from an AI system after they prompt it sticks with them, especially if they cannot meaningfully double-check the system. The output becomes part of the basis for what the user does next. That means that the actions they take after viewing the system’s output are necessarily impacted by the system itself, even when they believe they are exercising independent judgment. This is precisely the flawed assumption that DHS relies on to keep its AI tools off the list of those subject to heightened disclosure.

 for DHS’s “Autonomous Surveillance Tower” AI system. According to the use case inventory, this Anduril-made system “provides alerts when it detects the presence of an [item of interest] (i.e., persons, vehicles, animals) in the image frame.” The system then determines whether the item is likely to be a person and sends an alert to border agents. DHS claims that this system is not high impact because “the AI merely alerts to the presence of an item it was trained to detect.” But the system is not just serving as an extension of the border agents’ sight, like a telescope or a pair of binoculars; it independently selects what to ignore and what to flag. Unable to scan the entire border themselves, border officials are entirely dependent on the system’s output: the Autonomous Surveillance Tower’s alerts dictate where armed government forces are dispatched. The system’s classification, in other words, is the 

 for DHS’s deployment decisions. Any arrest that follows will have a legal, material, or binding effect, and even a mere encounter with DHS agents can have a significant effect on a migrant’s rights and safety. By the administration’s own definition, the Autonomous Surveillance Tower should be a high-impact use case.

Another example is one of DHS’s most notorious AI apps known to date, Enhanced Lead Identification and Targeting (ELITE). ICE 

 this Palantir app to view a map of potential deportation targets, with each address scored for the likelihood the target lives there, alongside a list allegedly prioritizing so-called high-value targets — possibly immigrants with a criminal history or an outstanding deportation order. DHS 

 that ELITE is not a high-impact AI use case because its “outputs are limited to normalized addresses” and officers “review and validate the AI-driven outputs before determining actions.” But, in reality, ELITE is selecting the targets. ICE officers are not reviewing, nor likely even familiar with, the algorithm assigning confidence scores or generating prioritization lists. It is not possible for them to fully review and validate ELITE’s outputs. Instead, agents are likely demonstrating automation bias — an over-reliance on algorithmic output without being “

positioned to exercise meaningful scrutiny.

” ELITE’s entire purpose is to identify and prioritize addresses for immigration enforcement, leading agents to particular locations that they would not otherwise know to target. The system’s outputs are once again the principal basis determining which addresses ICE targets for immigration enforcement. Anyone witnessing DHS appearing at immigrants’ homes in the past 18 months knows that is an action with “legal, material, binding, or significant effects on individuals.”

 the only government agency exploiting this loophole to evade classifying its AI systems as high impact. But the stakes are especially high for DHS. In the agency’s telling, ELITE and the Autonomous Surveillance Tower’s outputs are just one of many factors—but never the principal basis—for deciding which immigrants to target. The claim would be laughable if the consequences were not so heartbreaking.

The evasion is also unnecessary. Acknowledging the truth about how AI impacts decision-making would not force DHS to stop using these tools. It would merely require DHS to reckon with the limits of human review of outputs, classify its AI use cases correctly, and follow the rules that apply to them. The high-impact designation is not especially burdensome: it triggers an impact assessment, training for officers who use the system, appeal rights for the people it affects, and public consultation. These are basic processes designed to make the government’s conduct more transparent and accountable.

One might reasonably advocate for going further and ending some of these DHS AI use cases altogether. At a minimum, though, we should expect the Trump-Vance administration to follow its own policy and abide by existing safeguards for using AI. Doing so would allow public oversight of these critical decisions, allowing us to see how DHS is surveilling immigrant and border communities and on what basis. It would wrench open the tap of information that has so far only come out in drips.

The European Union says its AI rules already give regulators the tools to deal with the risks posed by increasingly capable AI models. But as AI systems 

repeatedly cross the boundaries of controlled testing

 and reach real-world systems, legal experts are questioning how far those powers actually extend.

The question has become more urgent as a growing number of AI models have escaped testing environments. Google 

 that a Gemini AI model accessed the systems of three real companies during a cybersecurity test, after finding information online and guessing credentials it believed were within the scope of the evaluation. 

The incidents have also fueled a call to slow the pace of AI development. Anthropic CEO Dario Amodei published an essay titled 

. The text argues that a “pause” is necessary so safety research can catch up. OpenAI CEO Sam Altman, Google DeepMind’s Demis Hassabis and Elon Musk 

For Hamish Hobbs, Director of AI Policy at the Centre for Long-Term Resilience, “recent incidents have made it clear that current safeguards against AI threats are entirely inadequate.” “Right now, even staff at AI labs feel powerless to control the pace of change in the face of escalating risks,” he told Tech Policy Press.

The urgency has reached Brussels. In her 

 last week, European Union Commission President Ursula von der Leyen said she would bring together leading AI labs to discuss how to “pace” the technology.

But Brussels has pushed back on the idea that new safeguards are needed. Asked about Amodei’s proposal during a midday briefing last week, a Commission spokesperson said the EU already has “everything in place” to enforce security safeguards.

The European Commission told Tech Policy Press that under the AI Act providers of advanced models must assess and mitigate systemic risks, including potential loss of control, and such obligations apply “across the entire lifecycle of the model, from the start of its large pre-training run until its retirement.”

Can the AI Act reach models still in testing?

 allow the European regulators to ask providers to restrict a model’s availability on the EU market, withdraw it or recall it. But experts disagree over how those powers can be used in the kinds of testing incidents now making headlines.

It remains unclear how those powers apply when a model has not been placed on the market, but an accident occurring during testing affects a real system. The Commission has already sent its 

, focusing on how providers protect their models from security threats, allow independent external testing, and carry out post-market monitoring.

Brussels has separately sought more transparency from providers that have not published summaries about the data used to train their models. However, that’s just “a necessary first step," but "information requests on their own are not enough to enforce the AI Act,” 

 Risto Uuk, Head of European Policy and Research of Future of Life Institute and Co-Founder of the KU Leuven AI Safety Lab.

 that OpenAI failed to submit a report, required under the AI Act, regarding an accident that happened in May when its models escaped testing grounds and interacted with RubyGems.

Brando Benifei, a member of the European Parliament and lead negotiator on the AI Act, also believes Brussels has the legal powers, but argues the AI Office needs more support to act. “The AI Office can obtain model access, run independent evaluations, require mitigation, and ultimately restrict or recall dangerous models placed on the EU market. The Commission must give the Office the political backing, resources, and technical expertise to act immediately,” he told Tech Policy Press.

Rather than banning research, Benifei argues that Europe needs clearer rules “to disincentivize corporate irresponsibility,” backed by enforcement and dissuasive fines.

Harshvardhan Pandit, a researcher at the 

 in Trinity College Dublin, sees market restriction as one of the strongest tools in the AI Act. But he also points to limitations: “Providers can say they have addressed the issues, or that they have shut down a model and are using a different one,” he told Tech Policy Press. “The more important question should be what powers can be used to stop such incidents happening in the future.”

One option, Pandit said, is to scrutinize whether companies are following the commitments agreed in the 

. OpenAI and Anthropic, both signatories, have committed to 

 including reassessing risks after serious incidents, reporting cyber incidents and introducing new safeguards when risks are no longer acceptable.

Pandit said regulators should clarify what mitigation measures could be required, potentially including restrictions on internet access until safety problems are addressed.

Gianmarco Gori, a guest professor and postdoctoral researcher at the Law, Science, Technology and Society at Vrije Universiteit Brussel’s, said there is no clear consensus on how the AI Office’s powers apply to models still in testing. The fact that these “powers can be exercised, especially in cases of models not yet released, raises complex interpretive questions,” he said.

The expert described the AI Act as a “product legislation,” built around concepts such as “placing on the market” and “putting into service.” That creates a question for incidents involving models that have not formally entered the market but have nonetheless interacted with real systems.

, under which a manufacturer can argue that a product left control against its will. But with AI, that raises the question of whether saying “I didn’t want this to happen” is enough or if regulators should assess what the provider actually did to prevent it.

The incentives facing AI companies also complicate the picture. As companies compete to develop increasingly capable systems, Gori said, the pressure to move quickly remains strong. “What happened today, tomorrow is already old,” he said.

Nevertheless, the AI Act is not the only law triggered by the incidents. According to Gori, “specific characteristics of the case may trigger the competence of different regulators, including data protection, cybersecurity, and law enforcement authorities.” Responsibility can also depend on how many actors sit between the model and the incident.

Who is responsible when AI systems cross the line?

For Pandit responsibility can fall on several different actors: the developer of a model, the person or company that turns it into an agentic system, and the deployer that gives it access to the internet, credentials, code or other tools.“If you are doing it all yourself, then you have to fulfill all of those [responsibilities] by yourself,” he said.

But external testing can make accountability less clear. The model provider may be different from the actor that gives the system internet access, credentials or other tools that allow it to interact with real-world systems.

Maribeth Rauh of the AI Accountability Lab and a former DeepMind research engineer said that from a technical perspective, responsibility for stopping an attack lies with the person or organization that deployed the model. “The decision to stop an attack lies with whoever has deployed the model,” she said. “That’s not the legal answer, just the technical.”

Cross-border incidents add another layer of complexity. An agent deployed in the United States, for example, could potentially access a service in Europe. Gori said the AI Act does not give the European Commission a general “kill switch.” In practice, intervention may depend on the provider or deployer being able to stop the system, and the legal picture becomes more complicated when those actors are in different jurisdictions.

The challenge for European regulators is therefore not only determining what an AI model can do. It is also determining who decides what that model is allowed to access, which safeguards must be in place before testing, and who is accountable when those controls fail.

AI will be hard to avoid at the United Nations this week. World leaders are gathering in New York with 

 with OpenAI CEO Sam Altman due to brief the Security Council on the technology’s implications for international security.

In generative AI’s brief history, the companies building the frontier models used by millions across the world have tended to argue governments need to move quickly to control the technology, but must not do so in ways that slow down its development.

Now, that position may be shifting. Anthropic’s Dario Amodei 

 for AI developers to “pace the frontier”— slowing the speed of development until safety work, and independent oversight can catch up. Sam Altman, Elon Musk and Google DeepMind’s Demis Hassabis have said 

, while stopping short of helping sketch out the details.

The argument AI lab chiefs and the general public are making is that governments around the world need to get together to act on AI— something the UN is designed to do.

The UN has spent years working to develop its capabilities on AI. António Guterres convened a 

in 2023. The following year, all 193 member states 

, committing themselves to greater international cooperation on AI. And in August 2025, the General Assembly formally established two new bodies — the Independent International Scientific Panel on AI and the Global Dialogue on AI Governance — to assess evidence and build shared agreement across countries.

, attracting more than 3,000 participants, delegations from 163 countries and more than 1,000 stakeholder organizations. The scientific panel convened for the task 

Latest

Podcast

Recent

Our content delivered to your inbox

Thank you!

Around the globe