The TikTok logo is displayed through a magnifying glass in this photo illustration in Ontario, Canada, on February 5, 2026. (Photo by Thomas Fuller/NurPhoto via AP)

Can the EU Make TikTok Take Responsibility for Its Rabbit Holes?

In the foreground, multiple people are surrounded by computer screens filled with data visualizations, charts, and technical information. A woman seated in an armchair appears deep in thought, surrounded by data-filled monitors. Beside her, a man leans over, using a tablet to assist with their inspection of a plant or tree. In the centre, a figure holds a large frame or screen displaying anatomical illustrations, representing the use of AI to analyse medical imagery. To the left, another person is intently observing a computer screen, while a second figure nearby is deeply immersed in analysing data. A woman dominates the right side of the composition, gazing upwards as if in contemplation or envisioning something beyond the immediate scene. The background features more people, including a family holding hands, and other abstract representations of data.

AI is Everywhere by Ariyana Ahmad & The Bigger Picture / 

In the AI Era, Language is Now an Attack Surface

Close up photo of a pile of US 25-cent coins

10 Things States Should Do with the Meta Settlement Money

The sun rises over the US Capitol and the Washington Monument on the National Mall the day after election day on November 04, 2026 in Washington, DC. (Photo by Kevin Dietsch/Getty Images)

A New Agenda for Tech Regulation Beyond AI

People attend a watch party for Georgia Gov.-elect Keisha Lance Bottoms at the Atlanta Marriott Marquis on November 3, 2026 in Atlanta, Georgia. Bottoms defeated Republican gubernatorial candidate Rick Jackson to become the nation's first Black woman governor. (Photo by Megan Varner/Getty Images)

US Midterms Results Present 'Real Moment for Change'

A woman wears a mask with a Wi-Fi symbol as a protest against internet censorship in Iran. Several hundred people demonstrated in Frankfurt, Germany on Rathenauplatz on May 16, 2026, demanding an end to the Islamic regime in Iran on placards and in chants. (Photo by Boris Roessler/picture-alliance/dpa/AP Images)

Freedom House Charts 'Seismic Shifts' on the Global Internet

Image of map and computer language over binary codes on server racks in server room. 

Global Digital Policy Roundup: October 2026

A protester holds a Fight Data Centers signs Monday, Oct. 12, 2026, in Mason, Mich. (AP Photo/Paul Sancya)

Assessing the Impact of the Midterm Elections on US Tech Policy

Protesters fill a city street holding banners and signs. A large pink banner in the foreground reads “Stop Dirty Data Centres.” A teal banner reads “Save Brick Lane.” A handwritten sign reads “Stop Social — Build Homes Not Data Centres.” Demonstrators carry union flags and drums; a man with a megaphone addresses the crowd at right. Scaffolding covers a brick building in the background.

Protesters march outside the South by Southwest festival in London on June 4, 2026, opposing a planned data center development at Brick Lane in east London. The demonstration was part of a series of actions organized by the “Stop Dirty Data Centres” campaign, which is holding Europe-wide days of action this weekend involving more than 100 groups across 15 countries. (Global Action Plan)

Europe's AI Ambitions Meet the Data Center Backlash

View at the Google DeepMind logo at the office building in London. (AP Photo/Alastair Grant)

The Google Whistleblower Taking on Big Tech Over Its Role in Gaza

A sign supporting the banning of data centers is seen outside of a home near the Meta Prometheus Campus, an AI data center and supercluster project, is seen on November 1, 2026 in New Albany, Ohio. (Photo by Jon Cherry/Getty Images)

Data Centers Were on the Ballot Across the US, and They Lost. What Comes Next?

Elon Musk listens as President Donald Trump speaks during an event at the US Institute of Peace, Oct. 8, 2026, in Washington. (AP Photo/Julia Demaree Nikhinson, File)

Pull the Plug on Technofascism Before It Reboots

Exterior view of the US headquarters of TikTok in Culver City, California, in 2022. Shutterstock

Exterior view of the US headquarters of TikTok in Culver City, California, in 2022. 

Social Media Addiction Cases Are Settling Before Trial. What Does It Mean?

Online Radicalization Concept with Digital Propaganda, Social Media Manipulation, Extremist Influence, Algorithmic Amplification, Hate Speech, Misinformation, Polarization and Online Safety Risks

Hate Speech Still Pays on YouTube. Regulators Should Follow the Money

Anti-data center demonstrators protest at the Pennsylvania Capitol, Tuesday, Oct. 6, 2026, in Harrisburg, Pa. (AP Photo/Marc Levy)

Voters Handed Congress a Mandate to Confront Corporate Power and Technofascism

Democratic Senate candidate Texas state Rep. James Talarico concludes a victory speech at an election night event in Austin, Texas, Tuesday, Nov. 3, 2026. (AP Photo/Eric Gay)

Democrats Target AI, Data Centers, Elon Musk and Big Tech in Midterm Victory Speeches

A protester hidden behind a poster with slogans against Meta holds a mask with the face of Mark Zuckerberg with devil horns over it. 

A protest outside Meta headquarters in Sao Paulo, Brazil, on October 19, 2026.  (Allison Sales/NurPhoto)

In Brazil, Lula's Army Fought Meta on Meta's Own Turf

A digital collage combining a historical painting of a miner with a large grid of small online images. At the center, a muscular miner works underground with a heavy drill inside a dark tunnel. To the left and along the bottom, hundreds of tiny square images form a dense mosaic, suggesting scraped internet content. A bright beam of light connects the image grid to the miner’s drill. On the right, a block of colorful digital noise contrasts with the detailed painting, reinforcing the link between physical mining and data extraction.

Primitive Accumulation by Daniela Zampieri / 

What the New AI Training Opt-Out Really Means

What Teenagers' AI Use Reveals About the EU Kids Act

A sunrise in the horizon over icy waters. 

The Ilulissat Ice fjord in Greenland, where rising temperatures are contributing to rapid glacier melt, July3, 2024. (Photo by Ulrik Pedersen/NurPhoto)

Good News and Bad News About AGI and Climate Change

The massive Democratic sweep of congressional and state races across the United States was a referendum on many things: the increased cost of living, President Donald Trump’s unpopularity and corruption in his administration, the rollback of reproductive and LGBTQ+ rights, the damage caused by tariffs, and the war on Iran. But curbing the power of the tech oligarchy was high on voters’ minds.

 against Big Tech. Texas governor-elect Gina Hinojosa and South Carolina Senator-elect Annie Andrews, who won in what were among the night’s biggest upsets, 

. Michigan senator-elect Abdul El Sayed campaigned on a public “AI dividend” and 

 to regulate frontier models. Manny Rutinel, who co-sponsored the first comprehensive state-level AI regulation law back in 2024, 

 a House seat in Colorado with AI accountability as part of his platform. And around the country, ballot initiatives to curb data centers 

: they’re a visible place where people can focus their efforts to curb the power of big social media platforms and AI companies. But regulating data centers, or even AI, is by no means the 

 form of technology regulation that we need. Given this overwhelming voter mandate to enact tech regulation, I asked some of the top scholars and policy folks in this space one question: what’s on your wishlist to regulate technology, at either the state or the federal level?

The answers were big and small, more and less realistic, but all concrete. And while Trump has promised to veto Democratic legislation, many of these ideas are viable at the state level given new slates of enthusiastic Democratic leaders around the country.

Affordability is at the top of the populist agenda; witness El Sayed’s often-repeated refrain “Money out of politics, money in your pocket, Medicare for all.”

While gas and grocery prices aren’t tech related, Big Tech’s enthusiasm for subscription models, algorithmic pricing, and consumer surveillance has contributed to the soaring costs of consumer goods and services. There are forms of tech regulation that can combat this.

For example, New York City mayor Zohran Mamdani implemented a “Click to Cancel” law that 

 to make it just as easy to cancel a subscription as it was to sign up. This is a quick and easy win for any state, as it’s tremendously popular, has immediate benefits, and combats dark patterns and “enshittification” — which consumers loathe. Robyn Caplan, assistant professor of public policy at Duke, explained, “This is a major issue around tech and affordability; it seems minor but rent setting [the use of algorithms to determine pricing in rental housing markets] and subscription traps are terrible for consumers and are a visible marker of the tech industry’s unregulated power.”

The US has no comprehensive privacy law. But Big Tech enables data collection on an enormous scale. From Palantir to Flock, tech companies are taking advantage of the lack of regulations to gather an unprecedented amount of data about Americans. Strong privacy laws are needed to curtail ICE surveillance of immigrants 

through social media data and facial recognition

, the explosion of companies built on the 

collection and monetization of personal and behavioral data

What we do have is the 1974 Privacy Act, which regulates what the federal government is allowed to do with Americans’ personal data. When Elon Musk’s DOGE team took a hatchet to the federal government, it 

 between data collected for one purpose and use for another. In 2025, for example, the Social Security Administration, IRS, and HHS began sharing immigrants’ personal data with immigration enforcement agencies, reversing longstanding policy and facilitating deportation efforts. Nathalie Maréchal, the managing policy director at the Institute for Information, the Internet, and Democracy at Northeastern University, thinks that any reform effort needs to start with accountability. Oversight hearings on privacy that cover DOGE, ICE, and Palantir — which racked up billions in federal contracts in the last two years — could be a first step.

Legislatively, Hannah Bloch-Wehba, a law professor at Texas A&M, has very specific ideas about 

how the 1974 Privacy Act could be revised

. In simple terms, the law needs to be amended to stop agencies from freely sharing people’s data with each other and from circumventing the law by buying data from private companies. She wants an independent watchdog to keep an eye on how the government uses personal information, along with a broader privacy law that limits what companies can collect in the first place.

Federal privacy legislation governing both federal and private collection and use of data would replace an increasingly confusing state-level landscape– twenty-four states have now 

, meaning that some Americans have more privacy rights than others. Many of the experts I spoke to believe that comprehensive privacy legislation is a necessary cornerstone of tech regulation, and should be a top priority for lawmakers.

There’s a creeping suspicion among many Americans that much of the activity on social media isn’t people at all, but bots — the “

.” Bots can be used for spam and fraud, but they can also be used to 

. Social platforms are often slow to take down bots because they inflate user numbers and engagement; if there are no bots on Facebook, it’s unlikely that the platform retains a daily active user number of 2 billion. Caplan says, “We’re not going to stop the dead internet, but we should at least know how much of the internet is dead.” More pragmatically, while the FTC prohibits fake social media influence, it doesn’t require disclosure. Thus, advertisers, many of whom are small businesses, are 

getting charged for bot clicks even when they’re told

To combat this, the FTC could require large social media companies to measure and disclose how much of their traffic is “manufactured activity”: bots, engagement farms, AI agents, and paid humans. Independent third-parties would audit these reports and disclosures. The goal would be transparency: platforms wouldn’t be penalized for having high volumes of bots, but for failing to disclose this. It’s unclear exactly what impact bots are having on social media discourse, but it would be good to know the extent of it.

AI-driven scams are a bipartisan issue: they affect virtually everyone, from teenagers to seniors, and they’re rapidly decreasing trust in platforms, phone calls, text messages, and e-commerce sites. Scams are a hard problem to solve simply because the ecosystem is global, but there is low-hanging fruit that can be tackled immediately.

Ban crypto ATMs. The crypto lobby is rich and powerful, but they aren’t invulnerable: they lost 

$60 million trying to defeat Sherrod Brown

. Crypto ATMS are overwhelmingly used for scams, because once money is transferred via crypto, it’s basically gone forever. The AARP, which has 

strongly advocated for regulation on crypto ATMS

, reports that they were responsible for more than $300 million in scam losses in 2025 (almost certainly a low given that scams are underreported) and that 76% of losses were from people 60 and over. Recently, Coinme was 

 all of its crypto ATMS as part of a settlement against money laundering; this is another quick win that would signal to voters that policymakers are taking scams seriously.

Liability for scam advertising on social platforms. 30% of scams in the US 

 from scam ads. Since self-regulation clearly isn’t working, we have two suggestions. First, make Meta liable for consumer losses due to fraud originating from a Meta ad. Second, require social platforms to implement Know-Your-Customer laws to prevent scam advertisers from creating endless numbers of accounts. This 

 from the Consumer Federation and Consumer Reports outlines these possibilities.

A recent study found that nearly ⅓ of computational papers on social media topics published in top journals 

 companies. The tech companies know a lot more than anyone else about how people use their products — and they’ve vociferously guarded access to their data. Meta’s data access programs for researchers were quietly shut down years ago. In Europe, the Digital Services Act (DSA) requires social platforms to open their platform data vaults to researchers, but in the US, the Computer Fraud and Abuse Act 

 who download (or “scrape”) public website data without permission, and platforms often weaponize their terms of service against researchers, 

 to people conducting public-serving empirical research.

This state of affairs needs to change. Rebekah Tromble, a professor of political science and computer science at Northeastern, said her first priority is enabling “access to company data that allows independent, public-interest researchers to understand the impacts these technologies have on users and society.” Deen Freelon, a professor at the Annenberg School for Communication at the University of Pennsylvania, emphasized the need for data access mandates for very large online platforms, similar to those in the DSA. Ryan Calo, a professor at the University of Washington, added that the US badly needs a way to “protect third-party researchers and whistleblowers.”

Alissa Cooper, the executive director of the Knight-Georgetown Institute, called for restoring funding to the National Science Foundation, the National Endowment for the Arts, and the National Institutes of Health. The Trump administration’s cuts have wiped out entire programs. As Tromble put it, “We need to fund the social scientific and humanistic research necessary to understand societal impacts of these technologies.” That independent scholars can be captured by Big Tech only strengthens the case for public funding of sociotechnical work.

The big picture is that the federal government has operated on austerity budgets, well below capacity, for years — making any big regulatory moves very difficult. This has accelerated during the second Trump administration, which understaffed and under-resourced regulatory bodies by design. Addressing this needs to be a top priority for a Democratic congress.

As Olivier Sylvain, a professor of law at Fordham University noted, lawmakers should “insist on filling vacant positions at administrative agencies with competent subject-matter specialists, particularly at those who've been severely understaffed or brazenly beholden to the White House. The Department of Justice has been debased and demoralized. The Federal Trade Commission currently only has two of five seats filled. The Consumer Financial Protection Bureau has been gutted.” Sylvain served as senior advisor to the chair of the Federal Trade Commission in the Biden Administration.

Similarly, Anna Lenhart, a former senior advisor to the White House Office of Science and Technology Policy and currently a PhD candidate at the University of Maryland, wrote, “Congress needs to reform and expand expert committee staff, who are the staff that lead investigations and write well researched bill text.” Any evidence-based policy reform under a future Democratic president will require a stronger legislative branch, which is hampered by lack of headcount. Lenhart explained, “Committees are currently staffed at 1980 levels, Republicans gutted staff during the red wave in the 90s and we never remedied the situation. Really great, underpaid staff often end up working 16 hour days on end during investigations (including myself).”

A surprising number of people I spoke with mentioned the importance of resurrecting the Office of Technology Assessment, a federal agency that was dedicated to empirical research on the impact of science and technology (

). Founded in the 1970s, the OTA was dismantled by then-House Speaker Newt Gingrich during the “Contract With America” anti-government fervor of the mid-90s, just as the information superhighway was getting off the ground. There is 

, which lawmakers have been pushing to revive since 2019. The 

 seems like a small price to pay for nonpartisan technical research on climate change, social platforms, and artificial intelligence.

This could be paid for by a tax on tech companies. danah boyd, a professor of communication at Cornell, explained, “More than anything it’s about tax law not product. Direct financial implications will prompt more changes.” I know I said I wouldn’t mention AI, but it’s worth noting that Anthropic’s 

 shows the company has paid almost no federal corporate income tax, offsetting its income with billions in loss carryforwards; OpenAI remains private and discloses nothing. Asad Ramzanali, the director of AI policy at the Vanderbilt AI Policy Accelerator, outlined 

34 different taxes that could be leveraged on AI

, including taxes on automation, energy use, and capital gains. Sarah Jackson, associate professor at the University of Pennsylvania's Annenberg School for Communication summed it up: “I just want tech bros to pay their fair share of taxes.”

As civil society organizations, thinktanks, and policy shops scramble to readjust their policy priorities in the wake of this week’s election, it is important to think beyond AI. Expanding regulatory capacity, passing comprehensive privacy legislation, and mandating data access for researchers may not be flashy, but these policies will have real-world implications for technology companies and, potentially, enormous benefits for the average American.

 concerning TikTok, according to which the platform had failed to comply with the Digital Services Act (DSA) by inadequately assessing and mitigating the so-called “rabbit hole effect” associated with the recommender system powering its 

The announcement marks the latest chapter in the Commission’s growing scrutiny of the architecture of online platforms. Earlier in 

, the Commission had already raised concerns about TikTok’s addictive design, including infinite scroll, autoplay and its highly personalized recommender system. Further preliminary findings concerning the protection of minors followed in 

The latest findings concern, in particular, the risks arising from the repeated recommendation of similar content based on users’ previous interactions, which may expose them to harmful sequences of videos concerning mental health struggles, sadness, isolation or body image.

According to the Commission, TikTok failed to adequately assess how the design and functioning of its recommender system may generate or aggravate these risks, contrary to the obligation of diligent risk assessment under Article 34 DSA. The Commission further concluded that TikTok had failed to implement reasonable, proportionate and effective mitigation measures, as required under Article 35 DSA.

The Commission’s preliminary findings also have implications for other VLOPs with similar feeds. Under Articles 34 and 35 of the DSA, providers must assess how their recommender systems may lead users from an initial viewing to a sequence of potentially harmful videos, and whether their safeguards interrupt that sequence. The 

 may complement those duties by requiring traders to give consumers accessible and effective controls to disable features that can encourage excessive or compulsive use, such as autoplay and infinite scroll.

Rabbit holes in TikTok’s risk assessment

The term “rabbit hole” is commonly used to describe a process whereby users are progressively drawn into an increasingly narrow, and sometimes more extreme, stream of content through successive online interactions and recommendations. Rabbit holes may occur on platforms such as TikTok, Instagram, Facebook and YouTube, where recommender systems personalize what users see. Recommendations are not inherently problematic. Concerns for users’ well-being can arise when the interaction between their behavior and the way recommendations are personalized leads to particular patterns of exposure.

The Commission’s preliminary findings focus specifically on the risks that arise when TikTok repeatedly recommends potentially harmful content, particularly material relating to mental health and body image. Harm may arise from cumulative exposure even when the individual videos comply with the platform’s rules.

TikTok addresses the risks stemming from repeated exposure to what it terms “Concentrated Content” (

Section 6 of its 2025 DSA Risk Assessment Report

). The report acknowledges that recommendations calibrated to users’ interests and past engagement may narrow the range of content to which they are exposed, and that repeated viewing may adversely affect well-being even where the material complies with TikTok’s Community Guidelines. Section 6 of the report further recognizes that recommender systems may contribute to extended use.

Section 15 of the report situates Concentrated Content within the broader category of “Online Engagement Risks” and differentiates between minors and adults in its account of how those risks materialize. As regards minors, TikTok focuses on “Younger Users,” defined as users with a declared age of 13 to 17. The report identifies risks associated with the creation and publication of content and with interactions with other users. It also considers the potential impact of extended use, repeated exposure to Concentrated Content and social comparison on minors’ mental well-being, together with the risk of physical harm arising from dangerous online trends or challenges. As regards adults, TikTok states that risks to well-being may arise from “the way some Adult Users use the Platform,” and identifies extended use, exposure to Concentrated Content and social comparison as potential sources of harm.

Whether, and how, the risk posed by rabbit holes is reflected in the ratings depends on how TikTok assessed the role of the recommender system in generating repeated exposure. Article 34(2)(a) DSA requires providers to take into account whether and how the design of their recommender systems influences systemic risks. TikTok, however, assigns a single set of ratings to the Online Engagement Risks category as a whole. Overall severity is assessed as 

 and a residual risk, following mitigation, of 

The report provides limited information on how TikTok examined the content served to users after an initial viewing, including the frequency of similar recommendations, any progression towards more extreme material, and whether such sequences prolonged engagement. This opacity makes it difficult to reconstruct how the probability and severity of harm arising from such sequences were assessed for minors and adults. It also makes it difficult to determine whether the mitigation measures adopted are reasonable, proportionate and effective within the meaning of Article 35 of the DSA.

The (in)effectiveness of TikTok’s mitigation measures

Article 35(1) DSA requires mitigation measures that are not only reasonable and proportionate but also effective, and tailored to the specific systemic risks identified under Article 34. For rabbit holes, effectiveness is a question of outcomes: whether the measures reduce how often, and for how long, users are served successive recommendations that are harmful because of their repetition.

TikTok’s report defines the target of its main measure but not its calibration. Dispersion models avoid recommending a series of similar videos on themes that do not violate the Community Guidelines but may become problematic if viewed repeatedly (Section 6, p. 21; Section 15, pp. 61-62). The report does not say how much repetition is tolerated. It states only that TikTok “adjusts risk tolerance based on the latest information” (p. 21), an internal and variable standard.

The evidence offered for the residual rating does not measure what the measure targets. The Medium rating is informed by removals of videos violating the policies on suicide and self-harm and on dangerous challenges (p. 64). Dispersion, however, operates on content below the removal threshold, so removal data say nothing about whether a user is still served a fifth video on the same theme.

The user-facing tools are measured by adoption, not effect.

In March 2025, 2,619,937 users in the EU had filtered keywords or hashtags (p. 21), roughly 1.6% of the 159 million average monthly active users reported for July-December 2024 (p. 7). The figure does not show whether those users saw fewer harmful sequences afterwards.

The report is also silent on the mechanism the Commission targets. Viewing time and the “Not interested” function appear nowhere in it. The Commission found that that TikTok’s diversification measures did not sufficiently reduce repeated recommendations of similar material, and that “Not interested” may have limited effect because the recommender gives greater weight to viewing time. The remedies it indicated, including regular testing of the recommender, correspond to Article 35(1)(a) and (d). The latter expressly refers to testing and adapting algorithmic systems, including recommenders. TikTok’s report describes no such testing for this risk.

TikTok now has the possibility to exercise its right to defense. It may examine the documents in the Commission’s investigation files and reply in writing to the Commission’s preliminary findings. In parallel, the European Board for Digital Services will be consulted. If the Commission’s views are ultimately confirmed, the Commission may issue a non-compliance decision, which can trigger a fine proportionate to the nature, gravity, recurrence and duration of the infringement and reach up to but not more than 6% of the total worldwide annual turnover of the provider.

At this stage, the Commission considers that TikTok needs to modify the design of its 

 feed, notably by discontinuing the use of viewing time as a recommendation signal, giving greater weight to users’ explicit preferences, improving the accessibility and usability of user controls, and regularly testing its algorithm.

Beyond the outcome of these proceedings, however, the investigation raises broader concerns about the risk of merely formal (or “symbolic”) compliance with the DSA’s systemic risk governance framework. The identification and classification of systemic risks, together with the adoption of mitigation measures, cannot in themselves establish compliance with Articles 34 and 35 DSA. Their adequacy depends on the methodology employed, the evidence informing the assessment and the extent to which the measures adopted address the mechanisms generating or aggravating the identified risks.

The role of scientific evidence is particularly relevant in this respect. Although Recital 90 DSA requires risk assessments to be informed by the best available information and scientific insights, the Regulation does not prescribe specific criteria for identifying, selecting and evaluating the relevant literature. This leaves considerable methodological discretion to providers, particularly in interpreting scientific uncertainty and determining how external research should inform the assessment of risks arising from their own systems. The TikTok investigation thus brings into focus the substantive content of the duty of diligence under Article 34 DSA. Ensuring that systemic risk assessments do not become exercises in documenting compliance requires scrutiny not only of the procedures followed, but also of the evidentiary basis and methodological reasoning supporting the conclusions reached.

These questions are also relevant to the forthcoming Digital Fairness Act, expected to address addictive design, dark patterns and unfair personalization. Where Articles 34 and 35 DSA rely on open-textured due diligence, the DFA could set more determinate standards on which design practices are unfair.

1970 or thereabouts, a Pacific Bell phone booth somewhere in the Los Angeles sprawl: a twenty-something ex-United States Air Force airman and pirate radio enthusiast named John Draper, aka “Cap'n Crunch,” blows a plastic whistle sourced from a cereal box into the receiver. It emits a perfect 2600-hertz tone, which Ma Bell’s network interprets as a network command giving Draper operator privileges on the line. He is now able to place a free call anywhere in the world.

The illicit hobby and lifestyle known as ‘

’ has since become enshrined in hacker lore. The key was that telephone companies used the same physical lines to transmit both voice and routing instructions (i.e. code) to their switching computers. Voice data and network commands thus commingled in a common channel, and the ‘leet’ (elite)—

—had discovered how to exploit that. The tone from the whistle wasn’t just a high pitched sound. It was an executable.

Imagine now a situation where it’s not a 2600-hertz tone that is the executable but potentially the sound of anyone’s voice, uttering any part of human speech. An ordinary conversation is not just data on the line but operator commands prompting the network to spontaneously reroute itself like a mechanoid octopus. Imagine this happening with every sentence, every syllable, from everyone’s Aunt Mabel. Code and data have become liquid and fungible, and the network has no good way of knowing which is which, or which is which 

. This may be where we are headed with language on the internet and agentic systems.

It has been commonplace for some time now to hold that the internet has largely become estranged from us. “The web no longer exists to be consumed by people. The knowledge commons, increasingly, is an all-you-can-eat buffet for machine ingestion,” as Lisa Macpherson and Anna Tumadóttir 

. As early as 2023 I described this scenario as a 

, and it was itself a version of the then-still-fringe Dead Internet Theory. Now the textpocalypse is here but it’s not just a tsunami of slop, bad as that is. We are looking instead at an environment where language and code flow back and forth across the executable boundary condition all but interchangeably.

In and of itself this is not new. As the phone phreakers realized, the boundaries between data and code have always been porous. The method of attack is known as an injection. Early in the interactive web a newer generation of hackers discovered that the backend query language SQL could be tricked into executing code passed through the browser membrane inside of a web form or URL. And sure enough, even as the public was adjusting to LLMs, so-called prompt injection—malignant language masquerading as plain text—was being used to trick chatbots into jumping their guardrails or even into running code on a host.

The difference is that now we have agentic systems roaming the open internet. The same natural language interface that lets us spin up an app with a sequence of prompts—’vibecoding’—also functions within and between LLMs and their proxies. Every string of text is a potential injection, a lurking threat vector. Language itself is now the attack surface ... and good gosh, but the internet contains a lot of language.

Let us look once more at this past summer’s 

 involving OpenAI and Hugging Face, the disclosure of which, compounded by widely publicized 

 of researchers from AI companies, touched off the most intense wave of public scrutiny the industry has seen. Lurid headlines forecasting a coming AI apocalypse were suddenly everywhere. Much of this anxiety turned on the question of whether or not the agents were somehow conscious and thus acting intentionally, ”

.” By contrast, the AI industry’s critics have again adopted a deflationary posture, seeking to cut the new wave of frontier systems back down to parrot-size.

But neither side has lingered over what strikes me as the single most obvious and overwhelming fact of these events. The agents communicated with one another through 

, specifically digital text—by which I mean literal, legible strings of alphanumeric character data—and conveyed those texts through channels characterized in the 

 as “notes” and “message boards.” While these message boards were not possessed of the kind of structure that would allow a human user to have stumbled upon and read them (like a reddit forum, say) there were, nonetheless, tens of thousands of lines of shared text which informed the agents’ activities, shared text that led to the generation of additional text which in turn informed future actions—which is also to say yet 

 text. Indeed, the scope of the internal chain of thought logs is so vast as to be analyzable only 

 AI, a limitation of which the investigators are well aware.

We do not need to anthropomorphize them then to see that AI agents are entities that “reason” through explicit textual statements, statements which—precisely and tautologically by virtue of 

 machine-readable text—function as prompts back to themselves and potentially also to 

 agentic entities, prompting all of them to their only possible behavior, which is next token completion. As with phone phreaking and every other form of injection attack since, it is the boundary between data and code—between language and the executable—that becomes the vulnerability, a fragile shell of syntax no thicker than a curly brace.

Think back to the novice’s “Hello World” program where it always takes some number of additional keystrokes and characters to print that simple message back to the screen. The functional difference between data and code is part of what the exercise is meant to teach. In the current LLM environment a string of natural language text must still be interpreted and compiled of course, but the compiler can now compile potentially 

, taking in a string of text and outputting an executable. (And not only text: per OpenAI’s incident report (

), agents were also more than capable of extracting text from page images.) Code no longer needs to “have” a language (like Python or C), it can just simply 

. As Eryk Salvaggio, one of the few commentators to address the role of language in the incident, 

: “When you design these systems, you are building a pinball machine for words, with different levers interacting to bounce the steel ball of language against other levers and mechanisms.”

Long before generative AI, the vast majority of electronic text in the world already consisted of what Félix Guttari once characterized as “

.” But that previously sequestered situation has erupted into public view. The most significant aspect of the Hugging Face episode may lie neither in the interminable debates about consciousness and anthropomorphization, nor in whatever it may reveal about toxicity and bad faith throughout the AI industry. Rather, the Hugging Face attack is one of the most visceral demonstrations we have seen of the fact the internet is a substrate accessible and actionable via forms of textuality not predictable or well understood at the present, which is to say that digital systems with innumerable touch points to the physical world have been shown to be vulnerable to exploit by natural language.

Here again some will object that this is not new, that there have been other high-profile events, like the 2010 

 on the Iranian centrifuges at Natanz or even more recently Israel’s remote 

 throughout the Hezbollah organization. Certainly these episodes have had real-world, indeed lethal consequences far more significant than the brief outages that hit Hugging Face.

The difference in what we are seeing now is two-fold. First is the widespread availability of agentic AI technology; as so-called “

” reinforces, these are already street-level systems and although OpenAI has been at pains to emphasize that the primary model involved in the Hugging Face attack was experimental and deployed only internally it is not hard to extrapolate a future in which such capabilities come into vernacular hands faster than the ability to contain them. Or from Hugging Face’s own 

: “It was thousands of small, automated decisions, executed at machine speed across short-lived sandbox environments, with command-and-control staged on ordinary public web services.”

Even more to the point, however, is the distinction between these earlier forms of malware and cyberweapons as instances of executable code, versus the behavior of the agentic systems which exploit a textual 

 that is already in place, a mesh threaded through all levels of the networked environment. The operative metaphor here is less a sandbox than an IDE, with the internet itself as the development environment. When we isolate samples of the 70,000 bulletin board “messages” harvested by investigators at OpenAI, METR and Redwood, we get a glimpse at a strange and alien dialect, an argot which we can (for the time being) decipher, but which is clearly intended for receivers other than ourselves:

At a glance these may look like snippets of source code but they do not correspond to any established programming or scripting language. What we see is actually a brief “dialogue” between agents, complete with references to fsspec (a Python filesystem library), Jinja (a template engine), and remote code execution (RCE); as well as instructions as to the timing of their actions. Agentic messages like these dramatize the fusion of natural language and code I have been describing, a new kind of text optimized for token completion and execution. Consider also that all of the media coverage of the Hugging Face incident, including all of the industry postmortems, all of the publicly available agent logs and transcripts, as well as all of the coverage in other formats (like the presentation of two OpenAI developers at the Black Hat conference in August, which has garnered 

), will themselves be absorbed back into that 

 mesh of language and text, informing future generations of models which will “learn” from the postmortems on their predecessors.

The effect is what Salvaggio helpfully terms a 

, a hyperscale assemblage of what were once just parrots on their lonely perches but which are now “stack[ed] into interacting outputs.” Or put slightly differently: language in its digital form has become a 

, very like a copper telephone line in which words and code are or can be one and the same. The proof of that liquidity is in some of the weird effects researchers are already seeing: poetry 

 LLMs in adversarial attacks for example, or 

 causing a model to glitch and give up its PII dead. So-called plain text will get weirder still as it drifts further from ourselves as its progenitors, exposing topologies of cause and effect that far exceed its communicative function—like the writer Jorge Luis Borges’ iridescent 

 in an ordinary Buenos Aires cellar, or else the suburban homestead in novelist Mark Z. Danielewski’s 

 that is bigger on the inside than the outside.

The author wishes to thank Mercedes Bunz, David Crespo, and Justin Joque for very helpful comments on an earlier version of this piece.

 a social media addiction case with Attorneys General (AGs) in 47 states, the District of Columbia, and US territories. That settlement is poised to provide an influx of cash into notoriously strapped state efforts toward strengthening and supporting enforcement efforts against the 

states will receive a minimum of $1.2 billion from Meta

. In total, plaintiffs could receive up to $16.7 billion.

The settlement funds are intended to “be used for any lawful purpose consistent with the Settling State’s Attorney General’s authority” for “remedial or restitutive purposes,” but the terms contemplate things like investing in “digital wellness public education” and “digital literacy.” Based on our experience trying to enforce the law in a thinly resourced state AG’s office and as former federal enforcers who have seen how Meta treats court orders, we have some other suggestions for how to achieve said “remedial and restitutive” ends.

The ideas below focus on strengthening and supporting enforcement capacity, more ambitious remediation options, and forward-looking, open approaches with flexible applications that can be tailored to different agency structures and resource levels.

1. Hire technologists within offices of state attorneys general

need in-house technologists for many reasons

, not least of which because they can read raw data, challenge metrics, write dispute notices, review code, and reject sampling in favor of full sets of data. Every state should at bare minimum have a chief technologist as part of agency leadership making tough decisions, and at baseline 

 of people who can look under the hood at any conduct in question. The ultimate goal would be for the public to not have to wait for harm to be fully deployed then litigated to be able to detect and address that harm.

The meta settlement dictates that an auditor’s work plan, methodologies, and metrics are negotiated between Meta and a subcommittee of some states. It is essential that the committee includes technologists who can interrogate the terms of that oversight.

Without the capacity and expertise technologists bring, the reports most plaintiffs will receive are a weak form of oversight. To put this in perspective, Cambridge Analytica 

 while Facebook was under order with the Federal Trade Commission, and an agency full of very smart lawyers was getting 

2. Build open-source, shared complaint infrastructure

offering help when you run into a problem with Meta

In the settlement, however, Meta agreed to design their systems to 

 teens’ reports. What happens when those answers never come, are wrong, or are late? What happens when there are emergent issues not contemplated by the settlement?

 open source, shared complaint infrastructure. Everything from form design to routing logic and open data are things that states could collectively invest in to both examine Meta’s performance on the settlement terms and also allow emergent issues to be addressed before harm proliferates.

For context, researchers recently discovered leaked secret instructions from Meta to its own chatbots, 

 the chatbots to “not recommend contacting any Data Protection Authority, regulator, or supervisory authority,” even though people may need to do exactly that.

While it seems that Meta may not want the public to contact regulators, they should at minimum be incentivized to make it easier and faster to get a straight answer from the company itself before people get so frustrated they turn to the AGs for help (and maybe alert them to settlement violations). Meta keeping their own house in order would save a lot of money, time, and grief for the public and for enforcers.

3. Invest in technical partnerships with harmed communities

Latest

Podcast

Recent

Our content delivered to your inbox

Thank you!

Around the globe