Online Safety and the “Great Decentralization” – The Perils and Promises of Federated Social Media

Samantha Lai is a senior research analyst at the Carnegie Endowment for International Peace's Technology and International Affairs Program, and Yoel Roth is a Non-Resident Scholar at the Carnegie Endowment for International Peace and the Head of Trust & Safety at Match Group.

Decentralized social media platforms offer the promise of alternative governance structures that empower consumers and rebuild social media on a foundation of trust. However, over two years after Elon Musk’s acquisition of Twitter sparked an exodus of users seeking new homes on the social web, federated platforms remain ill-equipped to meet the threats of abuse, harassment, coordinated manipulation, and spam that have plagued social media for years. Given the porous nature of decentralized services, these limitations will not just affect individual servers, but reverberate through the social web.

Early signals from the threat landscape

These are not unrealized fears from a faraway future, but a creeping reality. Already, signs of inauthentic behavior can be found on federated platforms. Take, for example, pravda.me, an instance of the federated social media platform Mastodon that had over 17,000 active users in 2023. According to the metadata the instance’s operators provide to the Mastodon network, pravda.me is hosted in Saint Petersburg, Russia, home of the disbanded Internet Research Agency (IRA), the infamous “troll factory” responsible for meddling in American elections in 2016. 

Although the content hosted on the server is largely in Russian, it shows signs of at least some attempts at cross-over appeal for American audiences. The server’s header image includes a screenshot of a tweet from Elon Musk in 2018, stating his intent to create a site called Pravda (the Russian word for “truth”), “where the public can rate the core truth of any article & track the credibility score over time of each journalist, editor & publication.” (Musk never followed through on this particular project.) Atop the image, pravda.me’s operators added, triumphantly, “Elon, we did it!”

Pravda.me bears many of the all-too-familiar hallmarks of social media inauthenticity. Back when the instance’s local timeline — a feed of all posts created by users whose accounts reside on the pravda.me server itself — was public in 2023, it had virtually no human-driven activity; the content, overwhelmingly, consists of seemingly automatically posted headlines and links to articles from Russian-language news sites including state-sponsored outlets such as Russia Today (RT) and TASS. A number of accounts exclusively post content such as Bible verses or the Russian-language dictionary. Clusters of apparent sockpuppet accounts amplify the automatically-posted news articles and Bible verses, seemingly to an audience of just themselves. Most of the accounts on pravda.me have fewer than 10 followers; accounts with larger followings seem to have attained their popularity only from their fellow fake accounts. The server’s operators, identified only as “Administrator,” have no posts and hide the list of accounts they follow. As of October 2024, the instance’s local timeline is no longer public, and the number of active users has been hidden. Individual accounts on the server remain active.

Pravda.me is clumsy and, by all appearances, unsuccessful at attaining even the barest hints of an audience. And yet, despite its shoddiness, pravda.me is a troubling warning sign of the risks facing Mastodon, Bluesky, Threads, and other federated and decentralized social media platforms: As new platforms, and new forms of internet governance, emerge where centralized services like X once dominated, they’re often ill-equipped to meet the threats of abuse, harassment, coordinated manipulation, and spam that have plagued social media for years.

A dearth of resources

Over the last two years, we’ve studied the trust and safety capabilities of federated platforms. In an article published in the Journal of Online Trust and Safety, we found that most major decentralized and federated platforms do not have the necessary technical tools for scalable management of harmful content and conduct — or even the enforcement of their own rules. These platforms lack essential capabilities such as media hashing and matching functions for identifying and removing CSAM, or automated heuristics for administrators and moderators to programmatically restrict URLs from spam and phishing sites. Instance administrators and moderators often have to screen and remove content manually, post by post, and are easily overwhelmed by large-scale harassment campaigns and spam.

Federated platforms also face architectural constraints in their ability to address collective security risks, like government-sponsored troll farms meddling in political discussions. On centralized platforms, trust and safety teams have access to unified, platform-wide telemetry about what bad actors are up to, which allows them to study activity across the whole platform to identify sophisticated adversarial threats. On federated platforms, however, administrators and moderators only have access to instance-specific logs, which makes it difficult to detect adversarial threats spread out across multiple instances. While platforms such as Mastodon and Bluesky have publicly-accessible APIs, differences in network structure make standardized and comprehensive data collection challenging. Administrators and moderators also lack the time, resources, and skills to do this difficult, and often quite technical, work.

What once may have seemed like niche issues impacting a handful of Mastodon instances now stand to become a major threat to the social web. In March 2024, Meta’s Threads began what the company called a “scaled approach” to integrating Threads into federated social media. As of June 2024, Meta users from over 100 countries can now join the fediverse, a subset of federated social media services that operate on the ActivityPub protocol. By opting in, they make their profiles available for those on other fediverse servers to search for, follow, and interact with. The 150 million or so people actively using Threads can now distribute their content across the more than 20,000 servers that make up the fediverse — and in turn, Meta’s largely orderly walled garden of Threads will be influenced by content from millions of accounts on instances far beyond the company’s control.

With over 15,000 moderators on staff and more under contract, Meta is perhaps uniquely equipped to wrangle the sprawling challenges of moderating federated social media. Across Mastodon, Bluesky, and other services, we shouldn’t expect others to be so lucky. If these platforms’ moderation capabilities fail to scale alongside their user bases, servers will quickly be overrun by spam, CSAM, and other content that degrades users’ experiences on the platform. We know how this story ends: People will either revert back to better-moderated, centralized options, or, at best, flock to Threads and other commercial entrants that have more capacity to moderate. This recentralization undermines the promise of federated services as a solution to social media’s problems, as it removes the availability of multiple governance options that users can choose from.

Centralized solutions for decentralized spaces

Addressing trust and safety challenges is essential for a viable future of federated social media — and doing so will require sustained and coordinated action, as well as resources to support the critical work of building out trust and safety capabilities for smaller and emerging platforms.

Centralized efforts in decentralized spaces may sound oxymoronic, but it’s a bit more complicated than that. Writing about the tradeoffs of moderation in decentralized spaces, University of Minnesota law professor Alan Rozenshtein drew an analogy to email: “In part due to the investments necessary to counter spam, [email] has become increasingly dominated by Google and Microsoft. If similar scale is necessary to fight spam and bot accounts, this could serve as a centripetal force to counter the Fediverse’s decentralized architecture and lead to a Fediverse that is more centralized than it is today.” Put another way, in a space of total decentralization, we may ultimately find ourselves gravitating back towards centralization anyway. The question then becomes: what centralization?

Moderation, fundamentally and unavoidably, is an expensive and time-consuming proposition. This reality, in turn, limits the practical feasibility of total decentralization, assuming we don’t want to backslide on safety and security. These are difficult tradeoffs to make, but essential ones. To remain viable while still maintaining an essential commitment to decentralization, federated services must embrace some degree of centralized moderation infrastructure that all administrators and moderators can adapt to suit their own needs.

This need not be viewed as antithetical to the premise of federation: The most contentious content moderation issues — like how to deal with sensitive political content or adult sexual material — are ones where decentralization can and should empower communities to set and enforce their own norms. But these issues don’t represent the totality of online trust and safety, and enabling community-level responses to these highly subjective questions shouldn’t have to mean reducing all moderation work to a completely unsupported environment where individuals act in isolation and without the tools and technology they need to be effective. Instead, we should view the task of centralized support for trust and safety as being one of building common components of trust and safety infrastructure, housed beyond the walls of private corporations, that communities can adopt to create their preferred models of social media governance.

This work should start with areas where there’s broad agreement about online harms, and what to do about them, such as the moderation of child sexual abuse material (CSAM) and spam. In these contexts, the moderation decisions of centralized platforms likely would not differ significantly from that of other services. Building on that alignment, stakeholders across federated social media platforms, including commercial entrants, could collaborate on three lines of effort to improve online safety on federated services: they can increase the amount of tooling available for moderation, establish a structured hub for detecting and addressing influence operations, and create infrastructure for administrators and moderators to conduct inter-server communications.

Where to begin

What’s holding back progress in this space? Our research identified three factors: Funding, data, and coordination.

Funding is the most urgent gap. Currently, moderation tooling for federated platforms is developed and maintained by volunteers either at their own cost or through short-term grants. However, both developing and maintaining these tools require significant long-term investments. Commercial entrants can make a significant impact in this space if they adopt a common-interest approach and contribute to tool development efforts. A report on fediverse governance by Erin Kissane and Darius Kazemi found that the moderation efforts of administrators and moderators of medium-sized servers on Mastodon could benefit from relatively straightforward improvements to tooling such as bulk-select and resolve controls for managing spam, visible context on why a certain piece of content or account was reported, and finer-grained account registration controls. Larger, established platforms almost certainly have already built these features, and at essentially no cost or competitive disadvantage could release them as open-source libraries.

In addition to the tools used to moderate, larger platforms could straightforwardly share information about the moderation decisions they’ve already made. While Threads has a publicly-available list of blocked servers, there still is no easy way for other admins and moderators to ingest and adopt that list through tooling. A suite of tools that help federated instance administrators understand, evaluate, and adopt moderation decisions made by other parties could reduce the persistent need for duplicate effort.

Even with sophisticated tooling, administrators and moderators may still struggle with identifying and addressing influence operations or longitudinal threat actors. There is no solution to this problem except coordination: Stakeholders across federated social media must work together to create a structured hub that curates pooled data for shared analysis, funds the storage and compute costs required for such analysis, and provides a core staff of paid analysts to develop systems expertise and promote longitudinal threat awareness in the space.

Finally, the people doing the difficult work of moderation need effective ways to communicate with each other and address network-level threats. Presently, channels for communication between instances are inchoate, and administrators and moderators often have to convene ad hoc on platforms such as Discord to information-share on previously-identified malign actors and discuss moderation decisions. Tool-builders could create structured platforms for moderators to communicate and share information with each other, or for moderators to vet their moderation decisions with users across other servers.

An ambitious vision

Relative to the decades-long development trajectories of centralized services like Facebook and X, federated social media platforms are still in their early stages of growth. Despite their nascency, there are already many who are passionate to ensure that scalable online safety is core to its service. The nonprofit Independent Federated Trust and Safety (IFTAS), Columbia University’s Trust and Safety Tooling Consortium, and individual tool-builders have been working to develop and curate tooling for scalable trust and safety governance. Expanding the availability of open-source tooling, creating infrastructure for longitudinal threat analysis, and allowing inter-server communication are necessary components for the long-term viability of an open and diverse social web. If we want to protect a future for the internet that empowers consumers and rebuilds social media on a foundation of trust, we need to start by supporting these efforts to build shared trust and safety capabilities.