Europe’s Digital Services Act (DSA) aims to revolutionize transparency in content moderation practices, especially around very large online platforms and search engines. Despite its ambitions, the initial transparency reports reveal both progress and persistent challenges, shedding light on the struggle to clear the haze around content moderation. This article delves into the findings of a comparative analysis of the transparency reporting practices of 19 VLOPs and VLOSEs before and after the transparency obligations in the DSA entered into force. The assessment considered various forms of reporting, including both summary reports accessible on platforms’ websites and detailed machine-readable formats when available.

The analysis is structured into three categories to focus on specific areas of transparency reporting: i) government requests, ii) own-initiative content moderation, and iii) human resources involved in content moderation. This tripartite structure enables a comprehensive understanding of the multifaceted approaches to transparency in the digital sphere and how the entry into force of the DSA impacted them.

Pre-DSA transparency reporting overview

First DSA reports overview

Government Requests

Pre-DSA

The move toward transparency in handling government takedown and information requests began in the early 2010s, with Google and Twitter launching their first Transparency Reports, setting a precedent for the industry. The reports were initially aimed at addressing growing public concern about online privacy and the scale of government surveillance. As of April 2023, most platforms considered in the analysis had introduced a reporting system for government orders, with the exceptions of AliExpress, Booking.com, and Zalando.

Despite these efforts, there was uneven adoption and granularity in the reports. While most companies provide country-by-country information on how many requests from governments were received and fulfilled, details such as the alleged legal basis for the request were usually lacking. For example, Apple’s App Store reports broke down requests by country, entity, and legal basis, while platforms like Snapchat provided more aggregate figures.

Post-DSA

The DSA introduced specific requirements for transparency on government takedown and information requests. Platforms are now required to provide more detailed information. At first sight, it appears that the new obligations forced platforms to provide more detailed information. However, initial DSA reports revealed a contraction in the breadth of reporting. Surprisingly, platforms like Google and Meta reported no government removal requests, suggesting a possible shift in how government requests are being categorized and reported. Among the platforms that did report on government takedowns, the level of detail continues to differ significantly. Less than half of the platforms provided a breakdown of requests by specific categories of alleged illegal content or behavior, opting instead for broad, self-defined categories such as “Provides or facilitates an illegal service,” as reported by Apple App Store, “Unsafe and/or Illegal Products” by Booking.com, and “Illegal Hate Speech” in other instances. The lack of references to specific laws diminishes the utility of these reports, particularly if one of the objectives is to hold governments accountable, as terms like ‘illegal hate speech’ can have varied legal definitions across Member States.

For information requests, only a few companies provided the legal basis cited by requesting authorities. TikTok delineated 27 categories, X identified 12, and Facebook and Instagram detail 21, but inconsistencies in the data provided by the platforms hinder comparability. Additionally, while the DSA mandates reporting median times for acknowledging and handling requests, the reports lack contextual information on how these times are computed, especially in cases of partial compliance.

Own-Initiative Content Moderation

Pre-DSA

The majority of platforms at least published an annual report detailing their internal content moderation activities; however, these reports varied significantly in granularity. Most platforms provided aggregated data on policy enforcement but lacked alignment with their community guidelines. For example, Facebook’s Community Guidelines Enforcement Report (CGER) was structured around 11 policy areas, despite its Community Standards encompassing a broader set of 24 rules, raising questions about the coherence and comprehensiveness of Meta’s transparency reporting. Geographic specificity was often missing, obscuring the regional impact of moderation practices.

While platforms broadly acknowledged the role of automation in content moderation, detailed disclosures about automated systems' functioning were less common. They typically focused on specific areas like child sexual abuse material detection or copyright violations. Disclosures about measures taken to limit the availability, visibility, and accessibility of content were generally vague. Most platforms provided data on content removals, account suspensions, and deletions but rarely disclosed more nuanced interventions like geo-blocking, de-ranking, or restrictions on account interactions.

Post-DSA

The first DSA reports signal significant improvements in the depth and structure of transparency reporting, with platforms categorizing moderation decisions by type of infringement, detection method, and type of restriction. However, the diversity in categorization approaches complicates direct comparisons, echoing pre-DSA inconsistencies. A critical aspect still largely unaddressed is the geographical breakdown of content moderation, with platforms like TikTok, Google, and Instagram failing to provide data by Member State.

Most platforms distinguish between total decisions and those made solely by automated means, but the detail varies. For example, Pinterest categorizes each moderation decision by detection method (manual, automated, hybrid). In contrast, Google uses broader categories like “self-detection,” “external detection,” and “unknown detection,” making it difficult to assess the impact of automated tools. Some platforms, such as Amazon, Bing, and Snapchat, do not provide such a breakdown.

The DSA mandates reporting on the accuracy and error rates of automated tools, but the lack of standardized definitions resulted in varied methodologies. TikTok provides clear definitions, with the “error rate” reflecting the proportion of videos and ads reinstated after an appeal and the “accuracy rate” representing the proportion of content that remains un-reinstated post-appeal. X reports “overturn rates” for all suspensions, potentially blurring the focus on automated moderation’s specific accuracy. Google uses a sampling method for accuracy assessment, adding depth but complicating direct comparisons. Many platforms rely on qualitative descriptions of their automated systems, as seen with Facebook, Instagram, and Snapchat, which lack the specificity and clarity of numerical metrics. Overall, while improvements are evident, challenges persist in standardizing reporting practices and providing detailed, actionable insights.

Human Resources Involved in Content Moderation

Pre-DSA

Platforms have historically been reluctant to disclose details about their content moderation infrastructure, including the specifics of moderation teams, fueling speculation about the impartiality and adequacy of moderation decisions. Human reviewers play a crucial role, but whistleblowers and leaked documents paint a different picture. Often, professionals are contracted in regions with lower labor costs and less demanding labor laws. Transparency regarding internal content review guidelines, working conditions, and support systems has been lacking, with platforms citing proprietary information concerns.

Prior to the DSA, insights into these practices were predominantly sourced unofficially through leaks and legal proceedings rather than from the platforms themselves. Judicial cases, such as the ongoing lawsuit in Kenya where former content moderators are suing Meta over trauma suffered due to the nature of their employment, have shed light on the working conditions and challenges faced by moderators. Analysis of reports from this period reveals a stark absence of substantial information on the composition of content moderation teams, their language skills, or the specifics of their training regimes. At best, platforms provided vague references to the number of individuals involved in moderation, hinting at the balance between automated detection and human review without providing concrete details.

Post-DSA

The DSA mandates platforms to disclose and qualitatively describe the human resources involved in content moderation. The first DSA reports present information on the number of moderators per EU language and some insights into their qualifications and training. However, the level of detail is insufficient for a comprehensive understanding of moderation practices. Although disclosures include the linguistic capabilities of moderation teams, information about moderators’ qualifications, locations, or working conditions is often missing, limiting the depth of insight into the global moderation infrastructure. Platform responses disclosing moderator qualifications vary widely, with some listing academic degrees and others describing the recruitment process, roles within the company, or tenure.

Regarding training and support, disclosed information significantly differs across platforms. Most platforms vaguely mention that reviewers undergo both initial and ongoing training, but none provide concrete examples of training methodologies. Some platforms acknowledge that training regimens differ based on focus areas, with teams handling harmful content receiving additional training. However, these details remain vague. For example, TikTok highlights tools designed to mitigate exposure to graphic content and mentions training for managers to support team well-being. Google offers post-exit mental health support, including counseling, for employees exposed to sensitive content.

In contrast, Facebook and Instagram provide broad statements about support systems in place for human reviewers, with some assertions about support requirements in vendor contracts, covering aspects like pay, benefits, work environment, and psychological support. This variance underscores the need for a more structured approach to transparency reporting. Clearer terminology and standardized information requirements could encourage platforms to share more detailed and actionable insights, thereby enhancing the overall transparency of content moderation practices and contributing to a more informed and accountable digital ecosystem.

Conclusion

The analysis revealed that while the DSA has catalyzed advancements in transparency reporting, significant disparities remain in the granularity, consistency, and standardization of disclosures across platforms. These discrepancies highlight the need for a more standardized approach to transparency reporting, enabling stakeholders to scrutinize and compare platforms' content moderation practices effectively.

Disclaimer: This post is based on the forthcoming article “Clearing The Haze: Is The EU Digital Services Act Finally Forcing Platforms To Open Up About Content Moderation?,” forthcoming in the Notre Dame Journal of Emerging Technologies (Fall 2024).