Prithvi Iyer is a Program Manager at Tech Policy Press.

The US 2020 research project is a partnership between Meta researchers and external academics to explore questions related to Facebook and Instagram in the 2020 US election. The external team of academics was led by Professor Talia Jomini Stroud, founder and director of the Center for Media Engagement at the University of Texas at Austin, and Professor Joshua A. Tucker, co-director of the Center for Social Media and Politics at New York University. Professors Stroud and Tucker selected 15 additional researchers to collaborate on this endeavor. 

Meta claimed that neither its researchers nor the company as a whole had the authority to restrict or tamper with the findings – irrespective of whether they are favorable or not for the company. However, Meta was closely involved in shaping the research questions, methodology, workflow, and design choices. As noted by Michael Wagner, a professor at the University of Wisconsin-Madison tasked with evaluating the rigor of the collaboration as its rapporteur, “Meta set the agenda in ways that affected the overall independence of the researchers.” Nevertheless, the studies themselves, he said, are “rigorous, carefully checked, transparent, ethical, and path-breaking,” and are thus important to understand, as they may help inform a number of policy questions related to social media and these platforms specifically.

Four research papers have been published as part of this project so far, with more to follow. This post serves as a resource to collect the studies and their findings in one place. It is intended for people interested in making informed assessments of the research and its claims by providing a snapshot of the research process and results. This resource also enables readers to examine the research papers in tandem, facilitating comparisons and assessments of the results in relation to one another. We hope that this primer on the research project serves to distill and simplify the key findings of these studies while ensuring that readers can still pick up on the nuance and caveats needed to accurately infer the implications. Of course, readers are also encouraged to read the full papers. 

Table of Contents

1. How do social media feed algorithms affect attitudes and behavior in an election campaign? (Science, 27 July 2023)
2. Reshares on social media amplify political news but do not detectably affect beliefs or opinions (Science, 27 July 2023)
3. Asymmetric ideological segregation in exposure to political news on Facebook (Science, 27 July 2023)
4. Like-minded sources on Facebook are prevalent but not polarizing (Nature, 27 July 2023)

---

1. How do social media feed algorithms affect attitudes and behavior in an election campaign? 

Published in Science, 27 July 2023.

Context

One concern about social media platforms is whether the algorithms that select content, such as the recommender systems driving Facebook’s News Feed, serve to polarize users. Some have suggested reverting to a “chronological feed” – a feed of posts ordered in reverse chronology as opposed to by an algorithm – might reduce polarization. 

Hypotheses

• Chronological Feed ranking would reduce issue based polarization vis-a-vis algorithmic based ranking. 
• Chronological feeds would “reduce affective polarization on an individual level.”
• Chronological feed would decrease knowledge about the 2020 election campaign and decrease recall of recent events covered in the news.
• “Chronological feeds would reduce both online and offline forms of political participation, including self-reported turnout in the 2020 election  and on-platform political engagement.”

Methods

• “Study participants were recruited through survey invitations placed on the top of their Facebook and Instagram feeds in August 2020. Participants were users residing in the United States who were at least 18 years of age and who provided informed consent.”
• “Users were invited to complete five surveys, share their on-platform activity, and participate in passive tracking of off-platform internet activity.” They could withdraw at any time.
• Participants were randomly assigned to a control group (Algorithmic Feed) or to the treatment group (Chronological Feed), in which the most recent content appeared at the top of those feeds.
• Outcome variable operationalized via Population Average Treatment Effect. This means that the study measured participation by looking at “an average of users’ predicted ideology, friend count, number of political pages followed, and number of days active, among other variables.”

Results

• “Users in the Chronological Feed group spent dramatically less time on Facebook and Instagram” (73% less daily compared to the control group).
• Treatment group members migrated to other platforms, spending more time on Tiktok and Youtube. “36% (2.19 hours) for Tiktok and 20% (5.63 hours) over the entire study period for Youtube.”
• “Users in the treatment group (chronological feed) showed lower engagement on both Facebook and Instagram.” Lower engagement was captured via lower number of likes, reshares and comments.
• Participants with the Chronological Feed reduced the share of content from ideologically “cross-cutting sources on Facebook (18.7 versus 20.7%, p < 0.005) and also reduced the share of content from ideologically like-minded sources on Facebook (48.1 versus 53.7%, p < 0.005).”
• On Facebook, the Chronological Feed “increased the share of content from designated untrustworthy sources by more than two-thirds relative to the Algorithmic Feed , whereas it reduced exposure to uncivil content by almost half.” These trends were not statistically significant for Instagram. 
• The Chronological Feed condition “did not express significantly lower levels of affective or issue polarization compared to  those in the Algorithmic Feed condition.” Thus, H1 was refuted.

• There were no statistically significant differences between treatment and control groups with respect to “election knowledge or news knowledge on either platform.”
• No differences in political participation (voter turnout) between treatment and control groups.

Discussion 

• What explains the disconnect in findings? As in, why did users in the treatment group report dramatic changes in online behavior (lower participation etc) but no changes in polarization levels? This result may seem counterintuitive. Some reasons for these findings  proposed by authors are:
  • Downstream effects of social media on political polarization may need a “longer intervention period”.  Three months may be too short for a robust intervention.
  • The US context may not be generalizable to other countries.
  • People in the treatment group (chronological feed) often saw posts from other users (friends/followers) that still had  the algorithmic ranking that could have biased their feeds. Thus, we cannot discount the possibility that users in the treatment group were still exposed to the algorithmic ranking by proxy.  Addressing this may require scaling the study to ensure exposure to content is also mediated by group assignment.

2. Reshares on social media amplify political news but do not detectably affect beliefs or opinions

Published in Science, 27 July 2023.

Context

The “reshare” function, present on platforms such as Facebook and Instagram, has been identified as a catalyst for content to attain a “viral” status. This study seeks to investigate the viability of a policy intervention involving the removal of resharing as a means to alleviate the detrimental impacts associated with viral content like increasing online polarization. 

Hypotheses

• “Withholding reshared content from users’ feeds may reduce affective and issue polarization by decreasing their exposure to emotionally or ideologically inflammatory content.”
• “Removing reshared content would, on net, reduce accurate knowledge about the election campaign.”

Methods 

• “Adult users residing in the United States who provided informed consent (N = 193,880, 1.3% of those who saw the invitation) were invited to complete five surveys and share their Facebook activity (the study was advertised to 14.6 million FB users).”
• Study compared two groups;  a control condition in which no changes were made to their Facebook feeds, and a treatment condition in which no reshared content (from friends, Groups, or Pages) was shown in the feed.
• As with the earlier study, the “main estimand of interest is the population average treatment effect, which is weighted by users’ predicted ideology, friend count, number of political pages followed, and number of days active, among other variables”

Results

• The control group spent “73% more time each day on average compared to US monthly active users, whereas time spent reduced to 64% more for those in the No Reshares group.” The treatment effects were gauged via  Ordinary Least Squares  regression.
• No significant migration to other social media platforms for the no-reshares group. This indicates that users who did not see reshares were still incentivized to remain on the platform rather than opt for other  platforms like X and TikTok among others.
• “The No Reshares treatment decreased the relative proportion of content seen by participants that is posted by their friends by an average of 10 percentage points while increasing the relative share of content from Groups by 8 percentage points and from Pages by 2 percentage points.”
• Feeds without reshares had less political news compared to the control group. “Untrustworthy news sources also declined from 6.2% TO 2.5% for the no reshares group.”
• “The No Reshares condition decreased the proportion of both like-minded (51.1 versus 53.7%, p < 0.005) and cross-cutting (19.7 versus 20.7%, p < 0.005) content, while increasing that of ideologically moderate content by more than 15% (26.2 versus 22.6%, p < 0.005).”

Results pertaining to core hypothesis

• No significant difference in polarization between both groups.
• “There was also no statistically distinguishable change in election knowledge, i.e., users were less likely to correctly remember recent events. However, when looking at sample average treatment effects show that the “no reshares” group had less news knowledge.”
• “The treatment does not have statistically distinguishable effects on perceived accuracy of various factual claims, trust in media (either traditional or social), confidence in political institutions, perceptions of political polarization, political efficacy, belief in the legitimacy of the election, or support for political violence.”

Takeaways

• “The reshares feature could be a double-edged sword: It facilitates encounters with both reliable news about politics and current events but, to a somewhat lesser degree, also content from untrustworthy sources that may exaggerate or fabricate information.”
• “Without reshares on their feeds, users were less likely to click on outbound links from news sources with highly ideological audiences. However, this  does not manifest in increasing issue or affective polarization.” Study concludes that though reshares may have been a “powerful mechanism for directing users attention and behavior on Facebook during the 2020 election campaign, they had limited impact on politically relevant attitudes and offline behaviors.”

3. Asymmetric ideological segregation in exposure to political news on Facebook

Published in Science, 27 July 2023.

Context

Social media plays a pivotal role in shaping how society engages with information. This study examines the “funnel of engagement” with respect to seeing political news in context to the 2020 US election. The “funnel of engagement” can be understood as the link between what users could potentially see on their feed, what they actually see and eventually what they engage with (via likes/comments/reshares etc). This study responds to the concern that political polarization is closely related to the structure of news feeds on social media platforms by examining how the funnel of engagement on Facebook and Instagram shapes what users engage with and if that content bolsters online polarization. 

Research Questions

• “How ideologically segregated is news consumption on Facebook, and are those patterns of segregation symmetric on the right and left.”
• “How does segregation vary with potential news consumption versus actual exposure ver- sus engagement.”
• “How does segregation vary if the level of analysis is URLs rather than domains (thus capturing curation of content within domains).”
• “How segregated is exposure on Facebook relative to the benchmark of browsing behavior (the predominant source of data in past research).”
• “How segregated are the streams of content from the major path- ways to exposure on Facebook (friends, Pages, and Groups).”
• “How prevalent is exposure to unreliable content on the right relative to the left.”

Methods

• The data in this paper draw from the set of  208 million US- based adult active Facebook users “whose political ideology can be measured and track all URLs classified as political news that were posted on the platform from 1 September 2020 to 1 February 2021.”
• In the  analyses, the authors “only examined posts classified as political news that contain a URL, which amount to about 3% of all posts shared by US adult users and 3.9% of all content that US adult users saw on the platform during our study period.”
• For each URL (and corresponding domain), the authors reported “measures of the potential, exposed, and engaged audience. The potential audience of a URL is the set of unique users that could have been exposed to that content, the exposed audience is the set of unique users that saw a post containing that URL on their Feed and the engaged audience is the set of unique users that clicked, reacted, liked, reshared, or commented on the post with the URL.”
• In total, the data comprised aggregated exposure and engagement metrics for “208 million US adult active users with an ideology score in relation to 35,000 unique domains and  640,000 unique URLs that were classified as political news and were shared more than 100 times during the study period.”
• The analyses rely on two measures: the segregation index (which offers a summary statistic of the entire information environment) and the favorability scores (which are associated with individual domains and URLs which allow the researchers to infer the ideological composition of their audiences). The authors dichotomized the ideology scores such that users with a score ≤0.35 are categorized as liberal, and those with a score ≥0.65 are categorized as conservative. For the favorability scores, 1 indicates the URL has a conservative audience, -1 indicates a liberal audience and 0 indicates equal distribution across partisan lines. The authors define audience polarization as the extent to which the “distribution of favorability scores is bimodal and far away from zero.”

Results

• “The segregation score based on exposed audience for domains fluctuates around 0.35 (i.e., the gap between the intersection of conservatives with conservatives versus liberals with conservatives is 35 percentage points).”
• “Algorithmic and social amplification are both contributing to increased segregation: As we move down the funnel of engagement (i.e., as the footprint of algorithmic and social curation becomes more evident), liberal and conservative audiences become more isolated from each other.”
• Mean favorability scores indicate that FB users consuming political news are mostly conservative. “There are more domains and URLs being favored by very conservative audiences.”
• Most sources of misinformation are favored by conservative audiences. The study also finds that “algorithmic and social amplification do not exacerbate the already existing audience segregation for misinformation content. However, misinformation shared by Pages and Groups has audiences that are more homogeneous and completely concentrated on the political right.”
• “News sources and stories consumed by conservative audiences depart more clearly from the zero line of cross-cutting exposure, which means that their audiences are more homogeneously conservative and, therefore, more isolated. These outlets on the right also post a higher fraction of news stories (URLs) rated false by Meta’s 3PFC program, which means that conservative audiences are more exposed to unreliable news.”

Takeaways

• Facebook is ideologically segregated.
• Ideological segregation manifests far more in content posted by Pages and Groups than in content posted by friends.
• Pages and Groups are associated with higher levels of ideological segregation  which suggests that the choice of which Pages to follow and which Groups to join is “driven far more by ideological congruence than the choice of with whom to be friends.”
• “Pages and Groups benefit from the easy reuse of content from established producers of political news and provide a curation mechanism by which ideologically consistent content from a wide variety of sources can be redistributed.”
• “Although there are homogeneously liberal and conservative domains and URLs, there are far more homogeneously conservative domains and URLs circulating on Facebook. This asymmetry is consistent with what has been found in other social media platforms.”

4. Like-minded sources on Facebook are prevalent but not polarizing 

Published in Nature, 27 July 2023.

Context

Social media has received criticism for creating “echo chambers” – spaces where users mainly encounter content that aligns with their existing beliefs, potentially deepening polarization. This study investigates the relationship between exposure to like-minded content and polarization, to assess the effectiveness of potential policy interventions designed to address polarization by altering the structure of news feeds in a manner that discourages the formation of these “echo chambers.”

Research Question

• To what extent do “echo chambers’’ on Facebook exacerbate political polarization in context to the 2020 US election?

Methods

• The authors use data from all active adult Facebook users in the USA to analyze how much of what they see on the platform is from sources that we categorize as sharing their political leanings.
• With a subset of consenting participants, the authors evaluate a potential response to concerns about the effects of echo chambers by conducting a large-scale field experiment reducing exposure to content from like-minded sources on Facebook. Thus, participants were randomized into treatment and control groups wherein the treatment group saw less posts from like-minded friends/pages/group.
• “Participants in the treatment and control groups were invited to complete five surveys before and after the 2020 presidential election assessing their political attitudes and behaviors. Two surveys were fielded pre-treatment: wave 1 (31 August to 12 September) and wave 2 (8 September to 23 September). The treatment ran from 24 September to 23 December.”
• For participants assigned to treatment, the authors  “downranked all content (including, but not limited to, civic and news content) from friends, groups and Pages that were predicted to share the participant’s political leaning (for example, all content from conservative friends and groups and Pages with conservative audiences was downranked for participants classified as conservative”
• Political leanings were measured via Facebook’s internal Machine Learning classifier that accounts for user data to make predictions about political affiliation. Users with predicted values greater than 0.5 were classified as conservative and otherwise classified as liberal, enabling the researchers to analyze the full population of US active adult Facebook users. 

Results

• Despite reducing exposure to content from like-minded sources by approximately one-third over a period of weeks for the treatment group, the authors “find no measurable effects on 8 pre registered attitudinal measures, such as ideological extremity and consistency, party-congenial attitudes and evaluations, and affective polarization.”
• “Despite the prevalence of like-minded sources in what people see on Facebook, extreme echo chamber patterns of exposure are infrequent. Just 20.6% of Facebook users get over 75% of their exposures from like-minded sources.”
• For the treatment group, “less exposure to like-minded sources did not lead to a proportional increase in exposure to cross-cutting sources.” Instead, it led to an increase in exposure to content that was neither like-minded nor cross-cutting.

Takeaways

• The authors find that only a small proportion of the content that Facebook users see explicitly concerns politics or news and relatively few users have extremely high levels of exposure to like-minded sources. However, a majority of the content that active adult Facebook users in the US see on the platform comes from politically like-minded friends or from Pages or groups.
• Reduced exposure to like-minded content also led to reduced exposure to uncivil content and misinformation.
• Some reasons why reduced exposure to like-minded content (opposite of an echo chamber) did not lead to increase in polarization?
  • “Political information and partisan news—the types of content that are thought to drive polarization—  account for a fraction of what people see on Facebook.”
  •  “Large shifts in exposure on Facebook may be small as a share of all the information people consume.”
  • “Persuasion is simply difficult—the effects of information on beliefs and opinion are often small and temporary and may be especially difficult to change during a contentious presidential election.”

– – –

Note: Tech Policy Press will periodically update this post as the US 2020 research project releases future publications. If you have a comment or suggestion on this material, please reach out to me.