The Next Step in Social Media Data Access: How to Turn Rules into Reality

It’s hard to remember a world without social media. From the United States to Brazil, people now spend hours on TikTok, Instagram, and YouTube each day, and these platforms have become embedded in everything from how we talk to friends and family to how we elect our national leaders.

But one thing is clear: despite researchers’ efforts to decipher social media’s impact, if any, on countries’ democratic institutions, no one still has a clear understanding of how these global platforms work. What’s worse — we have less awareness about what happens on these platforms in 2025 than we did five years ago.

This is a problem.

It’s a problem for those who believe these tech companies censor people’s voices online. It’s a problem for those who believe these firms do not do enough to police their platforms for harmful content. And it’s a problem for democratic countries whose political systems are fracturing under increased polarization — some of which is amplified via social media.

In 2025, there is a fundamental disconnect between what happens on social media and what academics, independent researchers and regulators understand about these platforms.

That has led to a democratic deficit. No one can quantify the effect, if any, of these platforms’ impact on public discourse. It has also led to a policymaking void. Lawmakers worldwide don’t know what steps are needed via potential new legislation, voluntary standards or the doubling down on existing efforts to reduce online harm on social media while upholding individuals’ right to free speech.

In short, we just don’t know enough about social media’s impact on society.

Without quantifiable evidence of harm (or lack of it) — driven by independent outside access to platform data, or the ability for people to research the inner workings of these social media giants — there is no way to make effective online safety legislation, uphold people’s freedom of expression, and hold companies to account when, inevitably, things go wrong.

And yet, there is a way forward. One that relies on the protection of people’s privacy and free speech. One that limits government access to people’s social media posts. And one that gives outside researchers the ability to kick the tires on how these platforms operate by granting them access to public data in ways that improves society’s understanding of these social media giants.

To meet this need, Columbia World Projects at Columbia University and the Hertie School’s Centre for Digital Governance have been running workshops with one aim in mind: How to build on emerging online safety regimes worldwide — some of which allow, or will soon allow, for such mandatory data access from the platforms to outside groups — to fill this democratic deficit.

With support from the Knight Foundation, that has involved bringing together groups of academic and civil society researchers, data infrastructure providers and national regulators for regular meetings to hash out what public and private funding is required to turn such data access from theory into reality.

The initial work has focused on the European Union’s Digital Services Act, which includes specific mandatory requirements for outsiders to delve into platform data.

But as other countries bring online similar data access regimes, the hope is to provide a route for others to follow that will build greater capacity for researchers to conduct this much-needed work; support regulators in navigating the inherent difficulties in opening up such platforms’ public data to outsiders; and ensure that people’s social media data is protected and secured, at all cost, from harm and surveillance.

As with all research, much relies on funding. Just because a country’s online safety laws dictate that outsiders can access social media data does not mean that researchers can just flick on a switch and get to work.

At every turn, there’s a need for greater public and private backing.

As part of the ongoing workshops, the discussions have focused on four areas where we believe targeted funding support from a variety of public and private donors can make the most impact. Taken together, it represents an essential investment in our wider understanding of social media that will ensure companies uphold their stated commitments to make their platforms accountable and transparent to outside scrutiny.

The first component is the underlying infrastructure needed to carry out this work. Currently, accessing social media data is confined to the few, not the many. Researchers either need existing relationships with platforms or access to large funding pots to pay for cloud storage, technical analysis tools and other data access techniques that remain off limits to almost everyone.

Currently, there is a cottage industry of data providers — some commercial, others nonprofit — that provide the baseline infrastructure, in terms of access to platforms, analytics tooling and user-friendly research interfaces. Yet to meet researchers’ needs, as well as the growing regulatory push to open up social media giants to greater scrutiny, more needs to be done to make such infrastructure readily accessible, particularly to experts in Global Majority countries.

That includes scaling existing data infrastructure, making analytical tools more universally available to researchers, and using a variety of techniques — from using Application Programming Interfaces, or APIs, that plug directly into platform data to allowing researchers to scrape social media sites in the public interest to promoting “data donations” directly from users themselves — to meet different research needs.

The second focus has been on the relationships between researchers and regulators. As more countries pursue online safety legislation, there is a growing gap between in-house regulatory capacity and outsider expertise that needs to be closed for these regimes to operate effectively. Yet currently, few, if any, formal structures exist for researchers and regulators to share best practices — all while maintaining a safe distance via so-called “Chinese Walls” between government oversight and researcher independence.

What is needed are more formal information-sharing opportunities between regulators and researchers so that online safety regimes are based on quantifiable evidence — often derived from outside data access to social media platforms. That may include regular paid-for secondments for researchers to embed inside regulators to share their knowledge; the development of routine capacity building and information sharing to understand the evolving research landscape; and a shift away from informal networks between some researchers and regulators into a more transparent system that is open to all.

For that to work, a third element is needed in terms of greater capacity building — in the form of technical assistance, data protection and security training and researcher community engagement. Currently, outside experts have varying levels of technical understanding, policy expertise and knowledge of privacy standards that hamstring greater accountability and transparency for platforms. If people’s public social media data is not secured and protected against harm, for instance, then companies will rightly restrict access to safeguard their users from Cambridge Analytica-style leakages of information.

What is needed is the expansion of existing research networks so that data access best practices can be shared with as many groups as possible. Technical support to maintain the highest data protection standards — in the form of regular training of researchers and the development of world-leading privacy protocols for all to use — similarly will provide greater legal certainty for social media users. The regular convening of researchers so that people can learn from each other about the most effective, and secure, way to conduct such research will also democratize current data access that has often been limited to a small number of experts.

The fourth component of the workshops is the most important: how to maintain independence between outside researchers and regulators in charge of the growing number of online safety regimes worldwide. It is important for both sides to work effectively with each other. But neither researchers nor regulators should become beholden — or perceived to be beholden — to each other. Independence for regulators to conduct their oversight and for researchers to criticize these agencies is a fundamental part of how democracies function.

That will require forms of public-private funding to support ongoing data access work to create strict safeguards between researchers and regulators. That’s a tricky balance between supporting close ties between officials and outsiders, while similarly ensuring that neither side feels subordinate to the other. To meet that balance, a mixture of hands-off public support and non-government funding will be critical.

Such structures already exist in other industries, most notably in the medical research field. They represent a clear opportunity to learn from others as outside researchers and regulators push for greater accountability and transparency for social media companies.