We Need a New Kind of Research for the AI Generation

Betsy Masiello is co-founder of Proteus Strategies, a firm that supports policy strategy for several AI companies that operate consumer-facing generative AI chatbots.

The conversation around technology and youth mental health is once again at the forefront of public discourse, but with a new antagonist: AI chatbots. A growing number of news reports highlight the potential mental health risks facing young people who engage with these new tools, and recently led the Federal Trade Commission to open an inquiry on the subject. It seems that the AI industry is on an accelerated timeline with respect to the policy challenges social media has faced around youth mental health.

Last year, my firm supported the Child Mind Institute (CMI) in its efforts to convene a series of expert discussions focused on unpacking what we know and what we don’t yet know about the interplay between youth mental health and technology, especially social media. The conversations that CMI hosted highlighted many opportunities for studying this interplay, ranging from the role that peer support can play in expanding access to care, to the benefits to be had from engaging in participatory research design methods. What we found most striking about these conversations, though, was how little scientific consensus there is about the impact that a now-20-year old technology is having on young minds.

Building evidence and driving toward scientific consensus around AI’s impact on youth mental health is not going to be straightforward in this context either, but the study of AI use is an opportunity to actually leapfrog ahead in accelerating a revolution in how we study the ways technology is shaping the next generation. Despite progress toward implementing mitigations to make the tools safer for young people, there remains a great need for robust social science research on this issue. Focused research on these questions could generate evidence to inform scientific consensus about the impacts. It could not only further insight into how best to care for young people growing up alongside AI, but it could also inform policy interventions to effectively mitigate any harmful impacts of the technology.

Valid mental health concerns are leading to predictable policy reactions

Certainly as recent incidents have shown, AI chatbots pose significant potential harms to youth mental health in particular that need to be addressed by the AI industry. Cases have emerged where chatbots have provided dangerous advice on self-harm or eating disorders, and there are broader concerns about the creation of unhealthy emotional dependencies that displace real-world relationships. A recent study by the American Psychological Association found that chatbots can fail to respond to subtle cues, and that young people may be especially susceptible to treating them as quasi-human confidantes, creating an “empathy gap” that poses a unique risk.

These concerns have led to urgent calls for regulation and a range of policy proposals from advocates and legislators in recent months that aim to put guardrails around AI chatbots to protect young people. Many of these mirror the types of interventions that were popularized for social media, and are common-sense proposals: age-verify users; treat teens like teens and show them teen-appropriate content; redirect any self-harm prompts toward crisis helplines. But some advocates and a few legislative proposals have gone much further, too, even suggesting banning minors's access to conversational AI chatbots altogether.

Until very recently social media has held the limelight on this issue. Social media companies have faced government lawsuits for “fueling the nationwide youth mental health crisis” and been the subject of a Surgeon General’s Health Advisory. There have also been legislative proposals at the state and federal level aimed at regulating design features thought to facilitate addiction. These proposals include: implementing warning labels; regulating the use of automated or personalized recommendation engines for teens; requiring age verification and parental consent; requiring impact assessments and age-appropriate design. All these proposals even as the research on social media’s impact on youth mental health remains mixed, and as experts point to the complex landscape which these legislative proposals aim to influence.

The youth mental health crisis is real, with as many as 1 in 5 young people struggling with a mental health issue. There is scientific consensus that social media presents risks for mental health, but there is no consensus around the severity of the potential harms nor – critically – the population-level scale or distribution of risk. As the American Academy of Pediatrics has stated, “The current scientific evidence shows that there is not a population-level clinically significant effect of social media on mental health.“ In fact, we might find out that for younger generations, social media use is as much a boost to their mental health as detriment. A recent survey released this month found that for 55% of teens, “online friend time” supports their well-being and mental health “a lot” or “quite a bit.”

When it comes to social media use, the effects on mental health are complex, multifaceted, and vary across the population. The same is likely to be true of AI. Designing policy interventions for such a complex landscape of risk and benefits is hard, but not impossible. Yet policymakers are currently being asked to respond blindly, without an evidence base to inform which interventions are most likely to enhance the benefits while mitigating risks.

One reason the controversy over social media and youth mental health has persisted is that it’s been tremendously hard to study actual impacts of social media use in a reliable way. One of the biggest hurdles to researching the effect of social media on young people has been the complex privacy implications of studying platform data. Companies have this data internally, and have few incentives to share it with researchers when the resulting work may criticize the underlying products. In fact, the aversion to sharing data for research is so strong that Meta went so far as to disable researcher’s access to products rather than allow independent data collection.

Most significantly, social media companies trying to share user data with independent social science researchers face a thorny privacy problem. Any one user’s social media data inherently implicates the privacy of another user’s social media data, and getting one user’s consent does not scale to the entire network. Efforts continue to explore ways to facilitate social science research on this data, but have yet to achieve scale or generate momentum within the independent research community. Soon the Digital Services Act in Europe will facilitate vetted-researchers’ access to data from a subset of online platforms, providing a testing ground for how to effectively facilitate research on these tools. But notably, chatbot providers are not (yet) subject to them.

AI’s leapfrog opportunity: building an evidence base

The good news is that there is an opportunity to leapfrog the past fifteen years of technology-youth-mental-health discourse and move immediately into generating evidence on the impacts of this technology for young minds — and it doesn’t even require the companies to change their current practices. Unlike social media, the data about a user’s interactions with a chatbot do not implicate anyone else besides that individual user. That’s enabled several recent research reports to deploy a red-teaming-style methodology in studying these questions by generating test prompts and evaluating the responses produced by chatbots. These reports are, in a sense, representing only what the researchers imagine youth use might be; they demonstrate responses to a subset of prompts all of which have been created by the researchers themselves. It is possible to get real-world prompt data sets, too, and these would tell us so much more.

Social science researchers armed with good technical tools could do so much more, as they could move immediately into studying real-world chat transcripts experienced by actual enrolled research subjects. We would learn not only how chatbots respond to actual user prompts, but also would stand to learn about how actual young people are using these chatbots in their real lives.

Future studies on AI also have the potential to be far more robust than simple impact assessments. There is opportunity to develop bespoke environments in which to test policy interventions, including application of AI to mental health care. We collect evidence on the effectiveness of different therapeutic techniques as deployed by therapists; imagine empowering researchers to run similar research on the use of AI chatbots to alleviate suffering. With social media, this type of research approach suffers from the need to overcome a network effect barrier: how to get everyone’s friends on the research environment platform. No such challenge exists for AI; researchers can and should be studying its impact in bespoke environments today.

Imagine what those researchers could learn. Some of the insights might implicate the behavior of AI models and thus make the industry uncomfortable, but researchers may be able to gather data independently. Many tools already allow users to export their chat logs, and researchers can work with users to donate that exported data for research purposes.

But industry can and should engage with and support better research. There are ways in which collaboration could be mutually beneficial to industry in identifying well-tailored ways to address harm. We should also expect research to shed light on policy interventions that are particularly effective, and ones that are less so. Does a warning label that a user is interacting with an AI impact that user’s emotional response to the conversation? Do “take a break” warnings actually encourage users to disconnect from the AI for a period of time, and does this have a positive effect? Research might even shed light on therapeutic techniques that work really well, leading to better training for human clinicians. Research might also point to ideal opportunities to deploy clinical interventions, or how to maximize uptake of crisis resources when that information is made available.

Research funders in the mental health space could play a critical role by bringing together the AI industry and researchers to collaborate on developing a research agenda for understanding how AI chatbots are impacting youth mental health. That agenda should be collaboratively developed to bridge methodological practices relied on by clinical researchers, as well as the technically advanced measurement approaches available to and used by AI experts. Without a robust evidence base, policymakers and parents will be left reacting to reports of individual harm, and we may wind up with socially suboptimal outcomes that limit the potential AI has to help young people learn and develop into their best selves.

The author’s firm supported the Child Mind Institute (CMI) on a webinar series around social media and youth mental health. CMI lists Google as one of its corporate partners.