What It Now Takes to Establish a Fact

Hilke Schellmann and Mihir Kshirsagar, both from Princeton CITP, detail findings from the report “Holding the Line: Authentication, Verification, and the Fight for Facts in the AI Age.”

In early July, a fake photograph of Senator Mitch McConnell (R-Ky.) on life support went viral on social media. Detection tools eventually caught it: the image carried an invisible watermark that some AI generators now embed. A few days later, McConnell’s office released an authentic photograph meant to end the speculation about McConnell’s health, and the forensic tools stuttered. Some found no trace of AI; others reported a low probability; none could definitively prove the picture was genuine. This incident highlights the core challenge in the age of AI: how do we sustain an information ecosystem in which facts can be established?

Today, faking things is getting cheaper and quicker. Meanwhile, verifying things can be expensive and slow. This asymmetry is getting worse since the reference points reporters and fact-checkers once checked against — websites, documents, public records, a voice on the phone — can now themselves be fabricated, so establishing one fact increasingly means establishing the facts beneath it. It’s a structural threat, more than any individual synthetic image. And dismissing something true is nearly free: anyone can call a genuine report machine-made, and the accusation collects a “liar’s dividend” reliably.

This is the type of problem we brought journalists, researchers, and technologists together to discuss in June in a workshop organized by Princeton’s Center for Information Technology Policy (CITP) and NYU Journalism. In a new report drawn from the proceedings out today, we tackle information integrity, authentication and verification in the age of AI.

AI and machine learning tools are helping researchers and journalists verify information and dig through copious amounts of data to find new insights and reveal patterns. For example, many news organizations used similar tools when reviewing materials from the Epstein files to break stories. But one of the critical findings in the report is that there is less maturity in forensic tools for assessing AI-generated material. Many AI-based deepfake checking tools are built without journalists, platform engineers or audiences participating in their development. As a result, these tools lag in adoption and are difficult to use.

Journalists that participated in the June convening noted that many of the current deepfake authentication tools’ confidence measures do not translate to real world reporting. A 70% confidence rating that an image is not AI-generated is not helpful for reporters, since they are more comfortable with a binary assessment about whether an image is authentic or not. The same goes for audiences that want to know if an image is genuine or not, rather than a confidence rating that reporters often cannot even explain since authentication tools are often “black boxes” themselves. We urge technologists to work with reporters to develop these tools together and then let reporters evaluate their usefulness. Without concerted effort across different actors in the information ecosystem, the problem will get worse.

Consider the challenge of content provenance: The Coalition for Content Provenance and Authenticity (C2PA), led by tech firms, developed an open standard that attaches a cryptographic manifest to a photograph or video at the moment of capture—a chain of custody that runs from the glass in the camera to the glass in a reader’s hand. But participants in the June event conveyed that, in practice, it is difficult to implement successfully. Such standards touch every step of the content creation and distribution workflow, and require a commitment to ongoing testing and retesting.

An engineer from a North American newsroom shared under Chatham House rule that their organization encountered problems with the C2PA implementation every step of the way. Their ingestion software stripped the manifest until the photo software vendor was persuaded to fix it. Then, the content management system did not support the standard, so the manifest was removed before publication.

So without the manifest, will audiences believe the news organization has a chain of custody for its files if it cannot offer "full transparency"? Would audiences essentially just have to take the news organization's word for it? Or would the newsroom weaken its own long-established trust signal if it introduces another credential? One participant shared that their legacy news organization declined to be part of C2PA; they argued that their brand is their credibility and another standard might weaken that in the eyes of the audience and take their agency away. But for many local news organizations that run national or international news, this might bolster trust if audiences understand the standard.

Moreover, before publication, many social media platforms strip out the metadata and cryptographic signatures from images and videos when they distribute the story.

In other words, having a shared standard is not the same as shared priorities. The risk is that the technology companies supporting the C2PA standard could lose interest, and news organizations may not be in a position to continue supporting it.

WITNESS, a leading organization documenting human rights violations, recently released a report highlighting a further complication: poor implementation of C2PA standards could result in enabling surveillance unless safeguards are put in place. Photographers today who are sent into hostile environments are deliberately equipped with cameras without the standard, for fear the manifest’s metadata could be used to track them—so the most dangerous reporting arrives looking the least verified.

Our report’s central observation is that the incentives to build tools that promote trust rarely align across all of the actors in the information ecosystem. Those with the deepest pockets, including many technology and social media companies, are often not interested in supporting careful fact-finding.

The verification challenges already in play are a preview of more structurally disruptive threats. When trying to verify if the information in an image is trustworthy by calling experts and eye witnesses, how can a reporter be confident that the person they are interviewing over Zoom or by phone is the person they say they are? How can they verify that a website genuinely belongs to the organization it claims to represent, when AI can generate convincing institutional facsimiles—functional websites, plausible domain histories, fabricated organizational records—at minimal cost? What will count as a trustworthy source in the age of AI? And how will this more challenging fact-checking environment play out, as generative AI could soon degrade the underlying information supply?

Pioneering efforts are underway, often with philanthropic support, to build new networks and institutions, such as Indicator Media, Bellingcat, WITNESS, and the Global Investigative Journalism Network. There is room for much more experimentation, however, with new business models for collaborative networks, including building out academic infrastructure to bring more long-term research and thought leadership to the space.

Two futures are visible from here. In one, we lose confidence in facts altogether and settle into a world of rumor and motivated speculation. In another, a smaller set of trusted producers hold their ground while the surrounding flood of unverified material is met with appropriate suspicion. Neither future is guaranteed, and both may coexist. Which vision predominates depends less on any single invention or standard than on whether the people who make, check, distribute, and read the news are actually communicating with each other.