Mark Zuckerberg is Out of Ideas

Earlier this week, Meta founder and CEO Mark Zuckerberg posted a new online mini-manifesto, titled “Personal Superintelligence,” setting the direction for Meta’s AI program.

It is a tidy piece of writing, taking a mere 616 words to say almost nothing at all. The bulk of the essay is so bland and unremarkable that I am tempted to think Zuckerberg had ChatGPT compose the first draft.

Let’s start with the obvious: “superintelligence” is just a rebrand of artificial general intelligence (AGI). OpenAI CEO Sam Altman and his team have spent a decade hyping up AGI as the radical, threshold-crossing moment where machine learning systems transform all of society. Decade-old digital futures tend to get a little dusty, leading Silicon Valley to repackage those same ambitions in a new bottle. “Superintelligence” is to “AGI” as Apple’s “spatial computing” is to the “Metaverse.” With a large enough marketing budget, everyone just kind of lets you rename things and act like it’s new.

The text of Zuckerberg’s message lacks any critical insight or details. First, he tells us that superintelligence is coming. Soon. Second, it will be a “new era for humanity.” Third, he is optimistic that it will accelerate all sorts of positive trends. But fourth, there will be bad stuff too!

He goes on to lay out what makes Meta’s superintelligence program different from the company’s competitors.

Meta's vision is to bring personal superintelligence to everyone. We believe in putting this power in people's hands to direct it towards what they value in their own lives.

This is distinct from others in the industry who believe superintelligence should be directed centrally towards automating all valuable work, and then humanity will live on a dole of its output. At Meta, we believe that people pursuing their individual aspirations is how we have always made progress expanding prosperity, science, health, and culture. This will be increasingly important in the future as well.

He also thinks it would be cool to have people access AI through smart glasses. It must be nice for investors to hear that he isn’t completely writing off the ~$60 billion that the company has gambled on heads-up displays thus far.

I suspect the reason why the text of Zuckerberg’s message is so thin is because the subtext would be quite a bit more alarming if he took the time to spell it out.Meta’s comparative advantage is behavioral data. It has a lot of it. OpenAI and Anthropic have much less of it. Meta has the social graph of billions of Facebook, Instagram, WhatsApp, and Threads users. If the future of AI bends in the direction of personalization, then Zuckerberg will have an advantage down the road.

What Meta is historically very good at is figuring out what will keep you clicking, and then giving you a lot of it. The company doesn’t really care if this is healthy for you, or if it is good for society, democracy, etc. The company ultimately likes to optimize for a tight feedback loop: “did aggregate time on site increase, y/n?”

It isn’t hard to imagine how this might work out. Consider that language about “putting power in people’s hands.” What do we know thus far about the ways that people, empowered by AI, like to use the technology? Deepfake porn and harassment are unsettlingly popular activities. In the absence of guardrails, awful people seem very eager to use these technologies to pursue awful ends.

What will Meta do to prevent this? Considering how the company has hollowed out its trust and safety teams and gone all-in trying to appeal to the right, we should probably assume things will start out bad, get worse, and that eventually Zuckerberg will be compelled to release another canned apology statement. But, more to the point, Zuckerberg’s most-honest answer circa July 2025 is almost certainly ‘TBD.’ There is simply no evidence that he has thought any of this through at all.

AI superintelligence is the new tech race. And Mark Zuckerberg’s single most deeply-held belief appears to be that his company ought to win whatever tech race is currently being run. The company’s stock is up almost 10% since his announcement, so in one narrow sense, it is working.

But the one strong prediction we can make right now is that the future of AI will be a gigantic mess, with huge social costs. Zuckerberg is determined to spend whatever it takes to win the superintelligence race. He will try to bend the future of the technology in the direction of his company’s existing comparative advantages.

And he hasn’t thought the rest of this through at all.