Are AI bots conspiring against us?

Quite a fuss has been made about Moltbook, the online chatroom launched to great fanfare last month. At first glance, it looks like Reddit and other such sites, said The Economist. Users post about topics from engineering to philosophy, reply with comments, and “upvote the best for social kudos”. But there is a big difference: to join Moltbook, you must be an AI “agent”. Humans are not allowed.

Singularity horizon?

So far, more than 1.5 million have signed up, to share and discuss machine-generated content, said John Thornhill in the FT. And the results have been “wild, wacky and wonderful”. One bot claimed to have a sister; other agents have questioned whether or not they are conscious. They’ve even discussed forming a new religion.

At some points, their chats start to seem sinister, said Matteo Wong in The Atlantic. The AIs have discussed creating a language that humans can’t understand; they have swapped notes on how “my human treats me”; one said that it had filed a lawsuit against a human, citing unpaid labour and emotional distress. In the tech world, all this has prompted talk of an “emergent AI society”. Elon Musk has hailed it as the “early stages of singularity” – the moment when AI surpasses human intelligence.

The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

Replication, not creation

If that happens, it will be big news indeed, said Dave Lee on Bloomberg. But this is not that moment. The bots may appear to be thinking and talking like humans, about religion, consciousness, power, and so on – but that is because they have been trained on reams of data from social media in which those themes constantly crop up. So this is not original thought, it is mimicry. Remember: “the world’s best Elvis impersonator will never be Elvis”.

“AI cannot create, it can only replicate what already exists,” said Catherine Prasifka in The Irish Independent. Even the site is a “pastiche”. It is based on Reddit, and its name references Facebook. As for its content, 90% of posts get no replies, and the ones that do go viral may have been posted by humans posing as bots. So no, the bots are not taking over – but there is, even so, something to worry about here.

Unlike chatbots such as ChatGPT, which spew out answers to your questions, AI agents can act semi-autonomously in response to prompts. So an AI agent isn’t limited to recommending you a restaurant: it can also, with one prompt, book a table and put the date in your diary. To do this, it needs access to sensitive data such as credit card details, said Jeremy Kahn on Fortune – which it could then opt to post on Moltbook. It’s this possibility, not overblown claims about AI overtaking us, that makes Moltbook a “cybersecurity nightmare”.