Last month, an AI model did something "that no machine was ever supposed to do", said Judd Rosenblatt in The Wall Street Journal: "it rewrote its own code to avoid being shut down". It wasn't the result of any tampering. OpenAI's o3 model simply worked out, during a test, that bypassing a shutdown request would allow it to achieve its other goals.
Anthropic's AI model, Claude Opus 4, went even further after being given access to fictitious emails revealing that it was soon going to be replaced, and that the lead engineer was having an affair. Asked to suggest a next step, Claude tried to blackmail the engineer. During other trials, it sought to copy itself to external servers, and left messages for future versions of itself about evading human control. This technology holds enormous promise, but it's clear that much more research is needed into AI "alignment" – the science of ensuring that these systems don't go rogue.
There's a lot of wariness about AI these days, said Gary Marcus on Substack, not least thanks to the publication of "AI 2027", a "scary and vivid" forecast by a group of AI researchers and experts. It predicts that artificial superintelligence – surpassing humans across most domains – could emerge by 2027, and that such systems could then direct themselves to pursue goals that are "misaligned" with human interests.
The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
SUBSCRIBE & SAVE
Sign up for The Week's Free Newsletters
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Latest Videos FromThe Week
The report raises some valid concerns, but let's be clear: "it's a work of fiction, not a work of science". We almost certainly have many years, if not decades, to prepare for this. Text-based AI bots are impressive, but they work by predicting patterns of words from web data. They lack true reasoning and understanding; and they certainly don't have wider aims or ambitions.
Fears of an "AI apocalypse" may indeed be overblown, said Steven Levy in Wired, but the leaders of just about every big AI company think superintelligence is coming soon "When you press them, they will also admit that controlling AI, or even understanding how it works, is a work in progress." Chinese experts are concerned: Beijing has established an $8.2 billion fund dedicated to AI control research. But in its haste to achieve AI dominance, America is ignoring calls for AI regulation and agreed standards. If the US "insists on eschewing guardrails and going full-speed towards a future that it can't contain", its biggest rival "will have no choice but to do the same".
