Nightshade: the 'data poisoning' tool boosting fightback against AI
Tool is like 'putting hot sauce in your lunch so it doesn't get stolen from the workplace fridge', says creator

A new tool used by creatives to "poison" artificial intelligence models and stop them from using their artwork without consent was downloaded more than a quarter of a million times in just five days.
Nightshade, a free tool created by computer science researchers at the University of Chicago, is the latest "weapon" helping to protect artists' work from becoming "machine learning fodder", said Mashable.
Ben Zhao, a professor of computer science and leader of the project, said he expected there to be "extremely high enthusiasm" but he still "underestimated" how much. In an email to VentureBeat after the number of downloads surpassed 250,000 earlier this month, he said the response was "simply beyond anything we imagined".
Subscribe to The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

Sign up for The Week's Free Newsletters
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
Why is it needed?
The availability of generative artificial intelligence (AI) models that can create text and images which mimic other people's work has caused an "uproar among creatives", reported the Evening Standard.
The sudden emergence and popularity of tools such as DALL-E, Midjourney and Stable Diffusion pose an existential threat to artists, who face an "uphill battle" protecting their work from being used to train AI models without consent, said TechCrunch. "Opt-out requests and do-not-scrape codes rely on AI companies to engage in good faith, but those motivated by profit over privacy can easily disregard such measures," said the tech site.
Some have decided to take legal action against AI giants, such as Meta and OpenAI, claiming copyright infringement, while artificial intelligence was one of the main factors behind last year's long-running Hollywood writers' strike. Actors have protested at the rise of digital replicas, and celebrities have raised concern about their voices being cloned without their consent.
But for the vast majority of artists, going on strike or taking their work offline is not an option as they do not have guaranteed employment and rely on social media exposure for commissions.
How does Nightshade work?
Nightshade, named after the deadly nightshade plant used to poison Roman emperors, "preys upon a security vulnerability inherent within AI models, which allows them to be tampered with", said the Standard. "It does this by adding changes to the pixels in a digital image that are invisible to the human eye. These modifications affect the image itself and the text or captions associated with it, both of which an AI relies on to identify what's in a picture."
Known as a prompt-specific poisoning attack, this can cause AI tools to malfunction and, for example, mistake pictures of cats for dogs or handbags for toasters.
"Unpredictable responses of this sort make text-to-image models significantly less useful", said The Register, "which means model makers have an incentive to ensure that they only train on data that's been offered freely."
Zhao compared it to "putting hot sauce in your lunch so it doesn't get stolen from the workplace fridge".
If applied at scale, it would be "incredibly difficult" for AI companies to fix, said Mashable, "as each poisoned image must be individually identified and removed from their training pool. This could create a powerful incentive for such companies to think twice before dragging a trawl net through the internet and using artists' work without their explicit consent."
According to The Register, the Nightshade team are looking to release Nightshade in combination with an earlier tool, Glaze, which works to prevent AI models from learning an artist's signature "style" by subtly altering pixels.
Will it work?
In a research paper published last October, the team claimed Nightshade "can provide a powerful tool for content owners to protect their intellectual property against model trainers that disregard or ignore copyright notices, do-not-scrape/crawl directives, and opt-out lists".
The aim is not to take down Big AI, said TechCrunch, but "to force tech giants to pay for licensed work, instead of training AI models on scraped images".
"There is a right way of doing this," Zhao agreed. "The real issue here is about consent, is about compensation. We are just giving content creators a way to push back against unauthorised training."
Eva Toorenent, an illustrator, told MIT Technology Review that she hoped Nightshade will change the status quo.
"It is going to make [AI companies] think twice, because they have the possibility of destroying their entire model by taking our work without our consent," she said, while another artist, Autumn Beverly, said the tool "can help return the power back to the artists for their own work".
Sign up for Today's Best Articles in your inbox
A free daily email with the biggest news stories of the day – and the best features from TheWeek.com
-
Having a mayor: Starmer's struggles with devolved leaders
Talking Point Andy Burnham made public criticisms of the Labour government policies without specifically naming Keir Starmer or Rachel Reeves
-
Why is Nasa facing a crisis?
Today's Big Question Trump administration proposes 25% cut to national space agency's budget in 'extinction-level event'
-
The 50-year battle for Western Sahara
The Explainer UK is latest country to back Moroccan plan to end decades-long dispute with Algerian-backed Polisario Front
-
Airplane crash-detection systems could be vulnerable to hackers
Under the Radar 'The idea scares the shit out of me,' one pilot said
-
Google's new AI Mode feature hints at the next era of search
In the Spotlight The search giant is going all in on AI, much to the chagrin of the rest of the web
-
How the AI takeover might affect women more than men
The Explainer The tech boom is a blow to gender equality
-
Did you get a call from a government official? It might be an AI scam.
The Explainer Hackers may be using AI to impersonate senior government officers, said the FBI
-
What Elon Musk's Grok AI controversy reveals about chatbots
In the Spotlight The spread of misinformation is a reminder of how imperfect chatbots really are
-
Is Apple breaking up with Google?
Today's Big Question Google is the default search engine in the Safari browser. The emergence of artificial intelligence could change that.
-
Inside the FDA's plans to embrace AI agencywide
In the Spotlight Rumors are swirling about a bespoke AI chatbot being developed for the FDA by OpenAI
-
Digital consent: Law targets deepfake and revenge porn
Feature The Senate has passed a new bill that will make it a crime to share explicit AI-generated images of minors and adults without consent