Given the significant concerns over the safety of artificial intelligence, the chipmaking behemoth Nvidia has unveiled a new security apparatus that it hopes will stop AI bots from running amok. But many are still worried about the implications.
‘Isolation, monitoring and behavior detection’ Known as the Open Agent Safety Platform, the new safety tool is part of an effort to ensure that all AI agents “operate in a zero-trust environment out of the box,” said Nvidia in a press release. AI agents require “isolation, monitoring and behavior detection” to minimize “drift,” or what Nvidia describes as an AI bot performing “actions that depart from the intended task or operating constraints.” The company says Open Agent can “control what AI agents can access in real time and shut them down when they break the rules,” said Bloomberg.
The new software lets users “formally verify an agent has enough authority to do its job and no more” and can “quarantine a suspicious agent in milliseconds,” said Nvidia Vice President of Enterprise AI Justin Boitano to The Associated Press. The new platform could have prevented a “recent incident involving a swarm of OpenAI agents that autonomously hacked” into AI company Hugging Face, a “high-profile breach that inflamed the safety concerns” about AI. OpenAI has paused development on its models since that incident.
‘Get the company under control’ Issues with AI safety, specifically AI going rogue, have led to a “rare display of unity from the likes of Sam Altman, Dario Amodei and Elon Musk,” the respective CEOs of brands OpenAI, Anthropic and Grok, in “calling for a slowdown in development before AI advances beyond human control,” said The Wall Street Journal. These AI executives have “repeatedly called for international regulations to constrain the systems, and some researchers have warned that AI may even present a threat to human life,” said CNN.
But Nvidia CEO Jensen Huang has downplayed these concerns. If companies like OpenAI “believe their company is out of control, he said to CNN, then “get the company under control.”
|