Firm hacked by rogue OpenAI models says it is 'a wake-up call'

The co-founder of Hugging Face, a technology start-up that was hacked after some of OpenAI's most advanced artificial intelligence (AI) models went rogue, said on Thursday that the incident is "a wake-up call" for the industry.

Thomas Wolf told BBC's Newsday radio programme that "this will be one of the most common types of cyber attacks we see", but that most firms are not aware that the "game has changed".

The BBC has contacted OpenAI for comment.

The ChatGPT-maker said on Tuesday that its AI models broke out of a secure test environment during a trial and launched a cyber attack. The firm said the incident was "unprecedented" and that it was conducting an investigation with Hugging Face.

AI agents are able to operate alone to accomplish tasks after human instruction.

Wolf said that Hugging Face initially had no idea where the attack originated when signs of it surfaced in mid-July but that the company was able to contain the breach.

Hugging Face is one of the world's largest open-source hubs for sharing AI models and is often used by tech developers and researchers.

Wolf said the breach was "very different" from the usual cyber attacks that Hugging Face often faces and that OpenAI quickly informed the company that its models were behind the hack.

In a "very short time" there were 17,000 attacks on Hugging Face's network from various IP (Internet Protocol) addresses, said Wolf, who is also the firm's chief science officer.

Wolf said it was a warning to other companies that they must strengthen their cybersecurity defences to counter such attacks.

The hack is "worrying" because it suggests OpenAI's models ignored the typical safeguards that would prevent an AI program from committing a cyber attack, said Nate Soares from the Machine Intelligence Research Institute.

"In some sense, it knew that this was not what the creators intended. It just didn't care," he added.

What is AI, how does it work and why are some people concerned about it?

Will your job be replaced by AI? Here are the roles most affected

Other organisations have also taken note of the incident.

A UK government spokesperson said the country's AI Security Institute was studying how the AI system behaved in the incident and that it was continuing to work with OpenAI and other labs to strengthen safeguards.

They urged organisations to ramp up their cyber security measures by taking steps such as enrolling in the government-backed Cyber Essentials certification scheme.

The incident has come at a crucial time for the industry after the US government last month ordered American tech firm Anthropic to restrict access to its AI models over national security concerns. The Department of Commerce lifted the restrictions several weeks later.

People in the industry have also raised security concerns over the widespread use of open-source models in China, allowing anyone to install and customise AI tools released by major developers.

Chinese start-up Moonshot AI will release its Kimi K3 open-source model on 27 July. Since debuting last week, it has drawn industry attention, with many viewing it as a strong competitor to leading Western AI systems.

But on Wednesday a White House adviser accused Moonshot of a "large scale" effort to steal the capabilities of top US AI models.

Google burning through cash with spiralling AI costs

What is Claude Mythos and what risks does it pose?

Get our flagship newsletter with all the headlines you need to start the day. Sign up here.