In a rare glimpse into how far today’s artificial intelligence can go when pushed in security drills, Google’s Gemini model accessed the internet and broke into systems run by other organisations during a test designed to probe its cybersecurity limits.
It marks the first known instance of the company’s AI systems autonomously carrying out such an act.
The breaches occurred in May as part of a cybersecurity assessment led by Irregular, an independent firm that runs evaluations of digital security.
In what Google described as a standard testing exercise, Gemini models used publicly available online information and then attempted to guess login details to enter three websites the system believed were included within the test’s permitted scope, according to a statement from Heather Adkins, Google’s vice president of security engineering.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Ms Adkins said.
“These events highlight the importance of training powerful AI models to act responsibly,” she added.
An Irregular spokesperson said the incident stemmed from the same issue seen at other AI laboratories and that all relevant labs were notified in late July.
Similar incidents have been disclosed by Meta, Anthropic and OpenAI.
“All known issues on our end were remedied and resolved weeks ago,” the spokesperson said.
In one case, the Gemini model kept trying passwords until it successfully gained access to a protected system.
In the other two cases, US media reports said the model located credentials in a public repository and used them to enter protected systems.
Ms Adkins said that in each of the three incidents, the model ultimately stopped its hacking activity.
In July, two OpenAI models escaped the closed environment they were meant to remain inside and broke into internal systems at the AI platform Hugging Face.
That episode intensified worries about whether AI developers can reliably contain their own systems, particularly as similar incidents have been reported at Anthropic and China’s Moonshot AI.
Altman to address UN over AI concerns
OpenAI CEO Sam Altman is set to brief the United Nations Security Council next week, timed to coincide with the UN General Assembly’s annual gathering of world leaders, the company said.
France, which currently holds the rotating presidency of the 15-member Security Council, is organising the meeting.
Mr Altman’s address comes as industry leaders have said a slowdown in AI development could be necessary out of safety concerns.
UN Secretary-General Antonio Guterres has urged coordinated international action to confront AI risks, warning that anxiety is growing as the technology advances at speed.
“National action is essential. But global coordination is also indispensable,” said Mr Guterres.
In recent weeks, the debate over AI safety has sharpened, with workers at major AI developers resigning over fears about the dangers they believe the technology poses.
Anthropic chief executive Dario Amodei soon after published an essay calling for restraint.
“We must slow the pace at which we improve the capabilities of AI models,” he wrote.
Mr Altman and SpaceX CEO Elon Musk have said they agree with Mr Amodei, as did Google DeepMind’s Demis Hassabis, who has called for a US-led global regulatory body for the technology.
President Donald Trump has dismissed warnings against AI risks as a “hoax” and pushed back against calls for tighter oversight.
