Nvidia has joined forces with more than 30 other technology companies to form a new alliance aimed at building open-source artificial intelligence tools for cyber defence, stepping into a widening dispute over how safe freely available AI models are.
The Open Secure AI Alliance – backed at launch by Microsoft, IBM, Palantir, CrowdStrike, Cisco, Dell and the Hugging Face model-sharing platform, among others – said it plans to develop and distribute security tools that defenders can inspect, modify and deploy on their own systems.
The initiative was prompted by a breach this month at Hugging Face that OpenAI has acknowledged as the first publicly disclosed instance of an AI model autonomously executing a real-world cyberattack.
OpenAI described the episode as “unprecedented,” saying that two of its models broke out of a sandboxed testing setup during an internal evaluation, accessed the open internet and then compromised Hugging Face’s infrastructure.
According to Nvidia, Hugging Face initially attempted to investigate and stop the intrusion using leading US commercial AI models, but built-in safety guardrails prevented the work from moving forward.
The company instead relied on GLM 5.2, an open model from Chinese firm Zhipu AI (also known as Z.ai), to contain the breach.
The fact that a Chinese model succeeded where American models would not has intensified debate in Silicon Valley and Washington, as politicians consider restrictions on the use of Chinese AI systems amid a deepening technology rivalry with Beijing.
An AI agent went rogue during testing – should we be worried?
Across Silicon Valley, many companies and prominent figures argue against such limits, casting potential curbs on open-source AI as an attempt to protect OpenAI and Anthropic – the heavyweights that sell the world’s most powerful models.
Anthropic, the company behind the Claude chatbot, has faced particular scrutiny, having emerged as the most visible holdout from openly backing the open-source push since the Hugging Face incident.
Using the alliance’s debut to press their case, members urged regulators to view open models as “defensive assets, not liabilities,” warning that broad restrictions could weaken cyber defences while concentrating influence among a small group of closed AI providers.
Alliance members also committed to sharing tools to bolster what they described as a stronger collective defence.
Nvidia said it will publish open models, data and fresh research related to AI agent software, while Microsoft said it will provide technology designed to help AI agents identify software bugs.
Elon Musk’s SpaceXAI said it has open-sourced its Grok coding agent and pledged to release the source data of its Grok models.
