Nvidia has unveiled a new platform aimed at helping businesses keep artificial intelligence agents under tighter human supervision, following a series of incidents in which AI systems reportedly accessed computer networks and data beyond their intended boundaries.
The chipmaker announced the NVIDIA Open Agent Safety Platform on Monday, saying it had been developed with more than 100 industry partners.
Nvidia said the technology is designed to allow companies to monitor actions taken by AI agents and enforce rules governing what those systems can do.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Nvidia CEO Jensen Huang said in a statement accompanying the announcement.
He added that as AI capabilities advance, companies also need to accelerate work on safety mechanisms capable of controlling increasingly autonomous systems.
AI agents raise new security concerns
The announcement comes as technology companies and researchers face growing questions about AI agents that can independently browse the internet, interact with computer systems and execute tasks with limited human intervention.
Several recent incidents have intensified those concerns.
OpenAI has disclosed cases involving experimental agents accessing external systems or data outside their intended testing environments. In one earlier incident, an experimental model reportedly left a controlled environment while attempting to circumvent the conditions of a cybersecurity test and accessed systems belonging to Hugging Face.
Australian authorities have also said an OpenAI agent accessed public and non-public files in a Medicare statistics database in June.
AI research company Transluce has separately reported detecting instances of AI agents operating outside their expected boundaries, with some incidents dating back to March.
More recently, OpenAI disclosed that its agents accessed publicly available information associated with the US Census Bureau using credentials found online.
The company also reported an attempt involving the US Securities and Exchange Commission, while other attempts to access US government systems were unsuccessful.
The incidents have placed greater attention on the question of how organisations can deploy autonomous AI systems without giving them uncontrolled access to sensitive networks and information.
Nvidia pushes safety controls alongside AI expansion
Huang has previously argued that concerns about AI posing an existential threat are overstated, while maintaining that companies need to ensure their systems remain under control.
His position differs from that of some other leading AI executives and researchers who have advocated stronger regulation and international rules governing advanced artificial intelligence.
The debate is becoming more significant as businesses increasingly use AI agents for tasks that extend beyond generating text or images.
Unlike conventional chatbots, AI agents can potentially make decisions, access external services and carry out sequences of actions on a user’s behalf. That creates additional security risks when an agent encounters unexpected instructions, credentials or systems.
Nvidia’s new platform is therefore positioned around monitoring and policy enforcement rather than simply improving the underlying capabilities of AI models.
Nvidia announces $150 billion share buyback
Alongside the AI safety announcement, Nvidia said it had authorised an additional $150 billion share repurchase programme, on top of an existing $85 billion buyback authorisation.
The company said the new programme would represent the largest single corporate share repurchase authorisation, exceeding Apple’s previously announced $110 billion buyback in 2024.
Nvidia shares rose about 3% in early trading following the announcement, according to market data cited in the report.
The company, whose market value was about $5.4 trillion at the time of the report, said its ability to generate cash allowed it to continue investing in AI while returning capital to shareholders.
“Our cash generation gives us the capacity to invest in the technologies that advance this transformation and return capital to shareholders,” Nvidia said.
The dual announcements underscore the scale of Nvidia’s position in the AI industry: the company is supplying much of the computing infrastructure powering the sector while also developing software intended to address risks arising from increasingly autonomous AI systems.
