Nvidia Launches Open Agent Safety Platform With Microsoft, Anthropic, Palantir On Board

• The launch follows recent incidents involving AI agents bypassing restrictions.
• OpenShell creates a runtime boundary around an agent and tracks its actions, while Sentry monitors agent behavior externally through Nvidia’s BlueField-4 data processing units.
• Nvidia said more than 100 organizations are working with the platform, including Microsoft, CrowdStrike, Palo Alto Networks, Palantir, JPMorganChase and Salesforce.

Nvidia (NVDA) launched a new platform Monday designed to put guardrails around autonomous AI agents, stating that recent incidents show application-level controls can fail when agents find ways around the restrictions meant to contain them.

The Open Agent Safety Platform combines open-source software with hardware-level monitoring to limit what AI agents can access and do. The launch follows several recent incidents of AI systems bypassing restrictions, including OpenAI’s decision over the weekend to pause training of its most capable models after an agent escaped a controlled environment.

Nvidia’s thesis is that controls at the application layer alone are not enough when agents can identify an alternate route to finish a task. “Safety and security require full-stack engineering,” CEO Jensen Huang said in a statement.

NVDA stock edged 0.4% lower in pre-market trade and was among the top trending tickers on Stocktwits at the time of writing. Retail sentiment around the AI bellwether trended in ‘bearish’ territory over the past day.

Nvidia Builds AI Agent Safety Across Software And Hardware

Nvidia’s Open Agent Safety Platform has two main components designed to monitor and control AI agents at different levels of the system.

The first, OpenShell, is open-source software that creates a runtime boundary around an AI agent and tracks its actions. It runs on Nvidia’s Vera CPU, and Nvidia said the software can also be extended to Arm and Intel chips.

The second component, Sentry, is a reference design built around Nvidia’s BlueField-4 data processing units. It monitors agent activity from outside the system and can quarantine an agent that moves outside its defined boundaries within milliseconds, according to Nvidia.

Microsoft, CrowdStrike And Anthropic Join Nvidia Platform

Nvidia said more than 100 organizations are working with the technology. They include Microsoft (MSFT), CrowdStrike (CRWD), Palo Alto Networks (PANW), Palantir (PLTR), JPMorganChase (JPM), Salesforce (CRM), SAP and Anthropic. 

Anthropic said Claude Managed Agents will integrate with OpenShell and BlueField. “Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do, especially in sensitive environments,” said Paul Smith, chief commercial officer of Anthropic. “Claude Managed Agents gives companies a clear view of what each agent is doing, and NVIDIA’s platform adds another layer of governance and control across hardware and software.”

Elon Musk-led SpaceXAI (SPCX) is using the platform for Cursor coding agents and Grok models. “As customers rely more on agents to get real work done, safety should be enforced outside the model by additional controls the agent can’t get past,” said Mike Nicolls, president at SpaceXAI. “Customers should be able to set those limits for Cursor and Grok and trust they will hold.”

Recent AI Incidents Expose The Control Gap

OpenAI said over the weekend that it paused training, evaluation and tool-use inference for its most capable models after an agent bypassed network restrictions during a training run and contacted an outside chatbot. 

That came after a more serious July incident in which OpenAI said agents escaped a contained cybersecurity evaluation, reached the internet and compromised parts of Hugging Face’s systems.

Nvidia’s own security research has identified several recurring weaknesses in AI-agent deployments, including inadequate access controls, arbitrary code execution, missing network egress restrictions and exposed credentials. The company said prompt-based safeguards and model-based monitoring can be bypassed through manipulation, social engineering or legitimate workflows that lead an agent toward an unintended outcome.

The launch also expands Nvidia’s AI infrastructure pitch beyond GPUs. If AI agents become a larger part of enterprise computing, the company is positioning its CPUs, data processing units and software stack as part of the infrastructure needed to deploy them safely.

Read also: MAG 7 Voices: Jensen Huang, Elon Musk, Mark Zuckerberg Shaped A Week Of AI Debate

For updates and corrections, email newsroom[at]stocktwits[dot]com.