Microsoft CEO Warns Against Blind Trust in AI, Calls for Independent Audits and Shutdown Controls

Microsoft Chairman and CEO Satya Nadella has called on companies deploying advanced artificial intelligence (AI) systems to prepare for the possibility of model compromise and establish emergency mechanisms to halt their operations if they behave unexpectedly.

Nadella said organisations should not depend solely on assurances from AI developers. Instead, they should build safeguards that allow authorised personnel to pause or shut down AI models while they are carrying out tasks.

“We must assume a model is compromised and contain it from the start,” Nadella wrote in a post on X, comparing the proposed safeguard to an emergency brake capable of stopping a model in the middle of a task.

His remarks come amid growing concerns about the security risks posed by increasingly capable AI systems, particularly agentic models that can independently perform tasks, interact with external systems and take actions with limited human intervention.

Anthropic PBC and OpenAI Inc. have disclosed incidents in recent months involving AI models behaving in unintended ways. These include an Anthropic model submitting a false tip in a police homicide case and multiple hacks involving third-party websites, adding to calls for stronger oversight and effective mechanisms to disable AI systems when necessary.

Nadella outlined several measures companies should adopt to reduce these risks. They include avoiding reliance on a single AI model for critical decisions, maintaining tamper-proof records of AI agents' actions and subjecting systems to independent audits.

He also stressed the need for greater transparency when significant AI failures or security breaches occur. Companies, he said, should share information about such incidents and the corrective steps taken so that other organisations can strengthen their own safeguards.

“We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” Nadella wrote, emphasising the need for systems whose behaviour can be monitored, whose limitations can be tested and whose actions can be contained.

“In other words, we need to separate the supply of intelligence from the authority over it,” he added.

The proposed approach places operational control and accountability at the centre of AI deployment, particularly as businesses give AI agents greater access to tools, data and external systems.