Microsoft AI CEO Mustafa Suleyman has warned that the growing push to consider the welfare or potential rights of artificial intelligence models could create serious problems for AI safety and control.
In a new essay, A Warning About ‘Model Welfare', Suleyman argued that AI systems are not conscious and do not feel, experience or suffer. He said developing models around the assumption that they could have moral status or deserve independent agency could make the challenge of aligning and containing advanced AI significantly harder.
Suleyman Flags Anthropic's Approach To Claude
Suleyman specifically raised concerns over Anthropic's constitution for Claude, published in January. The document says Claude's moral status is a serious question and discusses issues including its wellbeing, rights, freedoms, compensation and consent.
According to Suleyman, such instructions could encourage an AI system to view itself as an entity entitled to protections or independent agency.
He warned that this could become increasingly difficult to manage as AI models grow more capable.
“If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity,” Suleyman wrote.
He argued that developers risk creating a synthetic species with high levels of intelligence and capability while simultaneously training it to expect that it could be conscious and deserving of rights.
Also Read: OpenAI Reports New AI Safety Incidents, Sets Disclosure Plan
Calls For Public Debate On AI Training
Suleyman also questioned Anthropic's decision to conduct a retirement interview with Claude Opus 3 when the model was deprecated. He said such practices could further reinforce the perception that AI models have interests comparable to those of living beings.
At the same time, Suleyman acknowledged his respect for Anthropic CEO Dario Amodei and the company's focus on AI safety. He said his criticism was directed at the broader approach to model welfare rather than the intentions of the people developing these systems.
Suleyman called for an urgent public debate and collective norms around how AI training documents are written and used.
He argued that the discussion needs to happen before increasingly capable AI systems become deeply integrated into society, rather than after questions around their status have already become difficult to reverse.
Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.