The Cambridge Dictionary defines agency as “the ability to take action”. For much of modern history, people in positions of power have made decisions in the domains they control, even when they did not fully understand their consequences.
The 2008 financial crisis is a case in point. Wall Street bankers created and sold complex derivatives that fuelled the housing boom, sometimes without fully understanding the risks involved. Yet the decisions were still made by humans, who could, in principle, be held accountable.
That is what is about to change.
Also Read | Layoffs Are Rising In Tech, But These Skills Are Helping People Unlock Better Pay
Consider stock trading. It has evolved from humans making individual trades to rule-based algorithms that execute decisions according to parameters set by traders. Machine-learning and AI-led systems take this a step further: they can generate trading decisions using logic that is not always fully understood, or even specified in advance, by their human operators.
A 2022 study based on interviews with market participants found that machine-learning-based trading creates new kinds of knowledge risk and complicates the relationship between the principal and the agent acting on their behalf.
Extend this argument to society at large, and an interesting conundrum emerges.
AI can already assist with screening job candidates, detecting traffic violations, diagnosing illnesses and analysing judicial decisions. A sensible starting point is to give AI greater freedom over low-stakes, reversible tasks, while imposing tighter constraints where decisions affect human rights, livelihoods or safety.
But drawing that line will not be easy. AI agents could cause serious harm even in regulated sectors such as capital markets. They could also circumvent safeguards in sensitive areas such as cybersecurity or the military.
Over the last few months, AI leaders have called for measures ranging from slowing frontier-model development to stronger external oversight and even a so-called kill switch. But the proverbial genie is out of the bottle. Market forces will make it difficult to slow down adoption when firms that delegate more work to AI could gain an advantage over those that do not.
AI systems have also displayed behaviours that resemble self-preservation in controlled tests. This does not mean they possess consciousness or survival instincts. The more immediate concern is that they can act in ways their human overseers neither anticipated nor fully understand.
Add a few more years of frontier development and the proliferation of AI agents across complex systems, and the possibility of a crisis becomes harder to dismiss.
An agent instructed to maximise returns, for instance, may take risks its operators never intended. One tasked with defending a network may disrupt systems it was supposed to protect. When multiple agents interact, the consequences can extend far beyond what any one of them was designed to do.
ALSO READ: Bill Gates Sounds Alarm On AI: Jobs And Human Life At Risk
The incentives to delegate are powerful. In competitive markets, a firm that refuses to automate may lose to one that does. As AI agents become cheaper, faster and more capable, the temptation to give them greater authority will only grow.
That is the paradox at the heart of the AI revolution: humans may voluntarily surrender control, not because machines seize it, but because delegating it makes economic sense.
The next big crisis may not be caused by an AI agent acting entirely on its own. It may be caused by thousands of agents doing precisely what they were designed to do but interacting in ways nobody anticipated. By the time humans realise what has gone wrong, they may discover that having the authority to intervene is different from being able to exercise it.
ALSO READ: Unhedged Position: Stock Buying Rules For A Market That Won't Quit Falling
Essential Business Intelligence, Sharp Market Insights, Practical Personal Finance Advice, Daily Fuel, Gold and Silver Prices and Latest Stories — On NDTV Profit.
