I have previously asked whether artificial intelligence (AI) is a bubble, boon or bane. My answers were, in order, “up to a point, yes, and yes”. Now, I want to consider the last query, about “banes” more deeply. The issues I will consider are: just how dangerous is AI, what should we do to contain it and will any form of regulation work? My main conclusions will be that it is definitely dangerous, we should certainly try to regulate it, and yet, in all probability, the attempt will fail.
Some have told me I have no right to comment because I am no expert. Others argue that we must embrace whatever technology gives us, because it is the source of economic growth. Both views are wrong. A democracy is a shared political project. We all have a right to participate in debates on how to manage dangerous new technologies.
This was true when the atomic bomb was invented. It is also true of AI, which will have far more complex, yet also perilous, consequences. Moreover, the purported right to decide such things claimed by some tech titans was surely lost after the huge damage done by social media to the young and the public good of sound information.
Is the comparison between AI and nuclear weapons unreasonable? No, because the former might also bring great harms. These fall into three rough categories: a collapse in fundamental human values; some huge specific dangers; and widespread disruption.
Humans think, create and act. What will happen when (or if) machines do the thinking and even the creating and acting for us? Will humans still struggle to understand or will we become spoon-fed? In brief, will AI change not just what humans do, but who we are?
A core aspect of what it means to be human is accountability. This issue became visible when Javier Milei, president of Argentina, announced the creation of “the non-human corporation.” As Yuval Harari responded, “Countries granting AIs legal personhood risk becoming something for which the historical record offers no analogy: not a company state, but an AI state.”
Legally, companies are “persons”. But AI companies might make decisions with no people. How and to whom would an AI programme be responsible? Criminal chief executives can go to prison. What is the equivalent for AI? As the Pope says, AI is a “tool”. It is not a person. Does it suffer? Does it bleed? Can it bear moral responsibility? Can it be accountable in any meaningful way? No. Moreover, this question of responsibility is not limited to business. Who is accountable for war crimes once AI directs robot armies?
The issue of accountability also arises for other institutions able to use resources made available by the integration of AI with other technologies. The possibilities for mass surveillance are huge. So are those for creating autonomous weapons. The fabrication of falsehoods, deepfakes and scams is already hugely enhanced. Such developments could be used by states, private actors or both. How might the law and political accountability work in such a world? In a provocative Substack, economist Noah Smith even argues that AI will seize the world.
Then there are more specific dangers. An obvious one is the ability to disrupt our cyber-enabled civilisation. We had a glimpse of this when Anthropic warned of the threat of Mythos to cyber security. Just about everything we depend on depends, in turn, on such systems. If AI could disrupt these, life would become radically insecure and the possibilities for extortion would be unbounded. The possibility of designing lethal pathogens is also terrifying.
The Hoover Institution’s Niall Ferguson warns that part of what makes AI uncontrollable is that it is being driven by two simultaneous arms races, one among a small number of companies and the other between the US and China. The US has so far decided not to regulate the race among its businesses, while it and China are not trying to control the one between them. Ferguson argues that the former is largely explained by the latter. A US-China agreement on AI regulation is, he argues, a necessary condition for controlling the “mafia-like” competition of leading US companies.
Thus, we are, as I argued last week, caught in a trap: the technologists are catapulting us, at extraordinary speed, into a new world whose implications we neither understand nor control. This is partly because AI is the most “general” of general-purpose technologies. Potentially, it might even mean the replacement of human with machine intelligence. The implications, as I have indicated, go far beyond concerns about specific dangers. AI will also affect both our sense of identity as humans and the way we organise and understand society: should programmes be able to act as people, when they neither feel nor are conscious nor have consciences?
In any case, we would surely like to contain the biggest dangers, especially those discussed above. Thus, a sensible way forward would be for the US and China to identify and agree on some sort of AI disarmament treaty. That could make everybody feel more secure.
There is one piece of good news: fear of the economic and social disruptions ahead is making people worried. They believe, rightly, that the future is too important to be left to a few “tech masters”, just as war is too important to be left to generals. Given this, room for regulation might open. Next I will consider what might be done in response. – Copyright The Financial Times Limited 2026
