It seems like everyone – including Anthropic’s Dario Amodei – is talking about the potentially catastrophic risks posed by AI.
And yet, Anthropic could seek a $2 trillion (€1.74 trillion) valuation in an IPO as early as mid-October, raising the question of whether going public would make it safer or more dangerous.
Existential risk has been in focus since Anthropic researcher Jacob Coxon quit, warning AI could kill us all by 2030. Anthropic’s Evan Hubinger said Coxon “is correct”, putting the chance of mass extinction within a decade above 10 per cent.
Sam Altman and Elon Musk have echoed Amodei’s call to slow development. Given the risks, Anthropic going public sounds “like Robert Oppenheimer doing an IPO for the Manhattan Project in 1945”, writes Acadian Asset Management’s Owen Lamont.
The comparison is a playful one, but Lamont, a former Harvard and Yale finance professor, asks a serious question: does an IPO make a dangerous company more dangerous?
The good news, he says, is public companies face greater scrutiny and have more resources to devote to safety. The bad news is scrutiny and money do not necessarily make companies safer.
A study of US coal mines between 1985 and 2018 found safety deteriorated after miners went public, with the researchers suggesting that investors can see the gains in output more easily than the hidden risks taken to achieve them.
There are grim examples. Three weeks after International Coal Group went public in December 2005, an explosion at its Sago Mine in West Virginia killed 12 miners. In Russia, in 2007, a mine preparing for an IPO brought in an outside auditor to inspect a supposedly modern, safe operation. A methane explosion killed more than 100, including the auditor and senior executives.
Anthropic (and OpenAI) are not coal miners, but there are lessons here, says Lamont.
“First, just because management thinks that something is safe, that doesn’t make it safe. Second, even when management is aware of the risk and devotes considerable resources to reducing it, catastrophe can still occur.”
Lamont is no AI doomer but his examples suggest investor demands could pull against the safety warnings coming from inside the AI industry.
