Sam Altman’s Second Coming Sparks New Fears of the AI Apocalypse
Open AI’s new boss is the same as the old boss. But the company—and the artificial intelligence industry—may have been profoundly changed by the past five days of high-stakes soap opera. Sam Altman, OpenAI’s CEO, cofounder, and figurehead, was removed by the board of directors on Friday. By Tuesday night, after a mass protest by the majority of the startup’s staff, Altman was on his way back, and most of the existing board was gone. But that board, mostly independent of OpenAI’s operations, bound to a “for the good of humanity” mission statement, was critical to the company’s uniqueness.
As Altman toured the world in 2023, warning the media and governments about the existential dangers of the technology that he himself was building, he portrayed OpenAI’s unusual for-profit-within-a-nonprofit structure as a firebreak against the irresponsible development of powerful AI. Whatever Altman did with Microsoft’s billions, the board could keep him and other company leaders in check. If he started acting dangerously or against the interests of humanity, in the board’s view, the group could eject him. “The board can fire me, I think that’s important,” Altman told Bloomberg in June.
“It turns out that they couldn’t fire him, and that was bad,” says Toby Ord, senior research fellow in philosophy at Oxford University, and a prominent voice among people who warn AI could pose an existential risk to humanity.
The chaotic leadership reset at OpenAI ended with the board being reshuffled to consist of establishment figures in tech and former US secretary of the treasury Larry Summers. Two directors associated with the “effective altruism” movement, the only women, were removed from the board. It has crystallized existing divides over how the future of AI should be governed. The outcome is seen very differently by doomers who worry that AI is going to destroy humanity; transhumanists who think the tech will hasten a utopian future; those who believe in freewheeling market capitalism; and advocates of tight regulation to contain tech giants that cannot be trusted to balance the potential harms of powerfully disruptive technology with a desire to make money.
“To some extent, this was a collision course that had been set for a long time,” says Ord, who is also credited with cofounding the effective altruism movement, parts of which have become obsessed with the doomier end of the AI risk spectrum. “If it’s the case that the nonprofit governance board of OpenAI was fundamentally powerless to actually affect its behavior, then I think that exposing that it was powerless was probably a good thing.”
Governance Gap
The reason that OpenAI’s board decided to move against Altman remains a mystery. Its announcement that Altman was out of the CEO seat said he “was not consistently candid in his communications with the board, hindering its ability to exercise its responsibilities.” An internal OpenAI memo later clarified that Altman’s ejection “was not made in response to malfeasance.” Emmett Shear, the second of two interim CEOs to run the company between Friday night and Wednesday morning, wrote after accepting the role that he’d asked why Altman was removed. “The board did not remove Sam over any specific disagreement on safety,” he wrote. “Their reasoning was completely different from that.” He pledged to launch an investigation into the reasons for Altman’s dismissal.
The vacuum has left space for rumors, including that Altman was devoting too much time to side projects or was too deferential to Microsoft. It has also nurtured conspiracy theories, like the idea that OpenAI had created artificial general intelligence (AGI), and the board had flipped the kill switch on the advice of chief scientist, cofounder, and board member Ilya Sutskever.
“What I know with certainty is we don’t have AGI,” says David Shrier, professor of practice, AI, and innovation, at Imperial College Business School in London. “I know with certainty there was a colossal failure of governance.”
Shrier, who has sat on several tech company boards, says that failure isn’t just because of the obvious tension between the board’s nonprofit mission and the commercial desires of the executives and investors involved in the for-profit unit of OpenAI. It’s also a function of the company’s rapid growth in size and influence, reflective of the AI industry’s growing clout. “ChatGPT took six weeks to go from zero to 100 million users. The world wide web took seven years to get to that kind of scale,” he says. “Seven years is enough time for the human brain to catch up with technology. Six weeks, that’s barely enough time to schedule a board meeting.”
Despite the board’s supreme power on paper, the complexity and scale of OpenAI’s operations “clearly outstripped” the directors’ ability to oversee the company, Shrier says. He considers that alarming, given the real and immediate need to get a handle on the risks of AI technology. Ventures like OpenAI “are certainly not science projects. They’re no longer even just software companies,” he says. “These are global enterprises that have a significant impact on how we think, how we vote, how we run our companies, how we interact with each other. And as such, you need a mature and robust governance mechanism in place.”
Test Case
Regulators around the world will be watching what happens next at OpenAI carefully. As Altman negotiated to return to OpenAI on Tuesday, the US Federal Trade Commission voted to give staff at the regulator powers to investigate companies selling AI-powered services, allowing them to legally compel documents, testimony, and other evidence.
The company’s boardroom drama also unfolded at a pivotal point in negotiations over the European Union’s landmark AI Act—a piece of legislation that could set the tone for regulations around the world. Bruised by previous failures to mitigate the social impacts of technology platforms, the EU has increasingly taken a more muscular approach to regulating Big Tech. However, EU officials and member states have disagreed over whether to come down hard on AI companies or to allow a degree of self-regulation.
One of the main sticking points in the EU negotiations is whether makers of so-called foundation models, like OpenAI’s GPT-4, should be regulated or whether legislation should focus on the applications that foundational models are used to create. The argument for singling out foundation models is that, as AI systems with many different capabilities, they will come to underpin many different applications on top, in the way that GPT-4 powers OpenAI’s chatbot ChatGPT.
This week, France, Germany, and Italy said they supported “mandatory self-regulation through codes of conduct” for foundation models, according to a joint paper first reported by Reuters—effectively suggesting that OpenAI and others can be trusted to keep their own technology in check. France and Germany are home to two of Europe’s leading foundation model makers, Mistral and Aleph Alpha. On X, Mistral CEO Arthur Mensch