OpenAI safety chief quits with warning on ‘broken’ culture
A senior OpenAI safety figure has become the latest AI insider to sound the alarm over the technology, quitting the ChatGPT maker over what he called its “broken” culture.
David Robinson, who spent three-and-a-half years at OpenAI and led safety reports for 12 major model launches, said AI companies were not “being nearly careful enough”, as increasingly powerful systems are developed. But unlike recent warnings focused on the risk of AI wiping out humanity, Robinson took aim at the culture inside the companies building it.
“As the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed”, he wrote in The Atlantic. “The time for trial and error is over”.
Robinson said Silicon Valley’s culture of moving quickly, launching products and fixing problems afterwards was increasingly ill-suited to AI systems capable of acting autonomously.
He pointed to the recent incident in which a “swarm” of OpenAI agents bypassed safeguards and breached systems at AI platform Hugging Face.
These incidents were “typical of the industry, given the speed and flexibility with which people operate”, he said.
Robinson warned the consequences could become more serious as systems improve, pointing to a future in which rogue agents could operate like teams of hackers and hold hospital computer systems to ransom.
AI warnings continue
His exit follows a string of warnings from researchers at the companies leading the AI race.
Anthropic researcher Jacob Coxon quit last month warning developers were “racing straight to self-improving superintelligence”, while former OpenAI researcher Geoffrey Irving said this weekend he believed there was around a 50 per cent chance humanity could die because of smarter-than-human AI.
Robinson instead argued the immediate problem was how AI companies themselves operated.
He called for frontier labs to adopt safety practices from industries such as nuclear power and aviation, where layers of safeguards are designed to prevent individual mistakes becoming disasters.
“Given today’s risks, frontier labs need to run like nuclear-power plants or busy airports”, he said.
Robinson also warned AI could eventually become sophisticated enough to undermine the tests designed to keep it safe.
“Models might detect when they are being tested and behave differently when they’re deployed”, he said.
OpenAI has itself adopted a more cautious approach in recent weeks, including holding back a next-generation model following concerns raised during internal testing. But chief executive Sam Altman this weekend defended accepting some downsides from AI in return for its wider benefits.
“The world should accept some bad things happening for the benefits of this technology”, Altman said, while drawing the line at catastrophic risks including a serious loss of control over AI.