One of OpenAI’s safety leaders quit on Tuesday. He just explained why.

OpenAI is famously not all that open. Dazzling, cutting-edge AI products emerge without warning, generating excitement and anxiety in equal measure (along with plenty of disdain). But like its product development, the company’s internal culture is unusually opaque, which makes it all the more unsettling that Jan Leike, the departing co-head of its “superalignment” team — a position overlooking OpenAI’s safety issues — has just spoken out against the company.

Tweet may have been deleted

Something like this was partly anticipated by those watching OpenAI closely. The company’s high-profile former chief scientist, Ilya Sutskever, abruptly quit on Tuesday, too, and “#WhatDidIlyaSee” became a trending hashtag once again. The presumptuous phrasing of the hashtag — originally from March, when Sutskever participated in the corporate machinations that got CEO Sam Altman briefly fired — made it sound as if Sutskever had glimpsed the world through the AI looking glass, and had run screaming from it. 

In a series of posts on X (formerly Twitter) on Friday, Leike gave the public some hints as to why he left.

He claimed he had been “disagreeing with OpenAI leadership about the company’s core priorities for quite some time,” and that he had reached a “breaking point.” He thinks the company should be more focused on “security, monitoring, preparedness, safety, adversarial robustness, (super)alignment, confidentiality, societal impact, and related topics.”

Tweet may have been deleted

“These problems are quite hard to get right, and I am concerned we aren’t on a trajectory to get there,” Leike said, noting that he felt like he and his team were “sailing against the wind” when they tried to secure the resources they needed to do their safety work. 

Tweet may have been deleted

Leike seems to view OpenAI as bearing immense responsibility, writing, “Building smarter-than-human machines is an inherently dangerous endeavor.” That makes it potentially all the scarier that, in Leike’s view, “over the past years, safety culture and processes have taken a backseat to shiny products.”

Leike evidently takes seriously the company’s internal narrative about working toward artificial general intelligence, also known as AGI — systems that truly process information like humans, well beyond narrow LLM-like capabilities. “We are long overdue in getting incredibly serious about the implications of AGI,” Leike wrote. “We must prioritize preparing for them as best we can. Only then can we ensure AGI benefits all of humanity.”

In Leike’s view, OpenAI needs to “become a safety-first AGI company” and he urged its remaining employees to “act with the gravitas appropriate for what you’re building.”

This departure, not to mention these comments, will only add fuel to already widespread public apprehensiveness around OpenAI’s commitment, or lack thereof, to AI safety. Other critics, however, have pointed out that fearmongering around AI’s supposedly immense power also functions as a kind of backdoor marketing scheme for this still largely unproven technology.