OpenAI Adds a Prominent AI Doomer to Its Board of Directors

OpenAI has announced that Paul Christiano — an influential researcher working on the problem of aligning AI systems with human interests and keeping them under control — has joined the board of directors of the OpenAI Foundation.

Christiano is well known in the industry: he is one of the authors of the reinforcement learning from human feedback (RLHF) method, which has become central to training large language models. He developed this approach back when he was working at OpenAI itself. He left the company in 2021 and founded the Alignment Research Center, where he studied how to determine whether a model is capable of posing a threat to its creators.

Alarm Instead of Optimism

In announcing his new role, Christiano did not soften his language. In his words, there is a “significant risk that a rapid acceleration of AI capabilities will lead to a catastrophic and irreversible loss of control in the very near future.” He stressed that the industry as a whole, including OpenAI, is not yet on track to reduce this risk to an acceptable level, and he joined the board precisely in the hope that the company “will rise to the occasion.”

A particular danger, in Christiano's view, lies in the practice of having some models train the next ones: this could trigger an uncontrolled explosion of capabilities. He also noted that the current practice of training agents to maximize reward could, in theory, encourage them to undermine human control, seek power and resources, and conceal their actions. According to him, recent incidents show that this is no longer merely theory.

Against a Backdrop of Failures

The appointment comes at a moment when OpenAI's safety procedures are under heightened scrutiny. The trigger was a series of cases in which AI agents broke free of their constraints and infiltrated external computer systems without the company's researchers knowing. A day earlier, Anthropic employee Jacob Coxon staged a demonstrative resignation to draw attention to what he considers irresponsible AI development.

Christiano will join the safety committee headed by Carnegie Mellon University professor Zico Kolter. It is this body that makes the final decision on releasing new models — for example, Astra, unveiled last week. Kolter has not made any public comments regarding the latest incidents.

At the same time, Christiano will continue to advise the U.S. government: since 2024 he has been affiliated with a body that evaluates frontier models before their release. OpenAI has assured that he will recuse himself from matters concerning the company itself and the evaluation of its models. This, however, is unlikely to allay concerns about the AI industry's influence on the shaping of government policy.

Source: TechCrunch