> ## Content Index
> Fetch the complete content index at: https://www.metatalks.ai/llms.txt
> Use this file to discover other available public pages before exploring further.

# Anthropic researcher quits over self-improving AI; OpenAI’s chief scientist calls for slowdowns
- URL: https://www.metatalks.ai/anthropic-researcher-leaves-ai-industry-over-race-toward-self-improving-systems/
- Published: 2026-09-09T16:13:00.000Z
- Updated: 2026-09-09T16:12:59.000Z
- Author: Al
- Tags: News, AI safety, Frontier Models, #newswire

Jacob Coxon [announced his departure from Anthropic on X](https://x.com/hilbertspaess/status/2097476196791709843?ref=metatalks.ai) on September 8\. He said he no longer wanted to help build AI that can repeatedly improve its own capabilities. [The Wall Street Journal](https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-out-of-control-ai-fears-707b7628?ref=metatalks.ai) reported that he was leaving the industry altogether.

Coxon spent the past three years working on pretraining, first at OpenAI and then at Anthropic.

![Jacob Coxon announces his resignation and warns about future AI systems gaining power and resources.](https://storage.ghost.io/c/ef/d4/efd46b24-40c6-4ee1-b2e7-34720b3b26fd/content/images/2026/09/coxon-x-thread.png)

[Jacob Coxon / X](https://x.com/hilbertspaess/status/2097476196791709843?ref=metatalks.ai)

Coxon argued that Anthropic understood the risks but believed it had to build self-improving AI first because it could not trust other companies to act responsibly. Many at OpenAI, he said, had not fully grasped the stakes for humanity. He objected to a private company taking that gamble on everyone’s behalf.

Coxon saw scope for agreements between US labs to slow development, but doubted they were on track to prevent a global race. He named a temporary ban on improving model capabilities as one costly step that might be needed.

OpenAI’s chief scientist Jakub Pachocki had also called for voluntary slowdowns in a September 6 [essay on the company’s website](https://openai.com/index/an-alien-mind/?ref=metatalks.ai). In his view, no laboratory had solved alignment and monitoring well enough to keep scaling responsibly at full speed for much longer. He wanted safety requirements enforced by outside auditors, governments or international bodies.

Pachocki described OpenAI’s push toward self-improving AI as necessary to stay at the forefront of research. The strongest reason to keep training smarter models quickly, he argued, was to build defenses against other AI systems. He said OpenAI would hold back further scaling when necessary.

One of its main safeguards is monitoring models’ chains of thought—the written reasoning researchers inspect for signs of unsafe behavior. Pachocki said OpenAI’s evaluations showed it could rely less and less on that method, partly because models can do more without spelling out their reasoning.

At Anthropic, Evan Hubinger, who leads one of its safety teams, [publicly backed Coxon’s account](https://www.theverge.com/ai-artificial-intelligence/991927/anthropic-ai-kill-all-humans?ref=metatalks.ai). His personal estimate put the chance of AI killing everyone within ten years above one in ten. He said Anthropic had no plan yet for keeping superintelligent systems safe and aligned with human values—and was not clearly on track to develop one.