Jakub Pachocki

I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands. An Alien Mind: https://openai.com/index/an-alien-mind/

I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4. OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.

We temporarily slowed some frontier training to strengthen security and monitoring. Our largest planned frontier RL run remains on hold while smaller-scale training and evaluations help us test safeguards and gather more evidence of alignment. I expect confidence in safety to increasingly set the pace of AI development. We urgently need tools for labs and countries to coordinate on this, which is why I signed Pacing the Frontier. In the meantime, we’re taking practical steps ourselves - and will continue to share what we learn as our approach evolves. https://openai.com/index/pacing-model-development-cyber-capabilities/ https://www.pacingthefrontier.com/

❤️

Very grateful that Paul is joining the OpenAI Foundation board. We have a lot to do.

As AI becomes more capable, I'm focused on questions around the pace and implications of progress, and how they should inform the strategy of OpenAI and across the field. I asked Aleksander to come back to OpenAI to work with me on these topics. I value his judgment and am glad to have him on the team again.