Perplexity and Intelligence

November 22, 2026ResearchSafetyPerplexity and IntelligenceBy Jakub Pachocki, Chief Scientist at OpenAIFrom afar, it might seem like modern AI is built similarly to how software has been built in previous decades: with a diverse set of techniques targeting specific capabilities. There are indeed a variety of methods, but rather than designing something from scratch, they are mostly harnessing the intelligence coming from a much simpler and less interpretable process: training a compressor (or, equivalently⁠(opens in a new window), predictor) on a very large corpus of data.1I believe tracking the pace of progress on training better compressors is key to understanding the arc of advances in AI. In this post, I argue for establishing perplexity (or, equivalently, cross-entropy loss) as a…

Continue reading at Hacker News openai.com
Advertisement