November 22, 2026ResearchSafetyPerplexity and IntelligenceBy Jakub Pachocki, Chief Scientist at OpenAIFrom afar, it might seem like modern AI is built similarly to how software has been built in previous decades: with a diverse set of techniques targeting specific capabilities. There are indeed a variety of methods, but rather than designing something from scratch, they are mostly harnessing the intelligence coming from a much simpler and less interpretable process: training a compressor (or, equivalently(opens in a new window), predictor) on a very large corpus of data.1I believe tracking the pace of progress on training better compressors is key to understanding the arc of advances in AI. In this post, I argue for establishing perplexity (or, equivalently, cross-entropy loss) as a…
Perplexity and Intelligence
Advertisement