Talk to LittleLearner The hosted 5B model, live in your browser. Open in a new tab ↗ if the chat doesn’t load below. A controlled sandbox for studying how models acquire knowledge Modern LMs are trained on everything at once, so it is hard to tell whether a new skill was learned or merely elicited. We constrain the training distribution itself: an 88B-token corpus filtered to the U.S. elementary-school curriculum, with models trained from scratch on it and matched unfiltered controls. Dataset LittleCurriculum An 88B-token corpus distilled from FineWeb-Edu through a five-stage filtering pipeline aligned with Common Core standards (K–5). Concepts, facts, and vocabulary taught above Grade 5 are explicitly excluded. Models LittleLearner Three scales (0.6B / 1.3B / 5B)…
What happens when an LLM never sees material beyond fifth grade?
Advertisement