Hacker News

What happens when an LLM never sees material beyond fifth grade?

I've never seen anything scarier than an LLM with tool calls ...

Talk to LittleLearner The hosted 5B model, live in your browser. Open in a new tab ↗ if the chat doesn’t load below. A controlled sandbox for studying how models acquire knowledge Modern LMs are trained on everything at once, so it is hard to tell whether a new skill was learned or merely elicited. We constrain the training distribution itself: an 88B-token corpus filtered to the U.S. elementary-school curriculum, with models trained from scratch on it and matched unfiltered controls. Dataset LittleCurriculum An 88B-token corpus distilled from FineWeb-Edu through a five-stage filtering pipeline aligned with Common Core standards (K–5). Concepts, facts, and vocabulary taught above Grade 5 are explicitly excluded. Models LittleLearner Three scales (0.6B / 1.3B / 5B)…

Continue reading at Hacker News littlelearner-ll.github.io
Advertisement