Chatting naturally with a computer now feels routine, but as MIT Technology Review’s Elise Cutts reports, the large language models behind that fluency are shockingly data-hungry compared to human toddlers. Meta’s Llama 3.1 trained on 15 trillion tokens; a preteen raised in a language-rich home hears something like 100 million words. “We still have to burn down a forest and scrape the entire sum of all human knowledge to re-create this milestone that happens in our living rooms over the course of a year,” says Stanford cognitive scientist Michael Frank.
Closing the gap
Researchers are now using children’s own learning process as a blueprint. The BabyLM competition, founded by linguist Alex Warstadt, challenges teams to train models on just 100 million “developmentally plausible” words drawn from storybooks and child-directed speech. Meanwhile, projects like Princeton’s newly detailed 1,000-day home-recording study aim to capture what children actually see and hear. As Georgetown’s Ethan Wilcox puts it, “Claude has seen the amount of language that an entire city will experience in one generation” — humans still learn it in a living room.
Related on the SLP
|
Related on the SLP
ChatGPT and other generative AI suppresses brain activity. Is it slowly making us stupid?
|
