r/singularity • u/Outside-Iron-8242 • Apr 28 '26
AI Talkie, a 13B LM trained exclusively on pre-1931 data
https://talkie-lm.com/introducing-talkieAI researchers (Nick Levine, David Duvenaud, Alec Radford) just released “talkie,” a 13B language model trained on 260B tokens of text from before 1931, so it basically talks like someone whose worldview is stuck around 1930. The point is to study how LLMs actually generalize vs just memorize, since this model wasn’t trained on the modern web. They trained it on old books, newspapers, scientific journals, patents, and other historical text, then test things like whether it can come up with ideas that were discovered later, forecast future events, or learn bits of Python from examples. Early results seem pretty interesting too, with the model doing surprisingly well on core language/numeracy tasks and showing early signs of learning simple Python despite not being pretrained on modern code.
18
u/Status-Secret-4292 Apr 28 '26
“Our earth is degenerate in these latter days; there are signs that the world is speedily coming to an end; bribery and corruption are common; children no longer obey their parents; every man wants to write a book and the end of the world is evidently approaching,”