Post #2959253

I hate how LLMs are often used, but as a model of cognition I think they're fascinating. Take this paper, for example. Here they look at reasoning over long sequences of data that can't fit into working memory. The latest LLMs can solve problems like this, using hybrid memory models that consolidate "short-term" memories in an LRU cache into "long-term" memories in the form of persistent weights (which is good to know about generally). However, this paper shows that this is still a very hard problem. In particular, it's hard to organize the consolidated information in a way that's useful for doing future work, and that a "sleep-like" process of iterative offline processing can help significantly. It remains to be seen how important this will be and how "sleep-like" it really is. Still, I love how this work draws inspiration from nature and could potentially help us learn about animal minds and cognition generally. Much better than many LLM studies. #science #ai #llm