Hey @ricpruss, there’s an existing topic about it: "Introducing Nested Learning: A new ML paradigm for continual learning" by Google
I was critical of it at the time, but a follow-up preprint paper by the same authors paints a clearer and much more interesting picture: [2606.03979] Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
Essentially, during “sleep”, an RL-based student-teacher framework blends data from the context window into frequency-layer parameter updates for short-term memory integration, while avoiding catastrophic forgetting. Although, they talk very little about performance and scalability, and well, it’s still… RL, with the downsides that come with it.