Remindings and the Hippocampal Timeline

[Bryce_Bate]

1. Starting Point: What the November Discussion Left Open

As I listened to the Nov. 2025 TBP video “Discussing the Relationship Between Hippocampal Complex and Neocortex” (https://youtu.be/b_wV2hsdEGY?si=FG0ABrBLBAIZwYAF), my attention focused on the portion that talked about temporal ordering and the timeline of events that are somehow stored as part of an episodic memory. The discussion then seemed to turn to this musing about the mechanism behind these timelines—"Like a log, Monty could have the advantage of having an infinite, episodic memory.”

This resonated because it tracks with a project I’m working on to implement a kind of prosthetic hippocampus, not for Monty per se, but for being personally reminded of past thoughts that can inform my present ones without me searching my old notes. This cognitive behavior was given a name by the late cognitive scientist Roger C. Schank. He called them “remindings.”

2. Schank’s Remindings

Schank developed a theory of scripts in the 1970s and 1980s that governed his early approach to AI. I think that Schank’s scripts operate like Kahneman’s ‘fast thinking’ — automatic, unconscious, and efficient until reality violates the expectation. When that happens, the script fails, and the system must recover. Schank’s AI systems had to deal with these failure/recovery situations. Here’s one of his examples:

Every time you’ve gone out to eat, you’ve gone to sit-down restaurants in which the sequence is this:

1. You arrive and take a seat (or, in some variations are seated)
2. The server offers you a menu and comes back to take your order
3. The food arrives, and you eat
4. The server presents you with a check, and you pay
5. You leave

But now you’re going to a McDonald’s for the first time. You arrive and take a seat. No one takes your order. You wait endlessly. The SCRIPT FAILS.

What happens next? For some, you suddenly think of a past incident where waiting for service didn’t work because it was self-serve. Maybe you were reminded of your first time at the library, where you learned you had to find the book on your own and bring it to the checkout counter.

Schank calls these “remindings.” They are mechanisms for recovering from script/prediction failures. In TBT terms, you may have fetched the wrong reference frame, resulting in a prediction failure. They may be path failures in the neocortex, brought on by the retrieval of the wrong reference frame or other changes to expected features within the chosen reference frame.

3. The Teachable Moment Connection

Each script failure, or failure in fast thinking to correctly predict the immediate future, is what teachers call a “teachable moment.” The surprise that arises from a prediction failure occasions an unconscious process of reminding that constructs a more distant memory out of episodes whose parts, when put together, form a structurally useful analogy to the failure situation. This reminding memory surfaces seemingly out of nowhere.

It would be helpful to understand what happens in the disrupted sensorimotor behavior in the neocortex that triggers a reminding involving the hippocampus. The disruption must be the detection of a prediction error. The disruption becomes the opportunity to inform something in the present with an experience from the past. It need not be consciously requested.

4. What Gets Retrieved, and From Where

I’m proposing that the retrieval of episodic memory (more accurately, the retrieval of the salient aspects of the episode for assembly into a reminding) involves a more complex match than the retrieval of only semantic information. It involves at least these elements—temporal distance, structural gist, interoceptive/affective state, and semantic memory.

All of these elements are likely distributed across structures — episodic indexing in the hippocampus, semantic content in neocortical stores — retrieved together in the construction of a reminding.

Temporal distance, as discussed in the video, offers a significant advantage in a mechanism that can surface just the right episodic memory. It seems to play an important role especially in remindings that arise unconsciously. When we’re consciously trying to recall events, we’re strongly influenced by recency. There’s a priming effect of recalling recent things. This works great for recalling where you left your keys.

However, for prediction errors, temporal distance is what you need. It helps ensure a fix to the fast, automatic thinking in a way that is longer-lasting. Nearby episodes share too much surface context with the current failure to offer genuinely new solutions — they’re likely products of the same conditions that caused the failure in the first place. That’s why temporal distance may be better. It increases the chances that the error correction might survive.

The remaining elements all play important roles in the reminding. The structural gist of the episode serves as the backbone of analogous thinking, providing a framework for the solution. The interoceptive/affective state of the person at the time of the episode follows a concept in psychology called “State-Dependent Memory.” You are much more likely to remember a solution to a frustrating problem when you are frustrated again. And finally, there are the semantic facts of the episode that need to be blended to reconstruct the full episodic memory.

What I am proposing is a “reminding” behavior that is different from other memory retrievals—different in what triggers it, but also in the distance from which its elements are retrieved for memory reconstruction.

5. Implications for Thousand Brains Theory

Without a reminding mechanism, a Monty as a robot exhibiting prediction failures starts looking like an old Roomba vacuum cleaner caught in a corner, banging itself against the wall with no ability to get out of its predicament. Monty will eventually have to learn to self-correct when it fails.

Those “out-of-the-blue” remembrances we sometimes have when we’re struggling with something aren’t accidents. I believe they represent a purposeful mechanism for extending the structural advances of things we’ve learned from experience into the future. Because they seemingly arise with no conscious effort, I think we give them short shrift as though they are in the class of unexplained serendipitous things. But they are part of each of us and happen frequently enough not to be explained away as lucky memories.

I’m not offering any ideas on the physiology of this mechanism. I am, however, saying it warrants attention as the team investigates the hippocampus and its relationship(s) to the neocortex and cortical theory. It is a behavior of the mind that should be accounted for in the theory (TBT) and incorporated into Monty at some point.

I’d be interested in what others think of the implications for a Monty with the ability to spontaneously remind itself of a past episode that is relevant to a present prediction failure.

[rmounir]

These are interesting observations. I can connect some of what you’re describing to things Monty already does today, and to where we’re headed.

Prediction errors and burst sampling

Monty tracks prediction errors and bases important decisions on them. It maintains a set of hypotheses about what object it’s interacting with and where it is on that object. As it moves through reference frames, it uses those hypotheses to predict what it should observe next. If the actual observation doesn’t match, that’s a prediction error, and evidence for that hypothesis drops.

When the best hypotheses all produce high prediction error, meaning nothing in the current set of guesses can explain what Monty is sensing, we trigger what we call burst sampling. Monty initializes new hypotheses across all of its existing models. These sampled hypotheses are informed by the current observation features, not randomly initialized.

Learning failure-resolution associations

In my opinion, the remindings you describe, where past resolutions get retrieved when similar failures come up, could be built as learned associations on top of this. You can learn biased priors from experience to resolve failures, indexed by the context of the failure.

Say Monty has learned models of several containers: a mug, a bowl, a vase. It’s exploring an object, its “bowl” hypothesis fails because it encounters a handle, and it eventually recovers by switching to “mug”. That failure-to-resolution path (bowl fails at handle, mug was the answer) can be learned for more efficient future retrieval. Next time Monty is holding a bowl hypothesis and hits a similar unexpected feature, it tries “mug” first rather than searching blindly across all models. Over many such experiences, these associations can get reinforced. The system builds up recovery shortcuts indexed by the type of prediction failure it encountered.

Top-down guidance from compositional models

There’s another mechanism worth mentioning here: top-down connections in our compositional (hierarchical) models. In Monty’s architecture, higher-level learning modules build models of how objects are arranged relative to each other (a scene or a composite object), while lower-level modules handle the individual parts. When a lower-level module hits a high prediction error, the higher-level scene representation can guide it toward better solutions.

Take your McDonald’s example. Imagine the higher-level module has recognized “restaurant” as the scene. When the lower-level module’s “wait to be seated” script fails, the scene-level context of “restaurant” biases which alternatives get tested next. “Go to counter” gets favored over completely unrelated hypotheses, because the higher-level model knows what kinds of sub-behaviors tend to occur in restaurant scenes.

We can also relate this to Monty’s object recognition. Monty is exploring a mug with a logo on it. The higher-level learning module has a compositional model of the mug: handle, logo, and body as child objects at known relative locations. when the sensor moves off the logo, the lower-level LM would have no idea what will be sensed next, but instead of searching blindly, the higher-level module biases the lower-level module to try “mug handle” or “mug body” next, based on where the sensor is relative to the overall mug model. In this case, the higher-level context narrows the lower-level search.

We are currently working on this as part of the compositional modeling work, learn more about it here in this future works item.

[Bryce_Bate]

@rmounir, thank you for taking the time to give such careful consideration to the memory phenomenon I was trying to raise. Not only did I learn more about the deep thinking going on in the TBP work, but it made me want to sharpen my examples. Investigating certain narrow cases of episodic memory feels like I’m trying to firmly grasp a piece of aerogel so I can describe its physical characteristics.

To that end, I think my examples may have fallen short. It sounds as if Monty is (or will be) handling them quite elegantly. So, let me try a different example of the sort of cross-domain memory that seems to pop up under certain conditions. I’m going to draw upon an example I recall from neuroscientist Lisa Feldman Barrett citing (in “How Emotions Are Made”) psychologist Lawrence Barsalou’s work on goal-oriented dynamic concepts.

A car, a flyswatter, and a house seem to have nothing in common until you're being chased by some bees. Suddenly, each belongs to the concept of "Things to keep you safe from bees." 

That’s more like what I think a reminding is than statistical priors. It is the assemblage of concepts, each belonging to their own separate categories, that come together to form a new category of “Things to keep you safe from bees.” It feels like more than a priming effect going on in such cases. What makes the reminding unique may be this merging of other concepts to form a new one addressing a situational goal. How this forms a kind of episodic memory may require a bridge be built between episodic memories and dynamic concepts. I’m trying to bring out the structural gists of past episodes that seem to form certain remindings more so than semantic similarities. These structures cut across subject areas bearing no surface resemblance to each other.

I’ll keep pondering this. Thanks again for shedding fresh light on my examples. It has helped me sharpen my view of how to approach this elusive episodic aerogel.

– Bryce

[rmounir]

Thanks @Bryce_Bate, that’s a very thought-provoking example. This reminds me of the Dimensional Change Card Sort (DCCS) task [Zelazo, 2006] in developmental psychology. Kids sort cards (e.g., red rabbits, blue boats) by one dimension, say color, and do fine. Then the experimenter switches the rule to sort by shape instead. Three-year-olds reliably fail the switch and keep sorting by color. Five-year-olds can flexibly attend to the new dimension. It feels like remindings can be thought of as learning new dimensions along which to relate objects, or events, and assess their similarity. In the bees example, the new dimension is guided by affordances, rather than color or shape, “what can shield me from bees”. Perhaps learning and generalizing these dimensions can make it easier to retrieve “similar” events across different domains.

We’ve discussed the issues around classes of objects in our research meetings a few times in the past, but this is still an open question as far as I know.

[Bryce_Bate]

@rmounir– Your own reminding of the DCCS task was itself a perfect demonstration of the phenomenon — I appreciated the irony. It helped me better understand what I was struggling to explain about why I felt these instances were not cases of priming but of something else more structural.

Let me try to break this down by going back to the Bee Chase example. I think the “reminding” comes after the dimensional switching has taken place. Before the bees started their chase, the person had separate utility concepts for things like car, flyswatter, and house. And maybe these concepts each had their own priming effects in certain situations. For example, if asked, “Name something used for transportation” when sitting in a car, the priming effect would favor “car” over “bus” or “bike.” Or while holding a flyswatter and asked, “What’s a good tool for dealing with bugs in the kitchen?” the answer “flyswatter” would be favored over “broom” or “spray.” And similarly while sitting in one’s home, being asked, “What’s the best place to relax with family?” “house or home” would be more likely than perhaps “beach.” Consider these to be at some default cognitive level where the priming effect rules.

Suddenly in the panic of a bee chase, there’s an override of these default concepts–a suppression of their priming effect. We dynamically construct a new concept/category for the situation–“Things to keep you safe from bees.” And we SWITCH dimensions to this new one that then overrides the previous one (e.g., “A thing used for transportation,” “A tool used for dealing with bugs,” or “A place to relax with family”).

Maybe the sequence is like this:
1. Bees chase
2. We form a new goal: “Avoid being stung”
3. That leads to another goal: “Find protection from harm from flying insects”
4. A new category is dynamically constructed to serve that goal: “Things that keep you safe from bees”
5. Memory searches for instances of that newly constructed category — recruiting across previously unrelated semantic and episodic stores based on affordance rather than prior category membership (e.g., car, house, flyswatter, jacket, etc.)

I’m sure more is going on here than I’m conveying. My larger point is that your being reminded about dimension switching seems key in this process. I’m adding the Barsalou idea that the dimension (concept? category?) is created on the fly to serve a new goal. And in doing so, previous higher-order concepts for which cars, flyswatters, or homes are instances of quite separate categories, are suddenly now instances of a new dimension that suppresses the other dimensions. The car is now a place of shelter and not merely an item of transportation.

I’m not sure I have more to offer on this at the moment. My real goal was to bring this kind of cognitive phenomenon to the discussion of episodic memory and the hippocampus. I think it warrants more thought. Dimensional switching seems to be involved along with the suppression of previous priming effects at some higher-level.

I really appreciate your thoughts on this and time taken to share your own reminding. You’ll have to admit, remindings can be pretty powerful cognitive tools.

– Bryce