Hi @sergioval,
Yes, your understanding is correct. The object ID abstracts away the model details before it’s sent up to the higher level. That ID, along with the pose of the child object relative to the parent, gets stored at a 3D location in the higher-level LM.
But that same higher-level LM also receives direct sensory input from an SM with a larger receptive field, and that input is stored at the same 3D location. So from the higher-level LM’s point of view, the object ID coming from the lower-level LM is just another feature. It helps the LM recognize the object faster when it’s available, since evidence accumulates more quickly with multiple input channels.
Whether object IDs embed semantics, and how, is something we debate a lot in our research meetings. It’s an active research area for us, so our thinking here may shift as we work through the theory and how it connects to representing classes of objects. That said, I’ll take a shot at answering, keeping in mind that what follows is my own perspective on the topic and may not fully align with how everyone at TBP thinks about this problem.
In the thousand brains theory, every column runs the same cortical algorithm no matter where it sits in the heterarchy. An object ID arriving at a higher-level LM is treated as a feature input, the same way a lower-level LM treats pose or RGB-D features. The LM recognizes an object by comparing the incoming feature against what it has stored, and it adds evidence in proportion to how well they match. That’s a big part of why Monty generalizes well across sensor noise, lighting changes, and other variability. Even when features don’t match up exactly, partial matches still contribute partial evidence.
If we apply that same logic to object IDs, then to generalize well we probably need a semantic distance measure over object IDs. The distance between two IDs should be small when the two objects are interchangeable for recognizing the parent object. At the moment, Monty does this object ID feature matching as a binary match/no-match classification, but we may start to look at embedding semantics in the future when we work with larger compositional datasets and generalization becomes essential for scaling.
One way to get there is to force objects with similar morphologies to share more SDR bits, so their representations overlap. The EvidenceSDR implementation might be a useful starting point, though it isn’t the full picture, since objects can also be grouped by higher-level context like affordances, not just morphology.
The ego-thriving semantics you describe sounds like another high-level heuristic for grouping object IDs. It reminds me of a Lawrence Barsalou quote that @bryce_bate shared in another thread:
A car, a flyswatter, and a house seem to have nothing in common until you’re being chased by some bees. Suddenly, each belongs to the concept of ‘Things to keep you safe from bees.’
The grouping may depend on context, which makes me think a single fixed distance measure won’t capture it on its own, but it may rely on changing high-level context in the receiving column.
Hope this helps.