@vclay reviews the idea of using sequence memory to model behavior trajectories. This idea was presented by @jhawkins last week ( • 2026/05 - Trajectory Memory for Behavior M… ) and removes the use of reference frames from behavior models. The team discusses issues and questions around the new idea.
Summary Video
Main Video
0:00 Introduction
2:16 Trajectory Memory for Behavior Models
7:30 Open Questions Around This
37:03 More Questions and Discussion
40:34 Necessary Assumption: Behavior Model Input Refers to Entire Object Movement
1:13:03 Further Discussion
1:30:03 Selective Attention
1:40:24 Can We Use The Spatial Pooler to Learn the Most Common Movements?
1:45:09 Paper: Physiological Differences Between Neurons in Layer 2 and Layer 3 of Primary Visual Cortex (V1) of Alert Macaque Monkeys
1:56:27 More Papers Around L2/3 and Connectivity
2:01:33 More Open Questions
2:07:44 Do Trajectory Models Meet All Our Requirements?
2:12:32 Using Trajectory Models for Movement Outputs
I wasn’t able to follow all of the ins and outs of the discussion (I hope to give it another try RSN :-), but I was able to understand most of the words, unlike in some prior videos. Please commend the AV crew and video editor(s) for a very clean job.
That said, there are some enhancements I’d like to suggest, in the hope of making this resource even more valuable (e.g., searchable)…
- Make all of the visual aids available as PDFs.
- Create an accurate transcript of the video, including time stamps and info on who said what.
- Add links to the PDFs and transcript to the video discussion thread, for easy access.
- And a pony…
-r
Thank you for the suggestions, @Rich_Morin! 
We’re a very small team with limited resources (for example, we don’t have an AV crew), so it isn’t possible to make these enhancements right now. At the same time, we’ll do our best to make more PDFs available whenever possible, and we’ll keep track of these suggestions for possible future improvements.
Of the suggestions I made, transcript generation is clearly the biggest ask. I wonder if a combination of LLM and volunteer (e.g., student) labor might be able to address this.
That’s a thoughtful idea, thank you, Rich! We are already using LLMs for the transcription on Youtube (the subtitles/closed captions), but none of the LLMs we’ve tried have worked that well in transcribing the videos to a high level of accuracy. We don’t have the volunteer labor for now, so we’re going to stick to the current process, which our team can realistically support. We really appreciate the suggestion, and we’ll keep your suggestions in mind as our resources evolve.
