@vclay leads an open-ended discussion on the remaining open theory questions for the Thousand Brains Theory. The high-level categories we discuss are attention/segmentation/input sharing, scale & deformations, unsupervised learning, model representations, and goals and actions.
If you are curious, you can explore the Excalidraw board for the open theory questions: https://link.excalidraw.com/readonly/…
Summary Video
Main Video
0:00 Introduction
1:00 How the Excalidraw Board Works
2:30 Attention, Segmentation & Input Sharing
3:02 What Is Attention?
4:46 We Need The Ability To Segment Objects
6:11 Sharing Input with Many Learning Modules
8:48 Is Attention Egocentric when Model-Free and Allocentric when Model-Based?
13:56 What Is the Shape of an Attentional Area?
17:33 Recognizing Objects in 3D that Have Only Been Seen in 2D
25:42 Scale & Deformations
30:37 Recognizing An Object & Pose in a Novel Scale
41:56 Recognizing Volumetric Deformations
44:49 Model Representations
45:09 How Do We Represent Symmetry?
57:32 Representing Repeating Patterns Efficiently
1:00:36 How Do We Represent Sets?
1:02:59 Representing Where an Object Ends & Dealing with Incomplete Models
1:15:12 How Do We Learn Abstract Models?
1:15:52 Nice to Have: Performing Efficient Nearest Neighbor Querying
1:16:30 Nice to Have: Communicating Displacements Instead of Locations
1:17:19 Is There a Connection between the Canonical Views and the Abstract Models?
1:20:58 Unsupervised Learning
1:22:13 Weekly Supervised end-to-end Training
1:22:42 Learning Temporary Models
1:23:17 Introducing a Learning Speed Parameter
1:23:52 Consolidating Memories and Forgetting
1:25:21 How to Correctly Classify Whether an Object Is New?
1:28:08 More Thoughts on Unsupervised Learning
1:31:42 Goals & Actions: Manipulating the World to Achieve Goals
1:32:11 How Do We Learn Causality?
1:32:42 The System Needs to Define Its Own Goals & Decompose Goals into Sub-Goals
1:33:28 Model-Based Planning
1:36:42 Simulating Consequences of Actions
1:37:10 Coordinating Actions of Multiple Actuators
1:38:17 Final Thoughts