@nleadholm and @sknudstrup explore what an attention system might look like inside Monty. They discuss a salience sensor module that segments 2D images into spatial attention regions, the use of voxel grids to represent and update those regions, and how attention could filter goal locations before they reach the motor system.
The conversation continues into some fascinating questions: How can Monty use attention to focus on individual parts of a compositional object? How can attention help Monty avoid revisiting areas it has already recognized? And should attention in Monty reflect the largely subcortical way attention operates in the human brain?
The conversation finishes with an architectural exploration of where an attention system would fit within the overall data pipeline of Monty.
Main Video
Summary Video
0:08 Introductory Remarks
2:38 New Components Needed for Attention
4:04 Clarifying the SalienceSM Component for Attention (Superior Colliculus Analogy)
5:46 The New Attention System Component (Overview and Discussion)
15:32 Question about Proposed Goals & Constraints for the Attention System
17:21 Some Ongoing Points of Discussion on Designing an Attention System
19:24 Question about Different Types of Location Representations
24:00 Model-Based Attention
26:12 Broad vs Narrow Attention for Object Recognition
27:22 Is It Really Model-Based During Inference?
33:32 Refresher on Existing Hypothesis Testing Policy
35:52 Is Default Rapid Inference a Model-Free Narrowing of Attention?
41:43 What Informs the Narrowing of Attention?
49:53 Attention Can Help Us Move On and Not Return to Areas We’ve Already Recognized
52:34
Should We Focus on Model-Free or Model-Based Narrowing of Attention First?
55:18 Could Top-Down Signals Determine an Attention Region and Bias Model-Free Segmentation?
1:00:00 Could Attention Narrowing Happen in the Attention System Itself?
1:07:39 Inhibition of Return
1:09:30 Attention is a Complex System in Its Own Right
1:12:09 What are the Biological Constraints on Attention?
1:21:39 Covert Attention Discussion
1:29:47 Spatial Salience (Attention) as Motor Readiness Discussion
1:31:51 In Monty, We Aren’t Constrained to Making Attention Biologically Accurate to Subcortical Systems
1:35:07 More Discussions on Bottom-Up and Top-Down Attention
1:37:18 Question about Monty Architecture and Attention
1:42:35 Subcortical vs Cortical Control of Movement
1:43:49 Monty Architecture: Data Flow Diagram
1:48:32 Is an Attention System a Phase in the Data Pipeline or a Global Filtering System for Every Message in the Data Pipeline?