I’ve spent the day experimenting with using ChatGPT to automagically generate accessible, indexed versions of ChatGPT sessions (ala Hand with Reflecting Sphere) that can dance around foibles in ChatGPT and this forum’s Discourse setup.
I seem to be making Real Progress ™, but it’s still very much a Work in Progress. So, I’ll keep at it for a while. Anyway, this post is supposed to contain a Markdown-encoded summary of my first query and the response I received. Posts for queries 2&3 will follow, then there will (it says here :-) be an index post. (Void where prohibited by law. No guarantees made or implied. YMMV…)
Query 1 — What are the benefits of SDRs in Monty? How are they used, implemented, etc?
ChatGPT said:
1. Why SDRs are valuable in Monty
1.1 Noise tolerance & robustness
- Very large dimensionality (e.g. 1–10k bits)
- Very low activity (e.g. 1–3% ON)
- Graceful degradation
- Sensor noise immunity
- Fault tolerance across distributed actors
1.2 Semantic similarity “for free”
- Overlap = meaning
- Hamming distance ≈ semantic distance
1.3 Composability and union
- Union (bitwise OR) without catastrophic interference
- Subsampling without losing identity
1.4 Stable identity across transformations
- Object identity, pose, orientation, scale, motion
1.5 Distributed consensus without central control
- Compact
- Mergeable
- Cheap comparison
2. How SDRs are used in Monty
2.1 Inside a Learning Module (LM)
- Feature SDR
- Location SDR
- Object SDR
- State SDR
- Prediction SDR
2.2 Evidence accumulation
- Compare new SDRs against stored SDRs
- Confidence += overlap
2.3 Cross-LM communication
- Exchange compressed belief states
- Hypothesis SDRs only
2.4 Action selection
- Predicted SDRs encode expected outcomes
- Actions chosen to reduce uncertainty
3. How SDRs are implemented (practically)
3.1 Representation formats
- Bitsets
- Index lists
- Bloom-style hashes
- Roaring bitmaps
3.2 Encoding sensory data
- Scalar encoder
- Coordinate encoder
- Orientation encoder
- Feature identity encoder
3.3 Learning = storing SDR associations
- Sparse associative memory
- Keyed lookup tables
- Hebbian-style increment counters
3.4 Scaling to many actors
- Good for UDP
- Missing bits don’t kill meaning
- Duplicate bits don’t amplify meaning
4. Why SDRs fit Monty better than dense vectors
- Dense vectors fragile to noise
- SDRs robust, mergeable, local learning
5. Key intuition
- SDRs let Monty think with partial truth