# Thoughts after monkeying around with Monty **URL:** \ **Category:** Research and Theory\ **Created:** [October 2, 2026, 7:22pm UTC](https://forum.thousandbrains.org/t/thoughts-after-monkeying-around-with-monty/1218 "2026-10-02T19:22:28Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Bryan\_Leavelle](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.thousandbrains.org/bryan_leavelle/32/751_2.png) [@Bryan\_Leavelle](https://forum.thousandbrains.org/u/Bryan_Leavelle)\ **Post date:** [October 2, 2026, 7:22pm UTC](https://forum.thousandbrains.org/t/thoughts-after-monkeying-around-with-monty/1218/1 "2026-10-02T19:22:28Z")
I’ve been messing around with Monty for a few weeks now, and it’s been a mind-bending experience. I learned early on that the eye and the finger are essentially different versions of the same thing (programmatically) with a shared failure mode. The “touch” is a zoomed in eye with color vision. I figured, if this is a brain, and TBT works how it says it should, that the best thing to do would be to try another sense. The senses are confused, and need more distributed helpers. A little more complexity is needed. _I TOO_ am confused, and writing is what I do to untangle myself, so I figured I would write up my thoughts around my experiences with Monty. I started with the ear - Richard Lyon’s CARFAC system which is open sourced on Git. Complex indeed. This whole time I have been operating on one rule: use the formatting that tbp.monty requires, don’t change it. Pipe existing peer-reviewed Python senses into Monty. This led to me having to run lots of mental experiments: 1. How is place found from audio signal? What IS transposition in audio? 2. What does audio “look” like to an eye, how can an ear “hear” a visual percept? 3. What is the bare minimum technical requirement for getting a thing to use multiple sensors to learn and do it in real time on minimal hardware? As it turns out, most of these are the same question asked in different ways, and y’all already handed me the basic outline for answering these questions: 1. Formatted Monty inputs solve this because everything gets a graph, and you just need to pick the right axes for the sense so it is readable and translatable across senses. 2. They fail different and this is where their expertise comes from, but they can relate to one another because of said previously-engineered formatting. 3. It’s a rasp pi with minimal Python. Cause Python is slow as F… shmuck. I really believe that this project is a few numba /EEL2 JITs and some creative thinking around formatting various other LMs for different senses away from something really special. Why? First, the speed and minimal training that I have found Monty needs to learn a voice from another voice, a beer can from a kid, is amazing. I sort of broke my rule and made FrankenMonty, a numba-jit of the real deal, to see if it could run on a pi. Thats the tech side of why I think this is amazing: it can work in real time on almost no compute resources and as many senses as you want. The world moves in real time, and bodies respond with various sorts of rapidity based on the nature of the bandwidths that any one primary sense is in charge of. Recognition is not immediate in organisms, it’s a process of leaky accumulators trading voltage based on their inputs, if you ask me, the amp builder guy. Evidence should drain just as it accumulates. Light is faster than sound. Bass is slower than transients in the high end. An eye can match a bunch of different things as the same thing because it’s not that great at telling them apart, and that’s where other senses come in. That’s how a mug and a warped mug are still mugs: the eye really does think it’s the same thing, because it cannot tell the difference. Varying failure modes between senses is key: those that fail together, die. Those that fail differently try-catch their way to fitness. Its Folie a Deux, embodied. A cup literally sounds different than an apple, and with only eyes and a finger that’s also an eye, that difference would never be sorted out. I have become convinced that making Monty run in real time while plugging in as many senses with disparate failure modes as possible will lead to something very special and different from anything we have today in AI. In short: faster, compute-forward operation with arbitrary N of allowable inputs that fail on different bandwidths and time scales is a cheap, fun and easy way to really stress test the Thousand Brains Theory. Complexity here IS a symptom of confusion, as someone wise once said, but not conscious confusion and perceived complexity: confused senses die, and more complicated circuits are born with no instructions through random mutation and amplified by fitness. Confusion kills, and so does unearned certainty. But confusion that leads to another evolved sense that fills the gap, a filter on another band, can easily work that out. Confusion over a complex world becomes a simple answer handed to us by physics and evolutionary pressure, a more complicated thing causing more confusion when its looked at, waiting for it to be turned into a simple answer again, which leads to more confusion, and more built complexity that results in simpler answers, over and over and over and over. A sense is a physical quantity plus a band. Each one owns a band nobody else does, and that’s the whole point of it. Below is a list of other potential biosensors (ive got parts in the mail!) that I have been keeping up on, that all have different failure modes and could be immediately implemented in Monty: ![image](https://canada1.discourse-cdn.com/flex034/uploads/thousandbrains/original/2X/b/b903bdf01d2638d54b0acffd30c80030ae9a2e77.jpeg) The key here is not copying senses. If it looks at pixels, it is an eye, no matter how much you zoom in, unless there are derived _findable_ features that one can sense and the other cannot. The test is simple: **does it fail where the others fail?** If it does, it’s one witness counted twice. The eye and our camera “touch” gave the same answer on 18/18 objects Monty had never learned. When I connect new senses to Monty, these are the rules I will follow: 1. **Its quantity can’t come from the camera.** If pixels give it to you, it’s an eye. 2. **Its band sets its leak.** Evidence drains as fast as that sense refreshes. 3. **Its failure mode is orthogonal.** Senses that fail differently catch each other’s lies. 4. **It’s in Monty’s format, unchanged:** location on the object, pose, features. For touch, location is the contact point, pose is the contact normal, and features are force, compliance, friction, vibration and heat. Not color and curvature. 5. **It can say no.** A sense that can only agree adds nothing. 6. **Find it twice:** once in MuJoCo, once on a ten-buck sensor. 7. **No Halting:** Halting is death. You have to give each sense the time it needs to drain, update, accumulate, in their own bandwidth domain. Charts below showing the difference between stock Monty, Monty running over and over again, and Monty never shutting off while changing the object it is looking at. ![image](https://canada1.discourse-cdn.com/flex034/uploads/thousandbrains/original/2X/6/611f06539c6b55bd42040c13bcf570f426d03e33.png) MuJoCo already knows a mug is stiff, slick and heavy, and a banana is soft, waxy and light. None of that reaches Monty (unless I’m missing something), while two senses argue over the same pixels. Touch should not be sight, zoomed in. It’s five senses wired to a breadboard, each on its own band, each failing its own way, and cannot tell the color of what its touching. This is where I’m headed now that the CARFAC fork is up, and I’m excited to see where it goes. Thanks again for everything, this community rocks. ---
**Author:** ![nleadholm](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.thousandbrains.org/nleadholm/32/48_2.png) [@nleadholm](https://forum.thousandbrains.org/u/nleadholm)\ **Post date:** [October 19, 2026, 11:24am UTC](https://forum.thousandbrains.org/t/thoughts-after-monkeying-around-with-monty/1218/2 "2026-10-19T11:24:34Z")
Thanks Bryan, just to add that any work adding touch-modality properties like texture / force to the Mujoco pipeline would be very welcome. I believe the YCB dataset would need to be updated (or a new dataset identified) to capture such physical properties, but it’s certainly something we want to deal with in the future.