THE FULL MONTY: a video game

Bryan_Leavelle · September 20, 2026, 3:02am

yo, Im working on a good way to distribute the executable for this next project so apologies for the lack of code on this one.

I love to make video games, and after I made the REAPER JSFX everything got very complicated. I think Monty works best when there are multiple sensors dedicated to their own bandwidth, and in an audio-only environment I had to be very creative with 2nd and 3rd “sensors”: what does audio signal look like to an eye? What is the correct “place” for a sound? If all I have is an ear, and no way to actually walk around and investigate the only signal i have and translate it to a world so i don’t hault/die in it, how can i use all the information in that signal in relation to its environment so that it survives? An audio signal in a DAW happens TO the observer. It’s a beautiful thing about audio, and a difficult as hell to work into a world model using it exclusively.

An ear is slower than an eye and touch, and all of them measure the world in their own unique way. Differentiating a room from a thing in a room requires more than just the provided touch and visual modules, and both need to be checked by one another and any other senses, including the observer’s own body as an object itself, and the room as the ultimate brute fact, for better or worse.

So, I fired up GODOT and made my bat a live thing. I made a numba jit of monty and the cochlea so it can run as fast as it can. I made a room that holds two columns, a chandelier (represented by 4 roof cones) and an organ (represented by 3 floor cones). The rules:

One: solve the room dimensions first, and then investigate its innards.

Two: If you see an object, investigate it. Find out where it connects to the room, at both ends, and find its dimensions. Assume the dimensions hold for the object.

Three: If you know an object by 3 senses, and another from one (echolocation is a good example here), go investigate them and add more sensory evidence for it.

I want to say that is it, but it isnt. Here is a video of the game in action:

Ill have a more thorough writeup soon. But anyone one with a linux or windows box that wants this program, send me your email and i can ship it. its about 200 megs.

Bryan_Leavelle · September 20, 2026, 3:05am

This also stores every single run in a SQL LITE db, formalized, with photos of the final map. One db init to a public server and people can run whatever they want and compare results. More info soon.

next thing i’d like to do is take learned shapes and apply them to similar shapes but it’s been a long day and Im tired (again). Monty rules, the greatest intuition pump i’ve ever used.

evanuno · September 20, 2026, 5:57pm

This is so great to see happening. There are a lot of people out there, who if they were aware of what Monty can do, would be doing exactly what you’re doing. I lost the ability to understand software languages about ten years ago, so my goal is just to get more people to come to the forum and at least ask questions. Meta’s ‘ai’ decided I was too active on threads and permanently deactivated my account a couple weeks ago because it thought I was a bot, so that’s limited to people I already know, or meet in person now That’s okay, I’ve got retirement things to do like making music with no commercial potential.

Looking forward to future developments.

Bryan_Leavelle · September 20, 2026, 11:53pm

right on dude glad you like it. next phase: robot. i’ve got the sense to motor done. monty and carfac will run on a rasp pi + arduino. finally out of the office and into the shop. more info soon… but i just loaded up at micro center. LETS GO

evanuno · September 25, 2026, 1:55am

OMG! I’ve been remodeling my RV, dropped by to check in and keep up on what’s happening here and

Bryan_Leavelle · September 27, 2026, 4:53am

WELL GOOD NEWS!!! if you’re into RC I NEED YOUR help! I’m making a full post right now but i’ve been locked in my garage working on this,

there’s a lot here but i give you BUD: the robotics system that works with CARFAC ears and Monty brain. ON A RASP PI. and a arduino and ESP32 but that’s just details. you could plug monty into one of your cars. sounds like your angle to get into the game?

Bryan_Leavelle · September 27, 2026, 4:55am

o RV but still … robots are just RVs! twelve volt gremlins!!!

evanuno · September 29, 2026, 4:57am

mine is 24V with 16 100W panels, 7.2kW of batteries, and a 3600W inverter

Bryan_Leavelle · October 1, 2026, 10:40pm

HOLY MOLY YES~!!! so sick!!!

sknudstrup · October 14, 2026, 11:52pm

Hey Bryan, Scott here. This is super cool – I’ve been meaning to get a response in for this project.

Just as a point of clarification, can you remind me if/how the tbp.monty repo is being used? Did you make an alternate, real-time optimized implementation for this and the other project you posted?

Bryan_Leavelle · October 15, 2026, 5:23pm

hey scott!

I’ve been working hard on various things related to this. It has wormed its way into my mind. I did fork the ear module for yall

Recently I was actually able to do a couple of cool things:

FrankenMonty - numba jit compiled version of stock monty for robotics and low-resource computer hardware.

2. 12 column Monty in a 6 object space with a body and a “No” vote - this is something new I’ve been working on this past week. I built a tiny ant with many senses that runs around a room with 6 Monty objects and wired all the columns together.I have been running experiments and testing outputs and have been trying to not get too excited, but it is very cool. This project also uses MuJoCo to give objects more physical parameters that are testable by other, orthogonal-failing sense organs (here’s a link to my post about this in more detail).

3. In general just reading and trying to stumble upon something else like the CARFAC for all other senses.

For 2, I did actually finally get the nerve to go in and fiddle with monty. I just found that it really needed a way to say “no” itself and some kind of confidence punishment that went along with it for incorrect columns. Assuming numpy import:

  from tbp.monty.frameworks.models.evidence_matching.feature_evidence.calculator import (

      DefaultFeatureEvidenceCalculator,

  )




  class NoFeatureEvidenceCalculator(DefaultFeatureEvidenceCalculator):

      NO_AT = 2.0          # x tolerance = a contradiction

      NO_EVIDENCE = -1.0




      @classmethod

      def calculate(cls, channel_feature_array, channel_feature_order, channel_feature_weights,

                    channel_query_features, channel_tolerances):

          evidence = super().calculate(channel_feature_array, channel_feature_order, channel_feature_weights,

                                       channel_query_features, channel_tolerances)

          # rebuild the stock per-column layout: query value + tolerance per feature column

          n_cols = channel_feature_array.shape[1]

          tol = np.full(n_cols, np.nan); query = np.full(n_cols, np.nan)

          circ = np.zeros(n_cols, bool); cat = np.zeros(n_cols, bool)

          i = 0

          for f in channel_feature_order:

              if f in cls.SKIP_FEATURES:

                  continue

              q = channel_query_features[f]

              n = len(q) if hasattr(q, "__len__") else 1

              query[i:i + n] = q; tol[i:i + n] = channel_tolerances[f]

              if f in cls.CIRCULAR_FEATURES: circ[i] = True

              elif f in cls.CATEGORICAL_FEATURES: cat[i:i + n] = True

              i += n

          diff = np.abs(channel_feature_array - query)

          if circ.any():

              c = channel_feature_array[:, circ]; qc = query[circ]

              diff[:, circ] = np.min([np.abs(cls.CIRCULAR_RANGE + c - qc), np.abs(c - qc),

                                      np.abs(c - (qc + cls.CIRCULAR_RANGE))], axis=0)

          diff[:, cat] = (channel_feature_array[:, cat] != query[cat]).astype(float) * 1e9

          # THE NO: any feature past NO_AT x tolerance -> this node scores -1, not the diluted average

          contradicts = np.nan_to_num(diff > cls.NO_AT * tol, nan=False).any(axis=1)

          return np.where(contradicts, cls.NO_EVIDENCE, evidence)




  To wire it in from Python (how ant_cols.py does it):




  EvidenceGraphLM(...,

      hypotheses_updater_args=dict(feature_evidence_calculator=NoFeatureEvidenceCalculator))




  Or from a Hydra config:




  hypotheses_updater_args:

    feature_evidence_calculator: ${monty.class:audiomonty.no.NoFeatureEvidenceCalculator}

Monty is usually done in sessions that run and stop and report. I wanted to see what happens if we treat all senses as if they were always on and trying to contribute. Accumulation after all is something that inherently happens over time, and if it can only grow, it will saturate fast and become a useless tautology before it can become useful to an embodied AI. This is all pretty time consuming cause you need to give all of the objects in MuJoCo properties that its various sense organs can id.

download (1)

download

graph: this shows where the bot is (orange dot) and where its vote says it is (purple star). The cloud of purple dots represent votes from various columns. Error on the top rght. Bananas are hard and get confounded with boxes.

Mainly though, I have been working on a fourth project: MEGA BUD. This is a 3 section ant with Strandbeest style legs. The “ant” I made for my 11 column Monty is a tiny version of this. I got tired of working around the tiny form factor and battery situation of OG bud and started on this insanity. I probably wont have anything super dope to show from this until next week or the week after… but when it’s up it will be cool as hell.

Lately this has all become a sort of “everything at the same time” kind of project, which is how I like em, but it means that my virtual desk is not exactly totally clean and the bench is covered in stuff. If there is anything you are particularly interested in me pushing and sharing to GitHub, im down, but it will have to come with the warning that this is the definition of a work in progress~!