2025/12 - Looking Inside Monty’s “Brain”

Video Discussions
1

@rmounir walks through how Monty populates and explores the hypothesis space for object and pose recognition with interactive visualizations. He shows how Monty prunes and samples new hypotheses as part of its sensorimotor process. He also shows what symmetric objects look like in Monty’s hypothesis space.

00:00 Meet Monty 2026: Looking Inside Monty’s “Brain”
00:40 Object Recognition Experiment Setup
02:49 Visualizing Monty’s movements
04:11 Visualizing the Hypothesis Space
05:06 Visualizing an Individual Hypothesis
05:37 Example Hypotheses
07:05 Hypotheses on Other Objects
07:29 Visualizing the Hypothesized Path
08:59 Animation of the Hypothesis Space Over Time
09:19 How Evidence is Calculated and Evolves Over Time
12:58 Hypotheses Around an Axis of Symmetry
14:19 Hypothesis Space Pruning and Burst Sampling
15:43 Visualization of Possible Sensor Locations in the Hypothesis Space
17:46 tbp.plot Repository and Other Resources

1 Like
2

This walkthrough was really good, so I wanted to reproduce the hypothesis-space plots he demonstrates. I already have the tbp.teleop live visualizer working. The piece I’m stuck on is the tbp.plot replay views.

With the latest clone of Monty and tbp.plot, I tried its plots against a randrot_noise_10distinctobj_surf_agent run logged with detailed logging. Maybe another existing experiment would have been better for this. I don’t know.

  • interactive_objects_evidence_over_time fails with KeyError: 'max_evidence'
  • interactive_hypothesis_space_correlation fails with KeyError: 'hypotheses_updater_telemetry'

It looks like these plots expect fields that my run’s detailed_run_stats.json doesn’t contain, and I can’t find any doc that says which experiment or logging config produces them.

Could someone help direct me with:

  1. Which experiment / config produces the data these tbp.plot visualizers need (e.g. max_evidence, hypotheses_updater_telemetry)? Is there a specific setup used for the video?
  2. Any logging setting required beyond the standard detailed logging presets?

Thanks!

— Bryce

(cc @rmounir in case this is squarely your area — apologies if I should’ve asked elsewhere.)

3

Hi @Bryce_Bate,

I’m glad you got tbp.teleop to work, tbp.plot is a bit more tricky because there are a few places in the configs that need to be changed to generate those very detailed logs. I have created a branch on my fork with two example configurations:

  • bryce_randrot_noise_10distinctobj_surf_agent : can run with the run_parallel.py script.
  • bryce_unsupervised_inference_distinctobj_dist_agent: should run with the run.py script.

You can use the bryce_configs branch to run the experiments. The changes I made for these to generate the correct logs are:

After you run the experiments, the detailed logs should have all the data it needs to display these interactive plots. Note that the experiment I showed in the walkthrough video is an unsupervised inference experiment; you can change the burst sampling parameters by overriding the defaults under hypotheses_updater_args to get different burst sampling behaviors, e.g., make it less likely to trigger bursts.

Hope this helps, let me know if you’re still facing problems.

4

Thanks so much, @rmounir, for the assistance and quick response. I’ll give it a try. Nothing has helped me more to finally understand reference frames, hypotheses, poses, and object identification in TBT more than your video walkthrough of the visualizer. Suddenly, a number of concepts clicked for the first time. That’s why I was eager to play with it on my own.

2 Likes
5

You’re very welcome @Bryce_Bate, really glad to hear that! :slight_smile: