Platform Current Reality Tree

[tslominski]

The Thousand Brains Project being open-source means not only that we open-source our code, but also that, as much as we can, we open-source our processes and planning. To that effect, I wanted to share with you my current view of the Monty platform, in the form of Platform Current Reality Tree.

What is a Current Reality Tree?

Definition

A Current Reality Tree (CRT) is a logical structure designed to depict the state of reality as it currently exists in a given system. It reflects the most probable chain of cause and effect, given a specific, fixed set of circumstances. (…) It’s functional rather than organizational, blind to arbitrary internal and external system boundaries.^[1]

Purpose

• Provide the basis for understanding complex systems.
• Articulate undesirable effects (UDEs) exhibited by a system.
• Relate UDEs through a logical chain of cause and effect to critical causes.
• Identify the critical causes that produce a majority of the system’s UDEs, including the worst ones.
• Isolate those few causal factors – system constraints – that must be addressed in order to realize the maximum improvement of the system.
• Identify the simplest potential changes that will have the greatest positive impact on the system.^[2]

How to read a Current Reality Tree?

Read the tree in the direction of the arrow. Each arrow points from a cause to an effect. Each part of the tree can be verbalized in sentence. For example, with the arrow point from entity (I turn off the light.) to an entity (The room is dark.), one would verbalize it as “If I turn off the light, then the room is dark.”

A round oval over the arrows means that all the causes are required to have the desired effect, an all-or-nothing cause and effect relationship. For example, with three entities on the bottom row (I turn off the light.), (The only door into the room is closed.), (The room has no windows.) all with arrows pointing to the entity (The room is dark), where those arrows are under a round oval, would be verbalized as “If I turn off the light, and the only door into the room is closed, and the room has no windows, then the room is dark.”

A MAG symbol over the arrows means that there is a magnitudinal cause and effect relationship between the causes and the effect. The causes independently produce the effect with some magnitude. Removal of any one eliminates some (not all) of the effect.

When arrows are indepent of each other, this implies independent cause and effect relationships. Each cause independently causes the effect and both must be removed to eliminate the effect.

The entire Platform Current Reality Tree is quite large to take in all at once. Below is a (collapsed) list of all the Undesirable Effects listed in the tree.

Undesirable Effects as of 4 Sep, 2025

• 271 Monty users are unclear about how to use Monty in different applications.
• 272 We don’t know if tutorials break when tbp.monty code changes.
• 273 Tests in tbp.monty/tests/unit are slow.
• 276 Platform is not integrated with a simulator that runs on Windows.
• 277 Platform requires unsupported Python 3.8.
• 279 Platform is distributed via conda.
• 278 Monty cannot be installed from PyPi via pip install tbp.monty.
• 239 Multiple components control the motor system.
• 109 Platform motor system and policies are entangled with the environment, data loader, and dataset.
• 242 Some people conceptualize the Platform as Deep Learning instead of Reinforcement Learning.
• 144 Configuring the Platform is complex.
• 108 Customizing Dataloader and Dataset for new environments is complex.
• 137 To use the Platform outside of tbp.monty repo requires configuration and run boilerplate.
• 110 Platform main loop is more complex than necessary.
• 152 Experimental framing is hardcoded into the Platform.
• 245 Platform main loop requires iteration by an external controller.
• 116 Platfrom detailed logging explodes memory.
• 103 Platform internal representations are difficult to visualize.
• 104 Platform logging is not sufficiently configurable.
• 134 Platform adds preconfigured logging handlers by default.
• 135 Platform logging is unstructured.
• 245 Platform main loop requires iteration by an external controller.
• 250 Platform uses two quaternion conventions (WXYZ and XYZW).
• 106 Monty system, Experiment, and Environment are all coupled together.
• 374 Implementing the Cortical Message Protocol (CMP) requires understanding of how tbp.monty works.
• 379 Implementing the Cortical Message Protocol (CMP) is toilsome.
• 118 I have to split one sensor state into two that use .depth and .rgba suffixes.
• 122 Inconsistent naming convention between configuration and use in code of learning module and sensor module names.
• 121 Module ID initialization is inconsistent between the sensor and learning modules.
• 212 Platform can’t control most robots.
• 207 We do not clearly understand how Monty should interact with multiple end-effectors.
• 251 Platform uses untyped dictionaries for the majority of data.
• 112 Coordinate system transforms are challenging to reimplement.
• 117 Platform has no legible performance profiling or metrics (e.g., wall-clock duration, CPU load, GPU load, memory load, network load, etc.).
• 100 I can’t provide a custom noise regime for sensor modules.
• 381 Platform API is unstable.
• 113 Platform is available only on x86_64 architecture.
• 115 Most experiment episodes take more than 1 second to execute.
• 371 Updating published benchmarks is toilsome.
• 402 MontyForGraphMatching can be configured with an incompatible policy.
• 120 BasePolicy is not compatible with MontyForGraphMatching model.

To find each one on the tree, press CTRL+F (or CMD+F) to open the search menu, and search for the entity text. You can then click on the search result to be taken to that portion of the tree.

Another way to conceptualize the Platform Current Reality Tree is that it is in the “problem space.” Ideally, you will see no solutions listed. This is one of the benefits of a Current Reality Tree, in that it allows us to focus on what is causing the problems without overindexing on the first solution that comes to mind.

How can you contribute?

The Current Reality Tree is intended to highlight existing, factual, cause and effect relationships. If you feel something is not a fact, if you feel the cause and effect relationship does not make sense, if you feel an Undesirable Effect exists but is not present, let us know.

I’m not sure of the best format yet. You can start by responding to this post and we’ll figure it out over time.

What’s next?

With the Platform Current Reality Tree in its initial iteration, next up I’ll be sharing a Platform Future Reality Tree. That’s where we will swap Undesirable Effects for Desired Effects, and figure out what actions we need to take to make the future reality happen. These actions will highlight some specific contribution opportunities to move the platform forward and some of them may expand into project plans (Prerequisite Trees).

For a small example of how Current Reality Tree + Future Reality Tree + Prerequisite Tree work together, you can take a look at Prototype: HypothesesUpdater operations on CUDA GPUs.

---

1. Dettmer, H. W. (2007). The Logical Thinking Process: A Systems Approach to Complex Problem Solving (2nd ed., p. 92). ASQ Quality Press. ↩︎

2. Dettmer, 2007, pp. 93-94 ↩︎

[carver]

I was checking out the quaternion convention conflict in UDE 250 & DE 314. As I dug into the first couple of steps, I found a related effect. In case you want to note it in the future reality tree, maybe leading into 317:

• scalar_first is only available after scipy==1.14.0
• scipy==1.14.0 requires Python >=3.10

[tslominski]

Thank you for looking into it… and of course it requires higher Python version .

I updated the Future Reality Tree and made DE 314 dependent on DE 123.

[Rich_Morin]

I’m a huge fan of progressive disclosure, so I was delighted to find out (from your usage above) that this forum’s Markdown implementation supports “turny triangles”. FWIW, here’s a sample of the relevant syntax:

[details="Foo"]
disclosable markup ...
[/details]

and the generated result:

Foo

disclosable markup …

[Rich_Morin]

I realize that the Platform Current Reality Tree (PCRT) is supposed to lay out issues, rather than suggest solutions. However, I don’t know how to do this (:-). Also, I have very little familiarity with Monty’s code base, so I may well get some things worng [sic]. So, please bear with me…

IMHO, a number of the issues shown in the PCRT are caused by failures to follow best practices, e.g.:

• Connascence of position should be minimized.

  Monty uses position-based structures (e.g., arrays, lists) in places where it could use keyword-based structures (e.g., maps). This can cause problems when elements are added, changed, communicated, removed, shared, etc. For example:

  • UDE 250: Platform uses two quaternion conventions (WXYZ and XYZW).

  • UDE 383: Cortical Message Protocol (CMP) changes as tbp.monty changes.

• Cohesion should be maximized; coupling should be minimized.

  In an Actor-based system (such as Monty is expected to become), this is accomplished by keeping processes small and tightly focused, then having them (mostly) communicate via messages. Unfortunately, some design choices have made this difficult:

  • As an object-oriented programming (OOP) language, Python has poor support for concurrency, let alone distributed computing. For example, program state tends to be stored inside opaque objects. So, if an object is being used by multiple threads at the same time, data corruption can occur.

  • Each cortical column (CC) is made up of many mini-columns, with multiple levels, many neurons, and even more synapses. A direct, actor-based emulation of a CC would thus require a huge number of (lightweight) processes.

    Each of Monty’s Learning Modules (LMs), in contrast, emulates an entire CC. This is a pragmatic design choice: even the BEAM can’t handle 15 million neurons, let alone 150 trillion synapses.

    However, this design choice has strong architectural ramifications: When it combines multiple biological entities into single digital “equivalents”, Monty is forced to depart from the actor model and employ other approaches (e.g., iteration, recursion).

  In short, we can expect some fairly major “speed bumps” as Monty is brought into production. These could be minimized (though not eliminated) by prophylactic changes during the prototyping phase.

• APIs should try to use broadly adopted standards.

  There are a number of popular standards that Monty could (and IMHO, should) use. For example:

  • UDE 127: Collecting attribute telemetry data for visualization requires a bespoke code change.

    As mentioned above, all program state in OOP designs is hidden within objects, making it necessary to create accessor methods, etc. In Elixir, the state of an actor is passed in and out as part of the process call, making it freely available for inspection, reporting, etc.

  • UDE 128: Selecting attribute telemetry data for visualization requires a bespoke code change.

    If Monty’s LMs (etc) supported GraphQL, any other actor could select (i.e., specify, request) desired data.

  • UDE 215: We don’t know what robotics API to integrate with.

    This issue is a special case of a much larger issue. There are many effectors, sensors, and other resources that Monty should be able to use. Choosing a robotics API might address one of these, but it’s only a partial solution.

    The Model Context Protocol (MCP) could be used to communicate with LLMs, service providers, etc. Because the MCP will be supported by a large range of resources, Monty can simply “come along for the ride”.

    Although it’s unclear how issues such as addressing, efficiency, and security will be handled as MCP develops, a clean adoption of MCP should let Monty take advantage of whatever solutions emerge.

[tslominski]

Hi @Rich_Morin,

A minor note.

@Rich_Morin

In an Actor-based system (such as Monty is expected to become)

This is not the intention of the Thousand Brains Project. If I implied this somehow, then I apologize, that was not my intent.

[tslominski]

Current Reality Tree Update - Motor System is decoupled from Environment Interfaces.

This is a short video update on the Current Reality Tree latest changes due to achieving the Desired Effect (105 Platform’s motor system and policies are decoupled from EnvironmentInterfaces.)

2026/03 - Current Reality Tree Update - Motor System is decoupled from Environment Interfaces

[tslominski]

Current Reality Tree Update - Linking (113 Platform is available only on x86_64 architecture.) to the HabitatSim part of the tree.

In our engineering meeting today we expanded a bit on of the chain of causes that limits our availability to the x86_64 architecture.

[tslominski]

Current Reality Tree Update - Updates to undesirable effect (371Updating published benchmarks is toilsome.)

In today’s engineering meeting, we made updates to the current state of benchmarks.

[tslominski]

Current Reality Tree Update - (402 MontyForGraphMatching can be configured with an incompatible policy.) and (120 BasePolicy is not compatible with MontyForGraphMatching model.) undesirable effects are addressed and removed.

In today’s engineering meeting, we noticed that these undesirable effects were addressed in scope of the work of decoupling the motor system from the environment.

As a result, the below part of the tree is removed entirely as it no longer reflects the current reality.

[tslominski]

Current Reality Tree Update - (411 Benchmarks do not provide enough confidence that results generalize.)

In today’s engineering meeting, we started to think through about our benchmark infrastructure. On the one hand, the benchmarks are reproducible (Desired Effect). On the other hand, we’re wondering whether they are exploring enough of the parameter space.

The wording of (411 Benchmarks do not provide enough confidence that results generalize.) is an initial draft and will probably require some updates. In a way, the benchmarks do provide us enough confidence to keep going on with the experiments and theory. So the current wording is probably not precise enough as to what we’re trying to express.

Currently, we’re coming at it from asking whether we need to engineer better benchmarking in order to support the project goals. But, as is the case with current reality trees, we first want to identify the precise undesirable effect we think we want to mitigate.

[tslominski]

Current Reality Tree Update - (200 Code duplication between sensor module implementations.)

In today’s engineering meeting, @jshoemaker brought up that we’re seeing code duplication across Sensor Module implementations. We started thinking through the reasons for code duplication.

[tslominski]

Current Reality Tree Update - (137 To use the Platform outside of tbp.monty requires configuration and run boilerplate)

With the changes to use Hydra for configurations and integrating with the hydra plugin discovery mechanism, using tbp.monty as a dependency outside of the repository is now much simpler.

Some of the previous facts are no longer true:

• (142 Platform benchmakr configurations are outside the tbp.monty source tree). These are now inside the source tree and shipped with the code in src/tbp/monty/conf.
• (141 Outside of the tbp.monty repo, experiment configuration is assembled from framework experiments, environments, simulators, and config_utils). Since we stopped relying on inheritance and because existing tbp.monty configuration is discoverable by Hydra, configuring an experiment outside of the repository can reuse the same components as existing experiments, or new ones can be added.
• (146 Platform configuration relies on inheritance hierarch). We removed inheritance hierarchy from our configuration.
• (147 Platform configuration is deeply nested). This was correlated with an inheritance hierarchy use, and since that’s gone, this is also no longer true.

With all of the above changes, the Undesirable Effect (137 To use the Platform outside of tbp.monty requires configuration and run boilerplate) is mitigated to the point that we won’t track it any futher unless something new and material surfaces (or if I made a mistake in getting rid of it).

The below image highlights these changes on the Current Reality Tree.

With the updates above, the remaining change is rewriting (143 Experiment configuration requires a minimum of…) fact. It turns out that with using composition over inheritance, we do end up with more configuration entries. I still believe this is the right direction for now, as all of these entries compose in a straightforward way without overriding or otherwise interacting with each other.

Lastly, with the Undesirable Effect (137) removed, (143 Experiment configuration requires a minimum of…) now contributes some magnitude to the Undesirable Effect (144 Configuring the Platform is complex).

Note: The consequent changes to the Future Reality Tree are pending.

[tslominski]

Current Reality Tree Update - (200 Code duplication between sensor module implementations.)

After a discussion during today’s engineering operations meeting, I added some causal arrows and updated some of the entities. Additionally, I think (204 Code explicitly specifies the transform sequence for each module.) is a Critical Cause if these relationships are valid.

[tslominski]

Current Reality Tree Update - (174 Sensor modules have pre_episode methods.)

Previously, 174 was (174 sensor_modules.Probe has pre_episode method.). We observe that this can now be encompassed entirely by (Sensor modules have pre_episode methods.), so the two are consolidated into the new (174 Sensor modules have pre_episode methods.).

Additionally, we surface a new entity that is a contributing cause to why telemetry collection has episode time quantization. Telemetry collection piggybacks on the state_dict() serializability function to retrieve telemetry data. So far, we only depict this being a cause of sensor modules having pre_episode, but I wouldn’t be surprised if we find state_dict() being used in other places for telemetry collection.

[tslominski]

Current Reality Tree Update - EnvironmentInterfaces do not cause Environment undesirable effects.

In the previous engineering meeting, we observed that some items listed as causes were not, in fact, causes. Since originally creating the current reality tree depicted, it became clear that EnvironmentInterface is an experiment artifact and not part of the Monty platform. As such, it is not something that passes its constraints to the Environment. In fact, as of this writing, beyond the fundamental .step(...), there is no other time quantization present in the Environment or simulators.

The update to the current reality tree recognizes the above and cleans up the causes.

It is still true that EnvironmentInterfaces contain episode and epoch time quantization, but that’s ok since they’re part of experiment code, which can have episodes and epochs.

[tslominski]

Current Reality Tree Update - (200 Code duplication between sensor module implementations.)

After further consideration, while true, some of the entities in the tree do not directly cause the actual undesired effect. So, I removed them from the Current Reality Tree.

With these changes, I believe the causes of the undesirable effect are clearer.

[tslominski]

Current Reality Tree Update - Undesirable Effect (273 Tests in tbp.monty/tests/unit are slow) is eliminated.

As highlighted in Platform Future Reality Tree - #12 by tslominski update, the unit tests now take only about 20 seconds to run. The Undesirable Effect (273 Tests in tbp.monty/tests/unit are slow) is now gone and the associated causes are no longer part of the Current Reality Tree.

Current Reality Tree Update - Undesirable Effect (293 Monty integration tests take more than 10 minutes to run on GitHub Actions.)

With integration now being separated, the new Undesirable Effect is (293 Monty integration tests take more than 10 minutes to run on GitHub Actions.)