Platform Future Reality Tree

The Thousand Brains Project being open-source means not only that we open-source our code, but also that, as much as we can, we open-source our processes and planning.

Previously, I shared with you the Platform Current Reality Tree, where I described how I viewed the Monty platform and the Undesirable Effects at the time.

Today, I share with you the Platform Future Reality Tree. The Platform Future Reality Tree is similar to the Platform Current Reality Tree; the primary difference is that it includes Injections. An Injection is an action we take to change the current reality into a desired future reality. Aside from that, instead of leading to Undesirable Effects, most of the tree leads to Desired Effects, which is the future we want.

The entire Platform Future Reality Tree is quite large to take in all at once. To find items of interest in it, press CTRL+F (or CMD+F) to open the search menu, and search for the entity text. You can then click on the search result to be taken to that portion of the tree.

How can you contribute?

The Injections listed on the Future Reality Tree are projects that we’ll need to complete to achieve the desired effects. Some of them may be single pull requests, while others may be large, multi-month efforts. It depends on the nature of the Injection.

If you are interested in implementing an Injection, let us know here or on GitHub if you need additional context or advice on how to proceed. Or surprise us with a Pull Request without saying a word :smiling_face_with_sunglasses:.

What’s next?

We are working on listing all Injections in our Future Work section in the documentation for easier discovery. There’s a lot to write, so it may take some time to update. The Platform Future Reality Tree is likely to be the most up-to-date artifact.

6 Likes

I think my favorite part is the open source of the process and planning.

4 Likes

After a year of work, we achieved the Desired Effect (105 Platform’s motor system and policies are decoupled from EnvironmentInterfaces.)

It is one of the big three for fully decoupling Experiment, Platform, and Environment from each other.

3 Likes

Future Reality Tree Update - Coordinate Transforms might partially address Platform quaternion convention.

At our engineering meeting today, we added a possible relationship between (DESIRED EFFECT 260 Coordinate system transforms are easy to use.) and (DESIRED EFFECT 314 Platform only uses the WXYZ quaternion convention.)

1 Like

Future Reality Tree Update - Coordinate Transforms unlikely to address Platform quaternion convention

Over time we highlighted other uses of quaternions that are outside the context of reference frame transforms. For example, when motor policies compute where to look, quaternions are used in the context pose transforms directly, outside of reference frame context.

1 Like

Future Reality Tree Update - (339 MontyForGraphMatching is decoupled from a specific motor policy implementation.)

When reviewing the Current and Future Reality Trees during today’s engineering meeting, we noticed that we achieved the Desired Effect (339 MontyForGraphMatching is decoupled from a specific motor policy implementation.) in scope of other work on decoupling the motor system from the environment.

2 Likes

Future Reality Tree Update - (367 It is unclear how to run tbp.monty experiment from outside the tbp.monty repository (e.g., as a conda dependency))

The Undesired Effect (367 It is unclear how to run tbp.monty experiment from outside the tbp.monty repository (e.g., as a conda dependency)) is now resolved.

We figured out how to do this. The template repository for using tbp.monty in your project, prototype, or paper is the best example: GitHub - thousandbrainsproject/tbp.monty_project_template: Quickly use Monty for your project, prototype, or paper. · GitHub. Additionally, we are updating our tutorial documentation to include similar instructions: docs: update tutorial to run experiment from a different repository by tristanls-tbp · Pull Request #968 · thousandbrainsproject/tbp.monty · GitHub.

2 Likes

Future Reality Tree Update - (198 Sensor Modules are free of episode time quantization.)

As a consequence of Platform Current Reality Tree - #17 by tslominski update, a similar update is made in the Future Reality Tree. (198 Sensor Modules are free of episode time quantization.) is consolidated to include the Probe sensor module.

Additionally, we include (201 Telemetry collection relies on state_dict() to retrieve telemetry.) entity which makes it more apparent why (Injection #81 Use structured logging to emit telemetry inline) is beneficial to achieve our desired effects.

Future Reality Tree Update - (219 Experiment and Platform no longer have Dataset and Dataloader.)

This is a big update, but it’s less about work completed and more about clearer understanding. The main insight was realizing that EnvironmentInterface is experiment code and not part of Monty nor Environment themselves. As such, any constraints in EnvironmentInterface do not count as constraints for the Environment. This allowed us to remove some causal arrows and it turns out that (204 Environment is free of episode time quantization.) and (213 Environment is free of epoch time quantization.) have been true for a while.

Similarly, since EnvironmentInterface code is experiment code, we do not need to get rid of pre|post_episode and pre|post_epoch from them. This lets us get rid of (Injection #41), (Injection #44), and (Injection #42). We also added the important fact that (240 EnvironmentInterfaces are part of Experiment code).

With the injections gone, we can remove the now-false expectations (208 EnvironmentInterfaces are free of the episode concept.) and (212 EnvironmentInterfaces are free of the epoch concept).

Similarly, getting rid of (Injection #45 Remove EnvironmentInterfaces) and other now-false expectations.

Lastly, highlighting that a lot of this has been accomplished for a while. Ultimately (219 Experiment and Platform no longer have Dataset and Dataloader.) has been accomplished and the Environment is not preventing the desired effect of (190 Platform is free of Experiment artifacts.)

1 Like

Future Reality Tree Update - (381 Creating a new Sensor Module adds only the necessary and sufficient code for the new functionality.)

Corresponding to the (200 Code duplication between sensor module implementations) Undesirable Effect, this new part of the Future Reality Tree outlines a path towards the (381 Creating a new Sensor Module adds only the necessary and sufficient code for the new functionality) Desired Effect.

Two sequential injections are identified (Injection #95 Refactor all Sensor Module functionalities into reusable transforms) and (Injection #96 Compose Sensor Modules from reusable transforms.).

Work on the RFC related to these injections is in progress: rfc: Sensor Modules as Transformation Pipeline by tristanls-tbp · Pull Request #1005 · thousandbrainsproject/tbp.monty · GitHub

Future Reality Tree Update - (186 Tests in tbp.monty/tests/unit are fast.)

It slipped through the cracks, but a while back in Q1 of 2026, thanks to the efforts of @TimothyAlexisVass et al., we achieved the desired effect of (186 Tests in tbp.monty/tests/unit are fast.) :partying_face:

We’ve been taking it for granted for a while, but nowadays, running pytest, which runs the unit and configuration snapshot tests, runs 591 tests in about 20 seconds

1 Like

Future Reality Tree Update - (Injection #35 Move Learning Module [pre|post]_episode methods to experiment) is now completed.

Thanks to @DSchumacher’s recent work, Injection #35 is now complete and (194 Learning Modules are free of episode time quantization).

1 Like

Future Reality Tree Update - Achieved Desired Effect (314 Platform only uses the WXYZ quaternion convention).

On May 7th, @sknudstrup came up with a novel (Injection #97 Introduce WXYZ Rotation wrapper), and with its introduction, achieved the desired effect without us having to wait for Python version upgrade and waiting for (Injection #74 Use scalar_first=True option with SciPy Rotation).

1 Like