Hi folks, I’m a human-AI systems architect at Boeing. I’m using the Thousand Brains reference frame framework to model the physiological cost of human-AI interaction.
My core argument is that when cortical columns attempt to build reference frames for AI-generated content, the lack of compressed human intent means the reference frames never stabilize. The brain burns its metabolic budget searching for a structure that isn’t there. I formalized this as a MaxEnt IRL failure under the Free Energy Principle.
One of the challenges of cognitive science research is that the practitioners are all talking about the same subject, using the same language, but the words used have vastly different meanings to the researchers depending on their academic background.
I think about spent about an hour trying to decode your abstract, mostly trying to build a bridge between your psych-based perspective and by CS-based perspective. I’m not complaining or anything. I found just that short experience very insightful.
I see that you are a subscriber of Schmidhuber’s mathematical definition of beauty, which I found fascinating, but struggled to find a practical use for. I like how managed to find a human-factors application in the meatspace world
The translation problem you’re pointing at is actually the reason this paper exists at the length it does. Every field has its own compiler for “intentionality” or “reward function” or “trajectory,” and the abstract is trying to do simultaneous translation across three of them (active inference, IRL, human factors) in about 200 words. I’m not surprised it took an hour (sorry). I’ve spent an incredible amount of time on it, and at this point it’s even somewhat of a self-referential intentionality-stack itself.
Re: Schmidhuber: yeah, that’s the reference I lean on hardest and it’s also the one I think is most under-deployed in practice. His “interestingness = compression progress” framing is gorgeous mathematically, but it ends with “here is a signal.” The question I kept running into is what you do with that signal once you have it, especially when the agent producing the interesting artifact is no longer a biological agent with a latent reward function to extract. Ghost Scale is basically the answer I landed on for the human-factors side. You don’t fix the viewer’s generative crash by making the model more interesting. You give the observer a cognitive affordance that lets them stop looking for intent that isn’t there. Same move applies on the alignment side via CIRL, just with the machine as the observer. And then, when applying this framework to world models, I also point to a long-term solution in teaching AI to interpret and speak in this same language of intentionality - removing the problem entirely.
Happy to get deeper on any of this here or elsewhere. Your BrainBlocks and HTM lineage (pulling from your email a bit, sorry) is probably the cleanest architectural fit for the intent-extraction side of what I’m describing, and I’d be curious how you see the reference-frame-instability argument landing against your implementation experience.
Current “AI” extracts information from knowledge and discards the meaning. It’s no longer knowledge, it just looks and feels like it. Dunning of D&K said that we are all carrying around a collection of garbage and random trivia that feels to us like knowledge. None of us are immune, and he admits that while the effect is repeatable his conclusions could be completely wrong.
Now we have “AI” that is designed to mimic humans and it is feeding us information from the pile of labeled garbage it has been fed. You can’t build knowledge strictly on information. It has to have meaning. LLMs are a stupidity amplifier that makes us feel smarter while what little knowledge we started with is gradually replaced by disconnected meaningless junk that may or may not be true. You can’t build knowledge without prerequisite knowledge, and current “AI” has none.
