We need better tools to ‘see’ the cortex: A measurement-first path

The Thousand Brains Theory makes concrete, testable claims about how the
neocortex represents the world: reference frames, repeating cortical columns, voting across columns, ….

What I keep running into isn’t a shortage of
theory to test. It’s that our instruments can’t yet watch the cortex represent anything at the resolution the theory actually operates at. So
lately I’ve become at least as interested in the tools as in the models.
I want to put that interest on the table here, plainly, and see who else is thinking about it, because I don’t think this is something any one person or
lab solves alone.
The walls everyone knows. Every non-invasive method trades spatial resolution for temporal resolution:
• EEG / MEG measure electrical activity directly at millisecond timescales, but recovering where the signal came from is a mathematically ill-posed inverse problem, and the spatial picture stays coarse.
• fMRI gives millimeter, whole-brain 3-D coverage, but it measures blood flow as a slow surrogate for neural activity, not the activity itself. It
physically can’t resolve fast electrical events.

The dream that would collapse this trade-off, is to directly imaging neuronal electrical activity with MRI’s spatial coverage and it looked briefly real in 2022 (DIANA), and then didn’t. The DIANA results were retracted in 2025 after multiple
groups failed to replicate it. I raise that not to dunk on anyone, but because it’s the most honest recent lesson in this space: the holy grail is
genuinely hard, and wanting it badly isn’t enough.
The real current frontier seems to be OPM-MEG (wearable, motion-tolerant magnetometers) and MEG–fMRI fusion (which already produces millisecond, millimeter “movies” of cortical activation) it’s impressive but does not reach the individual neuron.
My stance: measurement-first with minimal starting assumptions. Here’s the position I’ve landed on, and I’d genuinely like it stress-tested.

  • Don’t presuppose how the brain works in order to build the tool that’s supposed to tell you how the brain works.
 - An instrument specified from a
theory of representation can only ever confirm that theory’s assumptions. 

So, I want to specify the instrument by observables and their resolution:

  • what physical quantity

  • what spatial and temporal scale

  • over what coverage

    I try to say nothing, in the spec, about what those observables mean cognitively. Ideally, the goal is to let the measurement adjudicate between theories instead of quietly encoding one.
    Two things I’m deliberately not willing to assume:

    1. That single-neuron resolution is forever impossible non-invasively. It
      isn’t demonstrated, but a present-day limitation is not a physical law. I’ve watched “we can’t” get quietly upgraded to “we never could” too often. The honest move is to separate the walls that are engineering
      (steep, not forbidden) from the ones that are theorems (provable barriers you must route around, not through).

    As far as I can tell, only one wall in this whole problem is genuinely theorem-grade:

  • You cannot uniquely reconstruct internal currents from field measurements taken entirely outside the tissue. Everything else looks like engineering to me.

    1. The tool has to be a single instrument. This is more of an intuition, I believe the solution will involve a fused array, multiple observables, and almost certainly some sensing element inside the tissue, because going in-situ is precisely what escapes that
      one theorem.
      The path (how I’d actually chase it),not a finished design, but a data driven iterative loop:
      • Attack one wall at a time. For each wall: define it, decide what a proof-of-concept can achieve against it, specify what data would breach it.
    • build the minimal device, collect, correlate against an independent ground truth, adjust, move on.
  • Wall-ordered, not calendar-ordered.

  • Small-mammal-first, but honest about it. A mouse removes several of the hard walls (thin skull, ~1 cm depth, whole-cortex-at-once is nearly routine). That’s useful, as long as you track which walls a given experiment actually stresses versus which the small brain quietly relaxed.

    A mouse can’t flatter the walls that scale back up against a human, so those are the ones a mouse result can honestly validate.
    Decompose the proof-of-concept the way you’d decompose any hard system:
    bench/dish → slice → animal → up, each stage adding back exactly one wall, so a null result tells you which wall broke it.
    • Keep the end goal as the design constraint, human, whole-neocortex even while the first experiments are deliberately small.

    What I’m asking the community: Genuinely open questions, and I’d rather argue about them here than in private:

  • What tool do you wish existed for your own work with Monty / TBT?

  • Where is the current instrumentation the thing standing between you and a test
    you want to run?

  • Who here works on measurement rather than modeling: imaging, sensors,
    materials, inverse problems and would be willing to poke holes in the wall-by-wall framing above?
    • Is there prior art I should be arguing against instead of my own summary? If so, please point me at it. I’m doing this in the open public research vault to hopefully generate interest and collaboration. I treat all of the
    above as working hypotheses, not conclusions. If any of it is wrong, I’d
    rather find out fast. Happy to take this to a call with anyone who wants to go deeper.

1 Like

Hi Roger

Sorry I cannot help with the scanning question, not my field, but I do wonder how much we can figure out from scans. For sure we can learn a lot, but is it not rather like monitoring every transistor and memory cell in a computer motherboard and trying to reverse engineer Windows from the data ?

In another analogy, if we drag a window across the screen or run a particular program certain areas of the motherboard will ‘light up’ indicating their involvement in the task but it will not tell us how the computer actually carries out its task.

We may have to be content with using the clues from scans to build systems that behave in a similar way without ever knowing how a biological brain does it. But perhaps that’s the defeatest in me :slight_smile:

All the best

Alex

Tools like I am proposing are certainly going to create a highly complex set of Data, but with the help of existing AI systems I am confident that the data will eventually provide us with the geometries, correlations, and eventually relationships needed to significantly improve our understanding of BNNs. I am trying not to make anymore assumptions than necessary to build it, but I certainly hope it can provide new insights that will lead to a deeper understanding with the ability to prove those relationships and geometries.

Hi @Roger_Henley thanks for your post, these are interesting questions to think about.

This is indeed a very challenging area, but like you say, often “we can’t” eventually becomes “we never could”, so there’s room for optimism I think.

In terms of “what tool”, I wanted to highlight the appendix in our updated Thousand Brains Theory 2.0 paper (this is an update to the “Heterarchy or Hierarchy” paper that is on ArXiv), which I attach. In particular, we discuss the requirements and challenges around finding evidence of reference frames in cortical columns. As you can see, many of the constraints are not strictly imaging related, but bring in other challenges.

appendix.pdf (120.4 KB)

If I had to summarize the state of the art and how I could imagine it evolving in the coming years to address this gap, it might look something like this: optical techniques that are able to measure the activity of 10s or 100s of thousands of individual neurons simultaneously, and down to the depths of layer 6. Temporal resolution would need to be at the level of ms (or at least 10s of ms). Work that is moving in this direction includes the research from Alipasha Vaziri. Another line of research you might find interesting is the work from the “RoLi” lab, in particular developing a microscope that tracks and moves with freely swimming zebra fish in 3D space.

In short, there is a lot of impressive optical work being developed that can measure from many neurons, very fast, and all in an awake, behaving animal. The techniques aren’t perfect, but together with recent successes in making otherwise opaque tissues transparent, this is probably the research area that seems most promising to me.

Hope some of that is useful, I’m interested to hear what other methods you come across.

@nleadholm thank you for this reply. It gives me material that I was unfamiliar with and I appreciate it. I have recently resigned my position as VP of Engineering & Technology to have more time to devote to this topic. My intention is to self-fund research on the topic, with a somewhat lofty goal of designing a POC to build. At that point I would seek additional funding. I will review the content you provided and post back to this thread with any updates or insight it provides. I am also keeping everything in a public GitHub research journal that I will provide a link to at that time.

1 Like

Oh wow, that’s exciting, I wish you the best of luck on your venture. Yes please let me know if you have further questions - experimental methods are not my area of expertise, but if nothing else, I might be able to point you towards some useful resources.