Hi all,
I wanted to share a small, constraint-oriented paper that came out of me trying to clarify something that’s often left implicit in discussions of grid-like representations, internal simulation, and Thousand Brains-style models.
In spatial settings, displacement is explicit: velocity signals, oscillatory-interference style mechanisms, or sensorimotor loops drive movement along an attractor manifold, while the attractor itself stabilizes and integrates state. In more abstract settings (e.g. internal simulation, replay, or non-spatial “grid-like” representations), that displacement-like structure is often assumed to come from elsewhere: simulated sensory activity, higher-level control, or another interacting system.
What I explored was a deliberately stripped-down question: if that displacement structure is left implicit or withheld, what does local recurrent attractor learning converge to on its own? Not as a biological claim, but as a way of understanding what work the attractor is, and isn’t, doing inside a larger architecture.
The short answer: it reliably finds a shortcut.
Across simple successor tasks, unless evaluation explicitly enforces late-horizon stability, learning converges to what I ended up calling impulse cheating: the network briefly flashes the correct successor state to satisfy the loss, but the activity collapses during free-run. You get the right answer for the wrong dynamical reason.
Only when persistence is explicitly tested do genuinely self-sustaining dynamics emerge, and even then they are strongly shaped by geometry (a continuous ring works; a folded “snake” manifold fails at discontinuities). The geometric result itself isn’t the main point. It mostly serves to illustrate how much can be hidden until transient solutions are ruled out.
I don’t think this contradicts how anyone expects biological systems to work. If anything, it sharpens the intuition that stable internal simulation requires explicit displacement-like structure somewhere in the system, and that local attractor recurrence alone is strongly biased toward predictive shortcuts unless constrained otherwise.
Sharing this mainly as a boundary-setting result, and as a reminder that prediction accuracy can quietly hide non-attractor dynamics unless persistence is measured directly.
Paper + code here:
https://github.com/javadan/can-paper
- Daniel
