Adam Marblestone — AI is missing something fundamental about the brain

Summary

Adam Marblestone, CEO of Convergent Research and former research scientist at Google DeepMind’s neuroscience team, presents a deeply technical thesis about what AI is missing from neuroscience. His core argument is that the brain’s “secret sauce” is not its architecture (the cortex is relatively uniform) but rather the elaborate system of innate reward functions, cost functions, and loss functions that evolution has encoded into what he and AI safety researcher Steve Byrnes call the “Steering Subsystem” — the brainstem, hypothalamus, amygdala, and other subcortical structures that provide the reward signals that train the cortex’s “Learning Subsystem.”

The conversation explores how evolution solved the problem of encoding high-level abstract desires (like “don’t embarrass yourself in front of important people”) into a genome of only 3 gigabytes — most of which is devoted to basic cellular machinery shared with yeast. The answer, Marblestone argues, is that evolution builds a rich set of innate but specific reward circuits: flinch responses, social shame detectors, curiosity drives, and hundreds of other “Thought Assessors” that the cortex learns to predict and therefore generalize from. This explains why the genome can be so small yet produce creatures with sophisticated social behavior — the reward functions are compact but the cortex does the heavy lifting of generalization.

The discussion ranges into the practical implications: why the cortex may be doing “omnidirectional inference” rather than simple next-token prediction (aligning with Yann LeCun’s energy-based model ideas); why mapping the mouse brain connectome is essential and achievable for low tens of millions of dollars with new optical microscopy technology; how formal mathematical proof systems like Lean could transform both mathematics and cybersecurity; and why understanding the brain’s reward architecture may be critical for AI safety, since a paperclip maximizer might be buildable with far less genomic complexity than a prosocial intelligence requires.

Highlights

”Evolution’s secret: reward functions, not architecture”

yt-dlp --download-sections "*1:18-2:20" "https://www.youtube.com/watch?v=_9V_Hbe-N1A" --force-keyframes-at-cuts --merge-output-format mp4 -o "_9V_Hbe-N1A-1m18s.mp4"

“My personal hunch within that framework is that the most underappreciated piece is the role of these very specific loss functions. Machine learning tends to like simple, clean loss functions. Predict the next token, cross-entropy. I think evolution may have built a lot of intelligence through many different loss functions at different stages of development.” — Adam Marblestone, 1:18

”The cortex might be doing omnidirectional inference”

yt-dlp --download-sections "*3:30-4:40" "https://www.youtube.com/watch?v=_9V_Hbe-N1A" --force-keyframes-at-cuts --merge-output-format mp4 -o "_9V_Hbe-N1A-3m30s.mp4"

“What if the cortex is natively doing something where it can predict any pattern in any subset of variables from any other subset? Omnidirectional inference. That is a little bit more like ‘probabilistic AI’ — extremely similar to what Yann LeCun would say.” — Adam Marblestone, 3:30

”How does evolution encode ‘don’t piss off important people’?”

yt-dlp --download-sections "*7:04-8:00" "https://www.youtube.com/watch?v=_9V_Hbe-N1A" --force-keyframes-at-cuts --merge-output-format mp4 -o "_9V_Hbe-N1A-7m04s.mp4"

“Evolution has never seen Yann LeCun. It’s never known what an important scientist or a podcast is. But it knows you should not piss off really important people in the tribe. It has to do this without knowing in advance all the things you might learn.” — Adam Marblestone, 7:04

”The genome is only 3 gigabytes — and most of that is shared with yeast”

yt-dlp --download-sections "*28:15-29:20" "https://www.youtube.com/watch?v=_9V_Hbe-N1A" --force-keyframes-at-cuts --merge-output-format mp4 -o "_9V_Hbe-N1A-28m15s.mp4"

“3 gigabytes is the size of the genome. Obviously a small fraction of that is devoted to neural stuff. We have a lot in common with yeast. So much of the genome is just going towards basic cellular machinery — it can get energy, it can replicate. The differences that matter include the social instincts.” — Adam Marblestone, 28:15

”A paperclip maximizer might need less genomic complexity than a prosocial agent”

yt-dlp --download-sections "*47:50-48:50" "https://www.youtube.com/watch?v=_9V_Hbe-N1A" --force-keyframes-at-cuts --merge-output-format mp4 -o "_9V_Hbe-N1A-47m50s.mp4"

“I’m concerned that the minimum viable set of instincts needed to make something smart is way less than the minimum viable set of instincts needed for ethics and social behavior. The Steering Subsystem is actually the hard part.” — Adam Marblestone, 47:50

”The mouse brain connectome is achievable for tens of millions”

yt-dlp --download-sections "*63:00-64:00" "https://www.youtube.com/watch?v=_9V_Hbe-N1A" --force-keyframes-at-cuts --merge-output-format mp4 -o "_9V_Hbe-N1A-63m00s.mp4"

“E11 technology and the suite of efforts around it could get the mouse connectome down to low tens of millions of dollars. A human brain is about 1,000 times bigger. With technology development, could we do a human brain for less than a billion? That’s not out of the question.” — Adam Marblestone, 1:03:00

Key Points

• Brain’s secret sauce is reward functions, not architecture (0:00) - The quadrillion-dollar question: what are AI models missing? Marblestone argues it’s the specific loss functions, not the learning algorithm or architecture
• Evolution encodes learning curricula through loss functions (1:57) - Evolution built intelligence by encoding many different loss functions at different developmental stages, generating specific curricula for what to learn
• Cortex as universal learning substrate (2:31) - The cortex has a relatively uniform six-layer architecture across all areas, but the question is what it’s being asked to learn — next-token prediction or something more flexible
• Omnidirectional inference vs. next-token prediction (3:30) - The cortex may natively do “omnidirectional inference” — predicting any subset of variables from any other — unlike LLMs which only predict the next token
• Learning Subsystem vs. Steering Subsystem (5:47) - Steve Byrnes’ framework: the cortex is the “Learning Subsystem” (general prediction engine) while brainstem/hypothalamus/amygdala form the “Steering Subsystem” (innate reward functions)
• Spider example: how abstract fears generalize (6:25) - The cortex learns to predict flinch responses (innate spider fear) and from there generalizes to abstract spider concepts, words, and situations
• Ilya’s question answered (6:32) - On Dwarkesh’s podcast, Ilya Sutskever asked how the genome encodes high-level desires. Marblestone argues Steve Byrnes has an answer through the Steering/Learning Subsystem framework
• Thought Assessors predict Steering Subsystem states (10:18) - The cortex trains “Thought Assessors” — predictors of whether the Steering Subsystem would fire for various situations. These enable generalization beyond the innate triggers
• Genome compression trick (28:15) - 3 gigabytes of genome (mostly shared with yeast) can produce sophisticated intelligence because the reward function in “Python pseudocode” is compact while the cortex does the heavy generalization
• Steering Subsystem has more cell-type diversity (30:00) - Single-cell atlases show the innate Steering Subsystem regions have far more diverse and bespoke cell types than the more uniform Learning Subsystem
• RL in the brain: model-free basal ganglia + model-based cortex (42:42) - The brain combines simple model-free RL (temporal difference learning in basal ganglia) with sophisticated model-based planning via the cortex’s world model
• Biological hardware advantages (50:31) - Despite running on only 20 watts, the brain co-locates memory and compute, uses stochastic neurons for natural sampling, and has evolved co-designed algorithms
• AI safety concern: simpler to build a paperclip maximizer (47:50) - A minimally viable intelligent system might need far fewer reward functions than a prosocial one, making dangerous AI potentially easier to build than safe AI
• Mouse brain connectome for $20-30M (1:03:00) - E11 Bio’s optical microscopy approach with molecular annotation could map the entire mouse brain connectome for low tens of millions
• Human Genome Project analogy (1:07:00) - Like the genome project, connectomics needs a technology-development phase that dramatically drops costs before attempting the full human brain
• Lean and formal verification for math (1:23:28) - The Lean programming language enables RLVR for mathematical proofs, creating a perfect verifiable reward signal that could revolutionize mathematics
• Provable software as cybersecurity tool (1:30:00) - Formal verification of code could produce provably unhackable software, a powerful cybersecurity application
• Brain-data-augmented training (1:35:00) - Gwern’s proposal: training AI with auxiliary loss functions that predict brain activity patterns could improve representation learning
• Architecture of the brain: still fundamental unknowns (1:38:18) - We still don’t know basic things like how many types of dopamine signals exist, whether prefrontal-auditory wiring is the same across species, or what the thalamus is actually doing

Mentions

Companies

• Convergent Research (description) - Adam Marblestone’s organization that incubates Focused Research Organizations (FROs)
• Google DeepMind (description) - Where Marblestone was previously a research scientist on the neuroscience team
• E11 Bio (1:03:00) - Focus Research Organization for connectomic brain mapping using optical microscopy
• Astera (18:30) - Organization funding a neuroscience project based on Doris Tsao’s work on visual representations

Products & Technologies

• Lean (1:23:28) - Formal proof programming language used for verified mathematical theorems
• AlphaProof (1:25:00) - DeepMind’s system based on formal proof verification for mathematical reasoning
• Energy-based models (15:42) - Alternative to backprop-based learning, potentially more brain-like
• RLVR (1:23:28) - RL from verifiable reward; formal proof checking creates a perfect RLVR signal for math

People

• Adam Marblestone (0:00) - CEO of Convergent Research, former Google DeepMind neuroscience researcher
• Steve Byrnes (6:27) - AI safety researcher whose Learning Subsystem/Steering Subsystem framework is referenced throughout
• Ilya Sutskever (6:32) - Asked the key question about how the genome encodes abstract reward functions
• Yann LeCun (4:45) - Energy-based models and omnidirectional inference vision aligned with Marblestone’s view of the cortex
• Doris Tsao (18:30) - Neuroscience work on visual representations that may inform more efficient AI architectures
• Peter Dayan (44:00) - Work on temporal difference learning and dopamine as prediction error signal
• Joel Dapello (17:42) - Created a model of V1 visual cortex incorporating retinal preprocessing
• Sam Gershman (description) - Work on what dopamine is doing in the brain
• Konrad Kording and Tim Lillicrap (1:00:00) - Paper “What does it mean to understand a neural network?” arguing full mechanistic understanding may be impossible
• Gwern (1:35:00) - Blog proposal on training AI with brain activity patterns as auxiliary prediction task
• Suzana Herculano-Houzel (31:30) - Research showing neuron count scales better with brain weight in primates than non-primates
• Max Bennett (description) - Author of “A Brief History of Intelligence” on connections between neuroscience and AI
• Joe Henrich (46:30) - Work on cumulative cultural evolution and how civilizations encode knowledge through traditions
• Andrej Karpathy (description) - Referenced episode on the podcast
• Richard Sutton (description) - Referenced episode on the podcast
• Fei Chen and Evan Macosko (29:00) - Single-cell atlas work revealing cell-type diversity in brain regions
• György Buzsáki (56:00) - Author of “The Brain from the Inside Out,” argues against mapping AI concepts onto the brain

Surprising Quotes

“The minimum viable set of instincts needed to make something smart is way less than the minimum viable set of instincts and ethics and social behavior. The Steering Subsystem is actually the hard part.” — 47:50

“I think the message I’m taking from this interview is that the people who get made fun of on Twitter — Yann LeCun and Beff Jezos — they’re kind of onto something about the limitations of current approaches.” — 53:00

“We have 3 gigabytes of genome. A small fraction of that is neural stuff. We have a lot in common with yeast. The reward function, in Python pseudocode, is maybe a thousand lines. So you can build a learning agent on compact reward specifications.” — 28:15

“Evolution has never seen Yann LeCun. It has never known what an important scientist or a podcast is. But it knows you should not piss off really important people in the tribe. It does this without knowing in advance all the things you might learn.” — 7:33

“The neuron just generates samples naturally. It’s like having the random number generator for free, whereas in a digital computer you have to write a random number generator in Python code.” — 52:30

Transcript

0:00 The big million-dollar question that I’ve been trying to answer through all these interviews with AI researchers is: we’re throwing way more data at these models than humans ever see, yet they capture only a small fraction of the total capabilities a human has. This might be the quadrillion-dollar question. You can make an argument that this is the most important question in the world. I don’t claim to know the answer. My overall meta-level take is that we have to just make neuroscience a more powerful science to actually be able to crack a question like this.

0:53 With modern AI, neural nets, deep learning, there are several components. There’s the architecture. There’s the properties of that architecture. How do you train it? Backprop, gradient descent. If we take the learning part of the system, it includes the learning rules. And then there are also cost functions. What’s the reward signal? What are the loss functions? My personal hunch within that framework is that the most underappreciated piece is the role of these very specific loss functions.

1:45 Machine learning tends to like simple, clean loss functions. Predict the next token, cross-entropy, these very general things. I think evolution may have built a lot of intelligence through many different loss functions at different stages of development, generating a specific curriculum for what to learn. Because evolution has seen many times what was useful. It can encode the knowledge of the learning curriculum.

2:23 Can different loss functions lead to different architectures? People say the cortex has got the universal learning algorithm that humans have. What’s up with that? I’ve seen models where the cortex has six physical layers in a slightly different arrangement. Any one location in the cortex has six physical layers. And those areas then connect to each other. I’ve seen versions of that where what matters is “How does it approximate backprop?”

3:09 What is the network being asked to do, if you think about it at the loss function level? Is it doing backprop on next-token prediction or something else? And no one knows. But one thought is that the cortex is just this incredibly general prediction engine. Can it learn to predict any subset of its variables from any other subset? Omnidirectional inference. Whereas an LLM is just seeing everything in one direction — a very particular conditional probability: given all the previous tokens, what are the probabilities for the next token.

4:05 You can ask an LLM to fill in “the quick brown fox blank blank” and it does okay because of the context window and everything. But what if the cortex is natively doing something much more flexible, where it can predict any pattern in any subset of variables? That is a little bit more like “probabilistic AI” — extremely similar to what Yann LeCun would say about energy-based models.

4:50 The difference between energy-based models and something like that is: what is the likelihood or unlikelihood of any configuration? If I clamp some variables and say these are the observations, then I can compute, with probabilistic sampling, what any other subset is going to do and sample from that. And I could choose a totally different subset and do inference. There could be parts of cortex that predict vision from audition.

5:42 The more innate part of the brain is the brainstem, the lizard brain. The cortex sits on top of a lizard brain. And that thing is a thing you’re not just predicting in one direction about. Is this muscle about to tense? Is my heart rate about to go up? Based on my higher-level understanding — like I can predict when this lizard-brain part would activate if there’s a spider. You learn to associate the two so that when someone says “there’s a spider on your back,” the cortex triggers the same response.

6:27 This is partly having to do with Steve Byrnes’ work. But on your podcast with Ilya, he said, “Look, how does the genome encode high-level desires or intentions?” These are exactly the questions about the loss functions. And it’s a really profound question.

7:04 Imagine saying the wrong thing on your podcast and Yann LeCun is listening and he says, “That’s not my theory.” That’s going to activate shame and embarrassment and you’re going to want to go hide. That’s important because you might otherwise get ostracized. So it’s important that I have this shame response. But of course, evolution has never seen Yann LeCun. It’s never known what an important scientist or a podcast is. But it knows you should not piss off really important people in the tribe — it does this without knowing in advance all the things you might learn.

7:59 The part that is learning — the cortex and other parts — it’s going to include things like concepts of important people. And evolution has to make sure that those neurons get properly wired up to the shame circuits, the reward function circuits. Because you need to learn from knowledgeable members of the tribe, exchange knowledge and skills with friends, but not get ostracized. It has to be able to robustly wire these learned representations from the cortex’s world model up to these innate reward functions. Because next time you’re not going to try to piss off the important person. We’re going to do further learning based on that reward signal.

8:59 How does a genome, which didn’t know about Yann LeCun, do that? The answer is that part of the cortex, or other areas like the amygdala, are modeling the Steering Subsystem. These more innately programmed responses — the system of reward functions, cost functions — there are parts of the amygdala that are able to monitor what the cortex is learning. How do you find the neurons that matter? You have some innate heuristics of social importance, heuristics of friendliness. And the Steering Subsystem actually has its own primitive sensory systems — the superior colliculus with innate ability to detect certain visual patterns.

10:14 So there’s a visual system that the Steering Subsystem has, and it has its own responses. The interesting neurons in the cortex — the ones that matter for the reward function — they’re the ones that predict those Steering Subsystem responses. You train a predictor in the cortex, and those predictors are what we call “Thought Assessors.” Those are the ones that can now generalize.

10:42 This is fascinating. I feel like I still don’t fully understand how this primitive part of the brain would generalize from “here’s literally a picture of a spider” to abstract spider concepts. The cortex learns that this is bad. But then it has to generalize to somebody just telling you the spider’s there. How does it do that without direct supervision?

11:15 The cortex is a powerful learning algorithm that does have this ability. The Steering Subsystem has its own built-in sensory systems — the brainstem, the superior colliculus. If I see something that’s moving fast toward my face that is small and dark and high contrast, that might trigger a flinch. There are these innate responses. There’s a neuron in the hypothalamus that is the “I-am-flinching” neuron. When you flinch, that’s a reward function that teaches you — I’m going to avoid that exact situation, maybe avoid some actions that led to it.

12:37 But you’re also going to do something else. The cortex is saying, “Okay, a few milliseconds before the flinch, could I have predicted that flinching response?” It’s essentially training a classifier. And I’m going to have classifiers for every biologically important variable that evolution needs to take care of: Am I near a friend? Should I laugh now? Is the friend high status? Am I about to taste salt? For each one, the cortex builds a predictor.

13:17 The predictor can generalize because its input data might be things like the word “spider” or the concept of spiders appearing in all sorts of situations. If you have a complex world model, it inherently gives you some generalization. This predictor can learn that whatever spider-related concept arises — be a book about spiders or a room with spiders — the heebie-jeebies response is appropriate.

14:18 The cortex inherently has the ability to make these predictions based on very abstract variables and concepts. Whereas the Steering Subsystem can only work with what its own primitive sensors — the superior colliculus and a few others — can spit out. Steve Byrnes has made this connection between different pieces of the puzzle. He’s an AI safety researcher. What I think he has is an answer to Ilya’s question about how the genome ultimately codes for these higher-level desires.

15:05 Very naive question, but why can’t we achieve this in AI by having the model not just map from a token to next token, but map every token to every token — omnidirectional prediction? Some people think that there’s a different learning algorithm that isn’t backprop. Energy-based models or other things that would be able to do this. Some think what the brain does is crappy versions of backprop. It’s kind of like a multimodal foundation model. But vision models maybe are trained in a different way, learning pieces or combinations in an extremely flexible way.

16:33 With a really powerful inference engine, you could ask what is the subset of variables it needs to consider. Two sub-questions arise. One, it makes you wonder if the key difference between current neural networks and the brain is less about the reward function and more about not representing data in a way where different modalities and concepts could intermingle. The brain seems really good at drawing connections between different ideas because all the representations are commingled.

17:28 We’re doing backprop-like learning and we don’t know how these areas are even connected well enough to get to the ground truth of this. My friend Joel Dapello actually did something interesting — he created a model of V1, specifically how the retina is doing motion detection and other preprocessing before feeding into a convnet, and that improves brain-like representations.

18:30 Astera, which is the same organization that funded our work, is funding a neuroscience project based on Doris Tsao’s work on visual systems that basically require less training by building more of the architectural prior — things like objects are represented in terms of certain types of shapes and relationships. Evolution may have also put architectural priors in, not just cost functions.

22:20 I want to talk about this idea of amortized inference. Right now, the way models work is that neural networks are amortizing a process. The real process of perceiving the world is Bayesian inference: you have some prior over how the world works, and you should be like, “Okay, here’s the best cause that explains what’s happening.” But exact Bayesian inference is computationally intractable. You have to do Monte Carlo sampling, taking lots of samples. And taking samples takes time.

23:53 Neural networks amortize this process. Instead of running inference from scratch every time, the neural network just goes observation to best cause directly. This raises the question of whether test-time compute — inference-time reasoning — is like doing more of those Monte Carlo rollouts. You literally read its chain of thought where it’s like, “Oh, let me think about this differently. Nah, I need a different approach.”

24:40 Digital minds which can be copied have advantages over biological minds which cannot. You can amortize more things because you can copy the things you’ve built in. In the future, as AI models become more intelligent and the way we train them becomes more sophisticated, there might be more and more things we can amortize into these minds, which evolution did not have the luxury of doing because you have to retrain every time.

28:15 This framework helps explain this mystery: if you want a model to learn efficiently, how do you explain the fact that so little information is in the genome? 3 gigabytes is the size of the genome. Obviously a small fraction of that is neural stuff. We have a lot in common with yeast. So much of the genome is just going towards basic cellular machinery. But if the key thing is the loss function — and if evolution found compact specifications for reward functions — it actually makes sense how you can build a human brain. Because the reward function, in Python pseudocode, is maybe a thousand lines.

29:30 It also gets to do this generalization trick we were talking about with the spider, where the cortex builds up complex predictors from these innate triggers. It just has to anticipate what variables are biologically important and specify how to wire up detectors for them. Fei Chen and Evan Macosko and others have been doing these single-cell atlases. The Steering Subsystem regions — the brainstem, hypothalamus — have far more diverse and bespoke cell types than the cortex. In the Learning Subsystem, there’s enough to build a learning algorithm. In the Steering Subsystem, there’s like hundreds of different specific cell types.

31:00 Why would each reward function need its own cell type? Well, in the Learning Subsystem you specify a learning algorithm — plasticity of synapses, changes in weights. It’s a relatively repeating pattern, just like how the Python code for a transformer isn’t that different from one layer to the next. Whereas all the Python code for “detect something that’s skittering and trigger the spider reflex” is bespoke, specialized wiring. The cortex doesn’t know about spiders innately.

32:30 So if the Steering Subsystem has its own visual system and own auditory system, there’s still a lot of complexity. Different features still need different specific wiring. This is the reason why a lot of the genomic real estate and cell types would go into wiring up the Steering Subsystem. Even the fly brain has innate orientation abilities and all these innate behaviors that I think go into the Steering Subsystem. There’s a lot of genomic real estate going into specifying what a fly has to be able to do.

35:00 How far back in evolutionary history do you go to find the cortex-like learning substrate? I think a mouse has a lot of similarity. Although there’s Suzana Herculano-Houzel’s research showing neuron count scales better with weight in primate brains than non-primates. Even birds may have reinvented something cortex-like. But even a fly brain has associative learning and something a little bit like this Thought Assessor concept.

42:42 On RL in the brain: there are parts of the brain that are doing something very much like model-free RL, the temporal difference learning stuff in the basal ganglia. It is thought that the basal ganglia has a certain repertoire of actions it can take — motor actions like “tell the spinal cord to do this motor action” and cognitive actions like “tell the thalamus to route information.” There’s some finite set of actions and it’s doing a very simple RL. But layered on top of that, the cortex is making a world model that can contain models of what types of actions in what types of circumstances lead to reward.

46:30 Joe Henrich and others have written about cumulative cultural evolution — how do you figure out that this bean that poisons you is edible if you do this ten-step cooking process? There’s no way to figure this out through individual reasoning. This is actually amortized optimization at a civilizational level. Evolution is model-free, basal ganglia is model-free potentially, but culture may be doing something more like model-based optimization.

50:31 Stepping back, is it a disadvantage or an advantage for biological hardware in comparison to digital? The brain has to run on 20 watts. It’s had to make a bunch of tradeoffs. But we can co-locate the memory and the compute. An obvious downside is you don’t have external read-write access — I can’t just edit something in the weight matrix. But otherwise maybe the brain has a lot of advantages. The co-design of the algorithm with the substrate is powerful. Hardware companies will probably try to use lower voltages, more analog computation, like the brain does.

53:00 I think the message I’m taking from this interview is that the people who get made fun of on Twitter — Yann LeCun and Beff Jezos — they’re kind of onto something about the limitations of current approaches. In fact, that’s what all the neuroscientists are saying. There’s a bunch of cellular stuff going on that may mean the cells are doing more work than the synaptic connections alone.

56:00 György Buzsáki has a book called “The Brain from the Inside Out” where he argues against applying AI concepts to the brain — that all this backprop stuff is just made-up vocabulary. He says we have to start with the brain and understand what it’s actually doing, rather than taking backprop and trying to apply it. He studies oscillations and timing. I think there should be both approaches — a totally bottom-up approach and one that tries to see if AI vocabulary maps onto what the brain does.

59:00 If we had a perfect representation of the brain — say we mapped the whole mouse brain — why think it would actually let us improve AI? We have neural networks which are way simpler and we can’t fully understand what’s in the weight matrices. Konrad Kording and Tim Lillicrap had a paper asking “What does it mean to understand a neural network?” They basically argue you could train a neural network to predict very complex systems, and we’re never going to fully understand every parameter. But we can still say the way it got that way was through this training data and this loss function.

1:03:00 On the practical side: E11 Bio is our main thing on connectomics. The Wellcome Trust put out a report estimating the first mouse brain connectome would cost billions. But E11 technology could get the mouse connectome down to low tens of millions of dollars. A human brain is about 1,000 times bigger. With technology development, could we do a human brain for less than a billion? That’s possible.

1:07:00 E11 has switched to an optical microscope paradigm. You can look at different molecular combinations and properties — not just who is connected to who by synapses, but which molecular types are present at the synapse. A molecularly annotated connectome. That’s a huge step beyond just structural connectivity. It’s like the difference between the Human Genome Project’s $3 billion initial cost and then the National Human Genome Research Institute getting companies to parallelize the process, dropping costs a million-fold.

1:10:00 The National Science Foundation just announced a BRAIN initiative grant which is somewhat FRO-inspired. They’ll be mapping the mouse brain with us in an open-source framework. But can companies also find value? If you could tell some story of not only mapping the brain but also improving AI with the results, you could go to AI labs and say, “Give me one training run’s worth of funding.” I sort of tried this seven or eight years ago. Maybe now there would be more appetite.

1:14:00 The idea of the Gap Map: in incubating FROs and moonshots, some scientists would say “Here’s what I find interesting, exploring these questions.” But the most compelling proposals looked like: “I need this piece of infrastructure. I need to have an organized engineering effort — the equivalent of the Hubble Space Telescope. Then I will unblock all the other researchers in my field.” These are mini Hubble space telescopes for different domains of science.

1:23:28 On Lean and formal verification: Lean is this programming language where instead of writing code, you express mathematical proofs. It’s a verifiable language so you can check whether the proof is correct. By itself this is useful for mathematicians — if I’m some amateur mathematician, Terry Tao is not going to just believe my result, but a verified proof changes that. It also enables mathematical proofs to be an RL signal — RL from verifiable reward. That becomes a perfect RLVR task.

1:26:00 I think RLVRing the crap out of mathematical proof is going to give us a really useful tool for mathematicians. But will it lead to new insights? That’s a harder question. A beautiful math proof or statement might be one that has implications for lots of other theorems — compact things that explain the rest. You could potentially measure the compression: what is the mutual information of this statement with the rest of the network of proofs?

1:30:00 There are really cool applied applications of formal verification. You could have provably stable, secure, unhackable software. “This code, not only does it pass these unit tests, but there is provably no way to hack it in these ways.” That’s a powerful piece of cybersecurity. And if you can prove the Riemann hypothesis, that gives you the ability to prove really complex things about very complex software.

1:33:00 Why hasn’t provable programming taken off more? One challenge is the specification problem. Engineers have things they want to formalize — code running the power grid or whatever — but they don’t necessarily know how to write the formal spec. People aren’t used to coming up with formal specifications. But if you project the current trend where LLMs are starting to be able to generate verifiable proofs, it’s possible that it just flips the tide toward formal methods.

1:35:00 There’s also Gwern’s proposal about brain-data-augmented training. Normally you train a neural network to classify cats and dogs with labels. But what if you also had “predict what my brain activity looks like when I see a cat versus a dog?” An auxiliary prediction task. Does that sculpt the representations in a way that’s more brain-like? It’s very easy to generate lots of labels, but brain activity patterns might provide richer labeling.

1:38:18 On the architecture of the brain: we still don’t know basic things. How many different types of dopamine signals are there? How many different types of Thought Assessors? Is the wiring between prefrontal and auditory cortex the same as between prefrontal and visual? The most basic things, we don’t know. And learning them through bespoke experiments is painfully slow, whereas mapping the whole thing at once through connectomics could reveal everything simultaneously.

1:42:00 On continual learning: is it just a matter of architecture — like the hippocampus consolidating to cortex — or does it require changes to the learning algorithm itself? At the architectural level, there’s probably something the hippocampus is doing that we don’t fully understand. How is it organizing and representing information? Is it training the cortex using replayed memories? There might be multiple timescales of learning operating simultaneously. From a neuroscience perspective, it’s not clear what causes continual learning except the general idea of hippocampus consolidating to cortex.

1:45:00 What is the timeline for neuroscience to meaningfully inform AI? If you’re in an “AI 2027” scenario where AGI arrives very soon, neuroscience may not affect what happens next. But if there’s more time — 5 to 10 years — neuroscience could influence the science of the future. My hunch is that there’s more like a 10-year-ish range, with probably a difference between a world where we have good understanding of the Steering Subsystem architecture and one where we don’t.

1:47:00 It has to be a really big push. If you could do the whole thing for low billions of dollars, and solving the sample efficiency problem of AI could save trillions, it seems like in the grand scheme of things it actually makes sense to do that investment. Even a few billion-dollar startups have launched recently in this space.

1:49:00 The thing I find interesting from the Gap Map process is that we started from neuroscience and biology and it branched out. We need genomics, but we also need connectomics. We need tools to engineer cells. Parts of biological infrastructure that wasn’t being built because no one organization was incentivized to do it. Fields that have had lots of funding may still have gaps where a critical mass-size project is needed. This is true in essentially every area of science — including mathematics, where researchers actually need verifiable programming languages and infrastructure, not just pencil and paper.

1:49:40 Cool. Adam, this is super interesting. Thank you so much. My pleasure. The easiest way to follow my work is through my blog, Longitudinal Science.