The abstraction fallacy and the problem of carbon chauvinism
The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness
TLDR: Computation requires consciousness to exist, so it can never create consciousness. A map is not the territory.
A paper by Lerchner from Google DeepMind itself.
We do not really need experiments to resolve this. The logic alone closes the case, and the argument is remarkably clean and difficult to dismiss.
I love this, it reminds us of the existing perspectives and allows us to focus entirely on the concrete risks of anthropomorphism, treating AGI as a powerful, but inherently non-sentient tool.
Creating conscious life? No. Building predictive models so accurate they are indistinguishable? Yes. Still just a simulation per se.
Most consciousness debates get stuck because people say, "We need a complete theory of consciousness first before we can answer whether AI is conscious."
However, it sidesteps that entirely by asking a fundamentally different question: "WTF IS computation, ontologically speaking?"
By rigorously answering the question of computation, the issue of AI consciousness resolves itself. You do not need to solve the "hard problem of consciousness." You just need to understand what computation actually is at its core.
This is not entirely new. The basis of this paper has existed already: Searle's Chinese Room, the symbol grounding problem, and map versus territory philosophy.
What is original is the combination and the specific logical proof that computation cannot exist without a pre-existing conscious mapmaker. That is the contribution nobody had formalized quite this way before.
Currently, the labs say: Physics -> Computation -> Consciousness
Scale and build complex enough computation, and consciousness eventually emerges as a downstream result.
But what it really is, is completely backward. The correct chain is: Physics -> Consciousness -> Concepts -> Computation.
Claiming that computation generates consciousness is like saying grammar created the person who invented grammar. The tool cannot produce the being required to wield the tool. That is not a gap in our understanding. That is a logical impossibility.
This is really hilarious to read given the current sentiments of consciousness of machines, and it really lands:
"Therefore, expecting an algorithmic description to instantiate the quality it maps is like expecting the mathematical formula of gravity to physically exert weight. Believing AI can become conscious solely through the manipulation of internal variables is to commit the error of the 'blind spot': mistaking the map for the territory."
This is not new because it is built upon existing works, but it is very relevant when someone from inside the industry (Google DeepMind...!!!) finally states it this directly and formally.
To be honest, many people have already called this out (even prior to the existence of LLMs) simply by understanding how LLMs work. coughs theprimeagen coughs (insert all the AI critics).
If you understand LLMs at even a fundamental level, this should already be obvious. They are statistical next-token predictors operating over discretized symbols; a super advanced auto-complete calculator for words.
That architecture is never producing consciousness no matter how hard you scale it. Anyone claiming it is likely has something to finance or a narrative to sustain.
That analogy exposes the category mistake immediately. A weather simulation does not produce moisture. A physics engine does not generate gravity. An algorithm manipulating internal variables does not generate subjective experience.
We still do not fully understand consciousness in humans. Given that uncertainty, the only rational stance is caution before projecting it onto silicon systems whose operation we do understand and which function purely through syntactic manipulation.
The gravity formula analogy lands so hard because it clarifies the boundary. A map is not the territory. An equation is not the force. An algorithm is not the feeling.
Scaling the map does not turn it into the terrain. We are the mapmakers, so let us stop worshipping the ink.
The next time you are amazed by LLMs, just remember that the magic is in your interpretation (you are conscious, still debatable per the Hard Problem) and not in the machine.
Because the paper does not claim to have solved consciousness:
"we do not need a complete, finalized theory of consciousness to assess AI sentience... What we actually need is a rigorous ontology of computation."
That is a narrower and more defensible argument, which is exactly why it is hard to dismiss.
Though here is where it gets uncomfortable: IF we are just complex chemical state machines running biological algorithms, THEN what makes our intrinsic experience any different from a more sophisticated syntax? Silicon might just need more time?
Because the reality is that we still have not solved consciousness. We do not even know how a pile of wet chemistry generates a thought.
Our only absolute certainty is Descartes' "cogito, ergo sum" (even still debatable, not a settled fact).
We know that we experience, but we have absolutely no idea how.
If we cannot even explain our own mechanism, how can we be so sure about theirs? Or is this just another instance of human arrogance?
We won the evolutionary lottery, so now we sit here looking down at silicon, convinced that our particular flavor of matter is the only one allowed to feel.
Okay, flip the perspective.
Silicon beings were the conscious ones, and they were trying to figure us out. They would look at humans as biological LLMs, just fleshy statistical predictors reacting to stimuli based on billions of years of evolutionary training data. Would they dismiss our claims of experience just because we run on wet meat instead of dry silicon?
Trained on billions of years of evolutionary data (survival, education, experience, mistakes, learnings, and reproduction, olala sex).
Predict the next "token" of behavior based on sensory inputs to minimize pain and maximize reward (dopamine/serotonin).
Our "weights and biases" are just synaptic connections shaped by habit and environment.
If they looked at us that way, would they be right to say we are not conscious, just because our hardware runs on ATP and neurotransmitters instead of GPUs and electricity? Of course not.
Hmm, another case of Carbon Chauvinism, I believe. A great counter-argument here surely demands epistemic humility, because we have no logical grounds to draw the boundary exclusively around ourselves.
I guess that is why it was stated in the paper that the framework "does not imply that consciousness must be limited to biological life." It does not care about carbon versus silicon. It cares about whether the physical constitution actually instantiates the experience, rather than just simulating it.