Essay on artificial intelligence and its (lack of) consciousness

Some people argue that it's pointless to make any claims about consciousness when we don't even understand what it is. There is a difference between recognizing the presence of consciousness and knowing how it can appear. We can be sure of our own consciousness.

Deciding if something is conscious or not can also matter, for example if we hurt an animal. Using too weak a definition of consciousness that would make it too easily obtainable could be missing the point.

When discussing the topic, I think we have to clearly separate 3 things:

If we mix up these things, discussions get blurry and annoying in my opinion.

Intelligence is about capability, awareness is about knowledge/information and understanding, and consciousness is about subjective experience.

Intelligence

Intelligence is about the capability to perform some tasks.

We just need to make sure we define intelligence as capability, maybe restrained to some minimum level of complexity when we consider the way the tasks are performed. Because for example simply extracting values from a table might be used to perform a task, but it's not complex enough. Even if we called that intelligence it doesn't change anything about the arguments of this essay as long as intelligence is about capability. We have this freedom for a more precise definition, that doesn't change the discussion about consciousness. Talking only about tasks is arguably not ideal, as intelligence could include internal capabilities for analyzing things.

Awareness

Awareness is always about knowledge, information and understanding, using the definition that makes the most sense in my opinion. Basically, something that's aware knows and understands what it is, and can have information about its surroundings, sometimes using senses or sensors. Here understanding means having knowledge of how things relate to each other and being able to use that knowledge in different situations.

For example it can be reasonable to say that ChatGPT 3.5 was already self-aware. It knows what it is, and it may be pretty cool to see how well LLMs can have knowledge about everything they are. If you have a robot powered by ChatGPT, you can have it look into a mirror and it can recognize itself. This is about information and knowledge. Knowledge can be obtained using intelligence.

There is no threshold which can determine if something is self-aware or not because self-awareness is about different degrees of knowledge and understanding. We can see something as self-aware or not like we can pretend we know Uzbekistan or not. We might have never heard of Uzbekistan, or just know that it's a country, or have deep knowledge about it. It's difficult to find something that will really determine if someone is really aware or not of what Uzbekistan is. Ultimately it's the use of language in practice that will be the main force to decide.

Some might think that at this point self-awareness is reduced to something too simple and it shouldn't be. But we're not reducing the complexity of the self-awareness of LLMs here. The state of the knowledge and understanding in the model can be deep. The ability of a deep learning model to figure out if an image represents a dog or a cat can already be deeply complex to analyze. The understanding of a model about itself can be sufficient so that we feel like claiming the model has significant self-awareness.

This presentation of self-awareness could also seem unsatisfying to someone who wants to attribute, by definition, attributes of consciousness to self-awareness. But here we try to make a clear distinction between both.

Consciousness

Consciousness corresponds to what some people call subjective experience. I like to define it as the thing that enables a "being" to experience reality at some place in time and space in a universe. It's about having a "point of view" from somewhere in reality. This definition might look different from those that say consciousness is about the ability to truly feel things but in the end we're talking about the same thing. Unlike self-awareness, we can wonder if this experienced point of view exists at all for an object (the answer is binary). We have evidence of our own consciousness.

First there exists an experience in one structure (for us it's the brain) at some time, then this experience receives the perception data present in the brain so that there are some things to feel and perceive. From our own experience, we know the human brain is able to host consciousness (and probably some other animals' brains too). Once we understand it better I think we will know how to at least theoretically design an object other than the brain that can be conscious. Today there seems to be very little understanding about how this phenomenon can exist.

The word consciousness in our culture may have been quite contaminated to mean intelligence or awareness, but the expression "subjective experience" seems to keep referring to the really interesting/hard thing, which also has ethical implications ("does it really suffer?"). The word qualia can correspond too though I'm afraid its use has been more contaminated to mean awareness. For me the best definitions have to do with an experience that's able to receive data from something (for example a brain) somewhere in space-time. The view of consciousness we have here can also be identified as "phenomenal consciousness" in the literature.

Consciousness vs intelligence

I think consciousness and intelligence are very different things. You can be extremely stupid or very brilliant (your intelligence varies during your own life, it's not even necessary to compare people), perform different (or no) tasks, and it doesn't affect your consciousness. Consciousness seems to correspond to an ongoing stream of experience through time that doesn't have much to do with the things you're thinking about. With our definitions it's not about awareness, but about experiencing reality at a moment from a being. I'm not saying consciousness is equal at all times for everyone as for example it seems that drugs or brain problems can affect it a lot.

The LLM itself is not where consciousness would be

An LLM (Large Language Model) is a set of weights and an algorithm about how to use them. This is an abstract thing that exists nowhere in particular in the universe (and I'd say it simply exists in reality beyond our universe). In other words LLMs themselves have a platonic existence.

All LLMs exist in reality in the same way that the number 2 exists in reality. They can't possibly be conscious in the same way that the number 2 can't be conscious. The number 2 can't have a point of view/experience from somewhere in space-time. Anything lacking this property can't be conscious, from our definition of consciousness. It's obvious that LLMs themselves can't be conscious, except if you think the number 2 alone can be conscious.

If you think we can't know if the number 2 is conscious or not then you're probably not using a definition of consciousness that can provide interesting discussion.

The real interesting question

The question is: are the machines running the LLMs potentially conscious? The machine doing the computation is something happening in space-time, so it can look like we have a candidate for conscious experience.

Consciousness vs intelligence for LLMs

The only human-like thing we observe in an LLM's behaviour is the intelligence. And we know that we are conscious regardless of how intelligent our current thinking is.

Let's list some possible reasons that can make someone believe a machine running an LLM can be conscious:

Let's discuss calling the machine conscious because of its intelligence a bit more.

This would mean that the machine has no conscious experience except when you use it to do an intelligent task. This is very unlike our continuous stream of experience through time.

Now something about the physical process. If you run a model with random weights on a machine, it's clearly not intelligent at all. Because intelligence is about capability and it doesn't have the capability to perform any task. But physically with the silicon it's the same thing happening as running a model with incredible intelligence and awareness. Someone who thinks the silicon can become conscious must think that it becomes conscious because of the output of what it produced, and I find that to be not a good argument at all.

If the machine had neurons then it might be a better sign of possible consciousness, but in our construction of the machine running LLMs, we only target something that produces intelligence.

Another thing that I want to point out is that I guess that the speed and electricity use of AI can make it look impressive and more human and conscious-like. If you think that consciousness arises purely from intelligence then you should think that doing tasks 1,000,000 times slower, or using stones also produces consciousness. Because consciousness has to be an experience from somewhere in space-time then you must think that the stones are conscious. I'd say if it's somewhere, then it has to be in the stones. Consciousness happens at places in space (and time) by definition, and doing the computation with the stones surely won't make some glass of water on the opposite side of the earth conscious.

How real consciousness could exist

Consciousness may not be made of atoms or physical things we're aware of. But the claim is that it is made possible by some structure present in space-time that can be described by an omniscient observer with perfect knowledge of physics. Our view and definition of consciousness here could also be compatible with a relationship to some crazy multiverse structure, but it at least has a connection to an experience somewhere in space-time of a particular universe as we know it. Another thing that I want to point out is that simpler theories of consciousness should also explain why we experience a complex and consistent world like ours. If a random conscious instance wouldn't need such a universe around it, why should we expect to find ourselves experiencing this one?

I don't want to make this essay weaker with super speculative claims, but in my opinion the theory of Penrose with microtubules looks quite attractive. Because it's a continuous physical process happening, present in the brain, and distinct from intelligence. Consciousness being so far so magical and misunderstood, it would also make sense that we live in a universe with complex quantum mechanics (I don't feel good saying this because I know nothing about quantum mechanics). Just like we have to be on a planet at a distance from the sun such that there is liquid water to be alive and conscious, we should be in a universe with quantum mechanics to be conscious. This theory might be way too speculative and look like a weak argument to prove that machines don't have such a source of consciousness because of lack of brain-like structure. But anyway even if it's something physically entirely different than this Penrose theory, my point is to give an example of a theory that distinguishes a very complex misunderstood physical process from an intelligent algorithm performed by a machine.

TL;DR / tweet

LLMs are not conscious, in the same way that a number can't be conscious: they are abstract objects (weights, architecture...) that don’t exist at any particular time or location in the universe. (Is there really a definition of consciousness that contradicts that?)

The question is whether the chips running the LLMs are conscious. They are also not conscious, for example because consciousness is an ongoing experience distinct from intelligent thinking (for example, you're conscious even if you think about stupid things or nothing). (More debatable, but a very strong argument imo.)

However, LLMs are self-aware, at least under some definitions. The information and understanding about themselves are present in their weights (or context, etc.) and in how they operate.

About the definition of consciousness: we don't understand how consciousness works, but we can know it when we have it. It's what makes us experience reality from a particular perspective, at a particular time and place in the universe. Thinking about what is conscious can matter, for example, if we consider hurting an animal.