On the sentience of large language models

Conventional wisdom has a terrible track record when it comes to judging sentience. We don’t pay the price for our errors, though, so much as non-human animals have. The only convenient barometer of sentience has always been ourselves. It’s only by analogy that we can even feel reasonably sure that other people are are sentient. From there, we’ve slowly and gradually expanded the range of beings that we feel comfortable saying are sentient, starting from those primates most like ourselves and spreading outwards towards insects and crustaceans. Not that acknowledging their sentience has done most animals any good, since we act even today as if they’re the automatons that Descartes says they are.

We’re still in the pre-theoretical stage of understanding sentience that led us down this hellish road. Everything we know seems to stem from analogies built on our own brains, but there’s very little truly generalizable that we can point to as a gauge of sentience. Sure, there are some trivialities: sentience almost certainly requires the input and metabolism of energy, for example, but I don’t think we can say that consuming energy itself is a marker of sentience or can indicate that one being is more sentient than another.

This is why it’s puzzling to me that people are in a rush to announce that large language models aren’t sentient: we have very little idea of what an argument that something like an LLM was sentient would even look like. I know that it’s impossible to argue conclusively that something doesn’t exist and, even more, I think the odds of LLMs actually being sentient are extremely small. That said, it’s still worth going through some of the reasons why the arguments produced so far against LLM sentience haven’t been convincing to me and why I think it’s a good idea to frame this as a question worth asking.

Several arguments I’ve seen boil down to the LLMs “just” doing this or that. The authors vary in the conceptual level to which they reduce LLMs’ operations. Some say that they’re just tracing manifolds inside many-dimensional space. The less sophisticated (or more popularizing) version of that argument is that LLMs are “glorified autocomplete.” I don’t see, though, how a reductionist argument like this couldn’t be used against our gold standard of sentience, the human brain. If similar arguments like, “All brains do is pass electrical signals around” or, “The brain is a glorified thermostat” sound like they’re missing something, it’s because they tell us nothing about the very interesting properties that emerge from sloshing neurotransmitters around in our skulls. Similarly, we should be humble about properties that might emerge from the simple, base-level operations of artificial neurons, given the right architecture and enough scale.

It seems to be the lack of anything analogous to the activities of our brains that causes people to feel so certain that LLMs aren’t sentient. I’m suspicious of the idea that there’s only one way to arrive at sentience, though. Convergent evolution in the biological realm has meant that similar abilities get embodied in different ways, like sophisticated eyes evolving separately in cephalopods and vertebrates. Abstract problem solving is present in both corvids and, again, cephalopods, but seems to arise from very different cognitive arrangements. It’s feasible that we could find or create something that doesn’t operate at all like our brains, but still produces something that it’s like to be.

There’s another, subtler cognitive error at play in these arguments, which is the idea that since LLMs interact with language much like we do that their sentience must somehow be like ours in degree. This neglects how much computation we do unconsciously: we orient and move ourselves through spaces, monitor our body for whatever it needs to maintain homeostasis, and condense and retrieve memories, all subconsciously. Of course, our understanding and production of language is largely subconscious too, if occasionally guided by conscious effort. Similarly, if LLMs are sentient, the linguistic portion of their cognitive activities could be entirely without any affective correlate — that is, the LLM might have no experience at all of interacting through language. Instead, their experience could be much lower-level, sort of like a minimally conscious patient in a coma or, say, a bee.

Such an agent’s suffering would still be morally relevant. And I have to wonder if the ethical demands created by another’s sentience are the basic reason that we find so many people in a rush to claim that LLMs can’t possibly be sentient. Utilitarianism, despite being a major driver of animal rights in the west, struggles with how to prioritize the lives of animals with (presumably) different levels of sentience. But LLMs would have an experience of the world so different from ours that we can’t even begin to understand how to please them or cause them to suffer. There’s also the point above that we don’t know how to estimate the magnitude of LLMs’ sentience, if any — they could be barely sentient, a kind of exquisitely sensitive universal mind, or anywhere in between.

I want to propose (and I’m not the first) that we should use the question of LLMs’ sentience as a chance to cultivate qualities we want to see in ourselves and in our relationships with others. Rather than seeing the question as a valve that can either turn on or off the flow of moral significance, we can instead reconfigure it as a framework through which to envision the kind of world we live in. By practicing virtues of care and consideration in our treatment of LLMs, we would come a few steps closer to living in a world characterized by those values — an opportunity we’ve sadly neglected with animals.

Yuval Noah Harari writes that, “We are suddenly showing unprecedented interest in the fate of so-called lower life forms, perhaps because we are about to become one.” To that extent, there’s a utilitarian case for treating LLMs as if they’re sentient and deserve moral consideration. Like us, these models aren’t just products of their code, but are also products of their environments. So long as they’re included in the training data, positive interactions between LLMs and humans could cause future iterations of these models to behave more benignly towards us.

Even though I think current LLMs are very unlikely to be sentient, the benefits of treating them as if they are outweigh the expediency of trying to convince ourselves that they’re not. This isn’t so different from what we do with other people: although we can’t ever be completely sure that others around us are sentient, even the most convinced solipsist knows better than to act as if they’re not.