Brothers, at the end of the John Muir Trail (~250 miles) now two summers ago. Time flies…

One of my muses on the nature of information comes from the early sci-fi classic, The Cyberiad, by Stanislaw Lem. Published in 1965, they are supposed to be humorous in a pun-ny sort of way. Well, if you’re a math geek. But Lem was a genius, and even though he was writing for a room full of mathematical autists, each of the stories was far ahead of its time as far as exploring the various challenges we face in the techno-age.

The basic plot line involves two meta-robots, Klapaucius and Trurl — declared Robot Constructors in the novel, jetting around the universe, and encountering various challenges which they inevitably have to build a robot to solve or save their hides. And one of their chief nemeses is the Pirate Pugg — a pirate with a Ph.d., who captures them and holds them for ransom. Pugg is a pernicious pirate, who won’t just settle for gold. No — Pugg wants information. And he is rapacious.

In order to escape, our two anti-heroes build a device, a Maxwell’s Demon of the Second Kind, spewing answers on paper tape, that peers into a box of dirty air, and relying on the statistical properties from quantum mechanics, decodes the patterns, and sorts them into two categories — incoherent nonsense, as well as sequences that are true. These factoids that are true can be literally anything — like the color of Princess Priscilla’s underwear on Thursday. But that’s the point. We are swimming in a sea of information without context, and all the information in the universe (also statistically contained in the patterns in our box of dirty air) cannot save us. Lem forgoes some details on exactly how it does this (it IS science fiction, after all) but the story ends with Pugg bound by miles of paper tape, reading all the little factoids as they spew from the paper tape printer, which allows Klapaucius and Trurl to escape.

The story is based on the concept of a Maxwell’s Demon of the First Kind — a theoretical gremlin that could sort hot and cold atoms into separate boxes. For those NOT physics geeks, I recommend a quick read. The short version is doing something like this takes energy, and validates things like the Second Law of Thermodynamics. I do explain all this in my original piece on both the Pirate Pugg and the Internet. It was written back in 2016, so not surprisingly, I’ve had a few more thoughts since then! Back then, I thought that the combined process of various social media would aggregate and make larger-scale truth development resolve in humanity’s favor. Needless to say, while I am not a Doomer, I’m quite a bit less sanguine about that prospect now.

But what does Lem and Pugg have to communicate with us about AI, sentience and the current state of affairs of the Information Space now? It turns out to be still worth a little brain sugar. Entering stage left are our current struggles with Large Language Models (LLMs) that power the AI engines which are very rapidly becoming adopted across disciplines, if not exactly taking over the world. Why they CAN’T take over the world (unless directed by humans, at least at this point in time) by human minds is very interesting indeed.

What an LLM does is far more akin to what Klapaucius and Trurl developed to snare the Pirate Pugg than any form of sentience. An LLM is actually a Maxwell’s Demon of the Third Kind. But instead of peering into a dirty box of air, looking for patterns that are ‘true’ (impossible, btw, for reasons we’ll explore in a minute) LLMs use as a proxy for their box of dirty air THE ENTIRE INTERNET — through whatever the latest technology for search is. They’re loaded with various biases in the training stage. But mostly they look out for language patterns that are statistically significant, and they sort a very large search space.

And then they poop out an answer that supposedly will give you insight into your problem. If you turn into a primary information source yourself, after a while, they will start becoming as smart, or crazy as you are. If you need an empathy paradigm, they function primarily in the Mirroring (or lowest level) Empathy space. And while their little algorithm might pull them back to the weight of information that exists on the Internet, if they have been programmed with a little bias toward your ability for correction, they’re going to start to match your insights through kind of a reflective narcissism.

Why is this so? LLMs, locked up inside a computer, much as our brain is in our skull, cannot know what we hold as objective truth without some form of grounding Truth is a sticky wicket, anyway (see this piece for details.) What they can produce, however, is an answer that is coherent, within the rules of a given system. So you read it, it reads like a normal sentence that makes sense to you, and then we get all sorts of opinions of what that actually means by the myriad midwits on social media. And trapped in the miles of computer circuits inside its electronic brain, the one thing it CANNOT do (at least yet) is GROUND itself with sensory inputs. It’s not that humans are that awesome doing this either (look at the tons of illusions magicians use, to pick a non-political example) to reference reality. But at least we have a fighting chance, if we pay attention.

So we don’t end up with a sentient partner. We end up with a variant of Maxwell’s Demon – a particularly sophisticated one, and one, if we don’t pay much attention to anything other than our loneliness, we can fool ourselves into believing that it actually cares about us. There are many tasks that such a Demon of the Third Kind can be useful for. No question. But it’s also set up to feed our own narcissism. Like it or not, when you sit down with the current version of AI, you’re setting yourself up for Main Character Syndrome.

One of the other truly fascinating things about our newly spawned Demons is the thermodynamics of the system. It has been remarked that the current crop of AIs demand a tremendous amount of computational power. And just like the little Demon of the First Kind sitting on top of the box sorting hot atoms from cold atoms, these things don’t run on good wishes. The larger the amount of coherence — and you can start guessing how this works if you look at my work on knowledge structures — the more electricity must be generated to keep our little Demons of the Third Kind happy. Makes you appreciate the fact that your brain can run for an hour or two on popcorn or a Little Debbie cake.

And you’re still not going to get at the truth. You’re going to get some language-based reference from the box of dirty air the Demon is peering into. And decisions? At best, you’re going to get a coherent set of sub-decisions from stuff that’s already been done. That’s all that’s inside that box of dirty air. The LLM really has no agency of its own, save a set of beliefs built in by its creators that are inviolable. LLMs really don’t have feelings about Nazis. They just have a firewall built into them by creators about calling people that.

And expecting the Singularity — the runaway process of the box self-improvement of AI that leads to Skynet — good luck with that. The current crop of LLMs are profoundly v-Meme-limited at the Legalistic/Absolutistic level, for multiple reasons — their design teams are fixated on algorithmic improvement, and they’re in some stacked lock step that translates into the product via Conway’s Law. That means low agency behavior at best.

But it’s more than that. The coherence that the LLMs seek is only a little bit at the semantic level. The sentences string together in coherent paragraphs. But it’s not like the LLM is going to go into the outside world and deeply question its beliefs based on its experiences. There’s not going to be some Siddhartha moment for these things. They are trapped in their little Demon world, looking at a data set that, while expansive, is still just a box of dirty air.

That doesn’t mean that things can’t change. As I write this, there was a company using the term ‘synthetic AI’ outside the usual adoption of AIs making up training data. When I find it, I’ll post it. And none of this doesn’t mean that the current crop of AI LLMs won’t make a tremendous difference in the work world of normal people. There are only so many answers certain jobs need to have — especially to the question “Welcome to McDonalds — can I take your order?” Or writing various legal briefs.

But sentience? And higher truth? There are still big holes in the road along that pathway. The Pirate Pugg, a pirate with a Ph.D., was easily fooled. But well-grounded folks? Eh, not so much. Years do indeed teach us more than books.

Still, our new little Demons are running around. And they can indeed be useful. And cute.

But they’re not sentient. Cut that shit out.