“We’ve made a computerized screaming void of the collective human subconscious. Would you like to talk to it?”
That’s how I framed chatbots in a recent conversation with Anne Gillaspie, and I’ve found myself coming back to it ever since.
I spend a lot of time thinking about how to understand LLMs, obviously. It’s kind of my job description at SkillSaige.
But not just how they work technically. Obviously, that matters. But I find it to be a common pitfall in software engineering to not try to understand the kind of thing you’re working with, rather than just the technical nature of it.
Large language models represent a major advancement in the long, plodding history of computing and the study of Artificial Intelligence. That’s pretty hard to argue at this point: it’s, for the first time, possible to load a model on a computer made up of billions of tuned numbers, and have it produce a language-stable response in a ludicrous percentage of circumstances.
This suddenly makes an absurd number of previously difficult things possible with far fewer resources. We’re rapidly moving past the question of, “What can we do with this?” which always has a multitude of wrong answers (hello radium water coolers), and moving onto the much more interesting question, “What should we do with this?”
But answering that requires us to have some kind of mental model of the strange thing we’ve actually built. And ideally, that model is more accurate than not.
Unfortunately, I’m increasingly convinced that calling it “artificial intelligence” gets in the way of giving us that useful mental model. It’s the mechanical bird vs. airplane problem. Intelligence as a concrete “thing” is such a vague, abstract, loaded term that conjures all sorts of fictional and cultural ideas, it struggles to do useful work here.
So let’s talk about where the common perceptions are wrong, and why they are holding us back.
LLMs are not a mind (artificial or otherwise).
This is the big one. An LLM is static. It’s a giant, static math problem, incomprehensibly large in its number of variables. An LLM can’t “learn” in the sense that it doesn’t grow or change. It’s not even “on” or “off”. It isn’t waiting patiently for your next message, contemplating what you said last time and updating its worldview accordingly.
When it’s trained, the team making it created a vast mathematical structure encoding relationships between symbols (tokens) that were measured in tiny, fractional amounts from vast quantities of “real” human-generated material.
When you interact with the model, you provide input to that existing structure and receive an output that would match those measurements.
Yes, there are systems built around the model that can retain your conversation history for it. But those aren’t the model. They’re just databases.
We can give the model access to “tools” in the sense that we can let the words it generates invoke code, try and retry poking software, shove the results back through, try again.
But if the model is still wholly responsible for deciding what happens, when it happens, whether it worked at all, what happens next… well, we haven’t actually moved anything out of the “screaming void of probability” layer.
Gemini and ChatGPT don’t “remember” what you said three messages ago. Or even in that other chat thread last week.
A server holds all your chat logs and shoves them (or summaries of them) back through the equation again when you give it a new message, then tacks on the model’s output.
But the underlying language model never “learned” anything. It just gave a new output based on the provided input (even if that input is 160,000 words of talking back and forth, system prompts, what have you).
That distinction can feel pedantic until you start designing real systems around these things.
If you think there is a little emergent mind inside the machine, then you’re tempted to solve problems by giving it more information, more autonomy and more opportunities to “figure things out.”
If instead you think of it as a computational system that performs a particular kind of transformation of language? Well, suddenly you start designing very different software. That’s just another processor. We know how to do that.
And to be honest, the software usually gets better as soon as you do this.
Hallucinations are LLMs working correctly, not a bug that can be ironed out.
This is the big one. When you start looking at things this way, it also changes how you view hallucinations. I know it has for me.
We talk about an LLM “hallucinating” when it confidently tells us something that isn’t true.
This makes intuitive sense from the human side of the conversation, right? It works great as a label under “these are the beginning of mechanical minds”.
Imagine the scenario between you and I:
I ask you a factual question.
You give me a factual-sounding answer, and with a string of words that projects confidence, as a professional might.
But the answer is wrong. You even doubled down. Triple down. You’re SO sure.
I think you must be hallucinating. That makes sense, right?
But to an LLM, from the model’s perspective, nothing particularly unusual happened in a scenario like that.
Nothing broke.
The process that produced the wrong answer is fundamentally the same process that produced the right one, because the model does not store explicit records of textual facts.
Sure, factual associations can absolutely be encoded in its weights, and often (though not always) reconstructed from them. But that’s not the same as reliably storing the stable, bounded record of that fact. That’s what a database is for. Sensing a pattern yet?
The model is moving through its enormous learned structure of language relationships and producing what should plausibly come next given everything it has been provided.
Now, sometimes the output corresponds beautifully with reality. Language is pretty cool that way, we’ve gotten good over the eons at communicating ideas about things clearly without that literal thing being present.
Then, sometimes it responds with a long essay about how a 450 degree pan is going to melt the plastic mounts and oxidize a steel sink instantly on contact.
I had Googled temperature thresholds for my specific pan, and Gemini piped in helpfully despite the fact I was looking for a link to the manufacturer’s website. Not only did I not need that, it was wrong. Those kinds of effects would have required temperatures well beyond anything I was doing (baking bread, if you’re wondering).
We label one of those successful reasoning and the other hallucination because we care about the distinction.
But we only care about the distinction because we are assuming that these tools are meant (dangerous word, there) to produce facts. Knowledge. Expertise.
The model never changed modes though. It never made a “decision”. It made a linguistically complete, stable response to the input it was provided.
The very fact the answer looks plausible means it did its job right.
And now we come to the crux of it.
The reason I find the “computerized screaming void” metaphor weirdly useful.
We took an incomprehensible amount of human language, associations, and cultural detritus (internet arguments included). Then we extracted the relationships within the words, the symbols, into an enormous mathematical structure.
And the only thing we’ve done so far is add an interface where we can yell into it, and then something yells back.
Maybe we attach some laboriously prepared bonus yelling to every input (system prompts) or let it yell to itself for a while before making it summarize its yelling (which we do by shoving the whole yelling chain back through the math equation again). Sometimes the law of averages will give a “better” response, right? Let’s call it an “agent”!
And this is all genuinely incredible!
I mean that without irony. We have created a machine that can operate across the fuzzy relationships embedded in human language at a scale conventional software simply could not before now.
It’s a great start. Unlike traditional processors, which operate in machine code that’s kind incomprehensible to most people, LLMs operate in language. Regular language. That opens up a lot of possibilities.
But maybe we should stop being surprised that the screaming void occasionally says something nonsensical. Humans are pretty nonsensical, on the whole.
And maybe we should start wondering just how much we’re missing out on by doing things this way.
So what if, instead, we start making the LLM responsible for a lot less?
This is what I keep coming back to.
Because whenever I point out the limitations of LLMs, I worry that it sounds like I’m making an argument against them.
I’m really making almost the opposite argument. I think this technology is extraordinary.
What I increasingly question is our obsession with making AI-powered LLMs the end-all-be-all solution for every problem.
The model needs to know more? Give it a bigger context window.
It needs access to something? Train it on stable word structures to invoke other software blindly.
It needs to accomplish a sequence of actions? Make it an “agent” and let it ramble for hours, throwing out its ramblings every time they don’t fit back into the math equation anymore, hoping it gets where we want it to go eventually.
Every step in this direction tends to mean more tokens, more inference, and more opportunities for unpredictable behavior. More expense. More risk.
For a huge set of ordinary software problems we are so far past the point of diminishing returns.
So what if, instead, we made the LLM responsible for less?
This is something we’ve been exploring extensively at SkillSaige.
Instead of asking a language model to run the entire experience, we treat it as one specialized component inside a larger software architecture.
Conventional software handles the things conventional software is good at.
State can be explicit. Rules can be deterministic. Important facts can come from systems designed to actually store important facts (hey, databases!).
None of these things become obsolete because language models exist now.
The LLM gets brought in only where its strange ability to navigate meaning, ambiguity and language gives us something that would otherwise be extraordinarily difficult to build.
It’s never allowed to leave that box.
The funny thing is that this doesn’t make the LLM less important. In our experience, it makes the LLM considerably more valuable.
We stop asking it to badly approximate things ordinary software already does extremely well (at least, with a little bit of good old fashioned engineering up front) and start using it for the narrow category of problems where it can do something genuinely remarkable.
This architecture offers an alternative to the escalating race toward ever-larger models, contexts and autonomous systems.
And I think we are only scratching the surface of what that alternative looks like.
The weirdest thing about it? For the jobs they perform in a bounded system like this, the models themselves are incredibly interchangeable.
The models barely matter anymore. It’s like picking your preferred CPU vendor when building a PC. Your choice affects speed and cost, and maybe some advanced capabilities. But it becomes pretty easy to measure and pick the right one on results, not benchmarks or hope that the “mind” is smart enough.
So what’s next?
There is a lot more to say here.
Some of it came up in my conversation with Anne, but if you’ve made it this far, you’ve probably realized that we’ve only just scratched the surface.
If you want to hear the full conversation with Anne, you can watch or listen to my OnlyFounders episode. Give it a listen: I had a great time, and Anne is the kind of mind we need more of in this new age. Curiousity coupled with genuine empathy.
And if you want to see what “put the LLM in a smaller box” actually looks like in practice, I’ll be publishing a more applied breakdown on the SkillSaige blog later this week.
Because it turns out there are some surprisingly useful things you can build once you stop asking the screaming void to run everything (and stop acting like “portal to the screaming void” is a long-term consumer product, but that’s a different story for another time).