At the end of my essay on the LLM’s place in a larger system, I mentioned an internal comparison at SkillSaige: a roughly fifteen-minute training scene ran with as much as a 97% lower token cost than a version that had an LLM-centered chatbot to run the scene end to end.

That’s a big number, I realize. We had to recheck it a lot. It deserves a little context. It’s a token-cost result from a host of internal bench tests. But even in our real-world testing, the reduction usually lands around 93-95%, give or take a few points.

So what changed? A chatbot that’s running the whole scene needs enough context to keep reconstructing the scene’s objective, history, rules, character behavior, and next move. And yeah, cached prompts can help… to an extent. But there’s an invisible cost with the rise of “thinking” models. More prompts means more reasoning tokens, means more processing power wasted discarding different results, rechecking output against instructions. Caching doesn’t help so much there. And caching is brittle.

But a system that owns those things can call a language processor for a much smaller operation, with the context relevant to that operation. The model doesn’t have to repeatedly “rediscover” what the application already knows.

The interesting part to me isn’t just that one bill gets smaller (although, honestly, that’s very nice and is why SkillSaige can be so affordable compared to its competition). It’s what the smaller bill reveals about the architecture. If most of the scene’s behavior lives in testable state and rules, we can change a rule without hoping the model notices a new paragraph in a prompt. We can examine what happened when a scene went wrong. And we can spend language processing where language is actually the problem.

I don’t want the 97% figure to stand in for a complete evaluation. Token cost tells us one important thing, not everything. Because the effects on the output, the experience, are FAR more dramatic than even those numbers can suggest. No hugboxing, no invention of objectives or exchanges, no hallucinated results. All the dynamics of language, but all the deterministic, tunable mechanics of a well-designed system.

Still, it’s a pretty striking sign that the model didn’t need to be the application to make the application work. And, perhaps unsurprisingly, when you do just that, you spend more time and energy letting humans design real, crafted systems for humans. And that, at the end of the day, is the whole point. The whole thing gets better when you let experts be experts.