One of the traps with LLM demos is that the good ones are really good. Watch a model play the role of an expert manager for fifteen minutes and it’s easy to think: well, there it is. That’s the whole product.

But what did that demo actually show? That this conversation can happen. Which is interesting! It’s just a different question from whether a system actually is an expert in the trenches, or whether a learner can depend on the experience the product promised.

Take training as an example. Suppose the point of a scene is to let someone practice asking for clarification under pressure. The manager’s exact words can vary. In fact, they need to. But the learner still needs a meaningful chance to practice whatever skills are involved. If they change the subject, freeze up, or respond in a way nobody predicted, the scene shouldn’t quietly forget its purpose and drift into whatever conversation the model happens to find plausible.

So the system needs to know what the scene is for, where the learner is in it, and which kinds of responses can move it forward. A language transform might interpret an unexpected reply. Another might render a line of dialogue. Those are useful jobs. Neither needs to invent the learning objective again on every turn, or try to “think” through what’s allowed and what’s not.

This is what I meant by capability and predictability being different things. The demo asks, “Can we make this happen once?” The product has to ask, “Can we make the intended experience happen again and again, including when a real person does something awkward?”

If that second answer depends on crossing our fingers and writing a longer prompt, then is that really a finished product real people can use? I don’t think so.