When people say "ChatGPT said" or "Claude wrote", they are naming a product. Underneath the product is a language model, and around it sits a good deal of ordinary software. Treating them as one thing causes real confusion: why did it know my document but not today's news? Why did the same question work yesterday and not today?
This article separates the layers.
What a language model does#
A language model estimates how likely a piece of text is to come next. Google's machine-learning course describes it as estimating the probability of a token, or a sequence of tokens, occurring within a longer sequence.
Take a sentence: "When I hear rain on my roof, I ___ in my kitchen." A language model gives each candidate continuation a probability. "Cook soup" might score higher than "warm up a kettle", and many others score low. Generating text is this same step repeated: pick a likely next piece, add it, then predict the next one.
A few terms are worth knowing:
- Token. The unit the model works with. It can be a word, part of a word or a single character. For English, a token is roughly three-quarters of a word, though this varies by language.
- Context. The text the model can take into account when it predicts. Large language models can consider a long stretch of context at once, which is why they handle a whole paragraph, not just the previous word.
- Parameters. The numbers inside the model that were adjusted during training. You never see them, but they hold what the model learned.
None of this means the model "understands" like a person, and none of it means it does not. What matters for use is simpler: it is very good at producing plausible text from context, and plausible is not the same as true.
Training and inference are different activities#
Training is how the model is made. It is shown enormous amounts of text and its parameters are adjusted until it predicts well. This happens before you ever use it, takes large amounts of computing and is done by the provider.
Inference is what happens when you send a request: the finished model reads your text and produces a response. Inference does not retrain the model.
This explains a common surprise. When you paste a document into a chat, the model can use it for that conversation because it is part of the context. It has not permanently learned your document, and a different conversation will not remember it unless the application stores and supplies it again.
The application is a separate layer#
A chat product wraps a model with other software. Think of the layers like this:
Step 1: App · Overview
The interface and account features.
Read all steps as a list
- App. The interface and account features. The chat window, history, file upload, settings and sign-in. This is the product you choose and pay for.
- Prompt and context. What is sent along with your question. Your message, earlier messages in the conversation, any uploaded files and instructions the app adds. Only what is included can be used.
- Model. The language model itself. Reads that context and generates the next tokens, one after another, to form a reply.
- Optional tools. Capabilities the model itself lacks. Web search, a calculator, a code runner, a lookup in your company's database. The app can let the model request these and use the results.
- Response. What you read. The generated text, possibly shaped by the app before it is displayed.
This is a simplified conceptual picture. It is not the internal design of any named provider, and real products differ.
Products such as ChatGPT, Claude and Kimi are examples of this kind of application. Each is built on one or more models and adds its own features and tools. Which models sit behind them, and what they can do, changes often, so check the provider's current documentation instead of relying on a list written here.
Tools give a model abilities it does not have#
A language model alone only produces text from its context. It cannot, by itself, open a web page, check today's exchange rate or open your file. Those abilities come from tools the application provides.
Compare two requests:
- "Explain the paragraph I pasted above." Everything needed is in the context. The model is working from provided text, which suits it well.
- "What is the current price of this product?" The answer lives outside the context. A model with no search tool can only offer a plausible guess from training. With a search tool it can fetch a page, but you still need to check the page it used.
The second request needs a suitable source or tool, and checking, not confidence in fluent wording.
Business integrations are a further layer#
When a company builds an AI feature into its own software, it adds more around the model: which documents the model may see, who may ask, how answers are logged, which actions need approval. This is where most of the work in a business system sits.
A useful consequence: the quality of a result depends on the whole chain. The task, the information supplied, the model's settings, any tools and the human check all matter. Blaming or crediting "the AI" alone misses most of the picture.
Why the same prompt can behave differently#
Three ordinary reasons:
- Models generate with some variation, so the wording of a reply can change between runs.
- The application may use a different model, tools or hidden instructions after an update.
- The context was not the same: a different earlier message, a file that was not attached.
A short checklist#
- Is the information I need in the context I gave, or does it need a search or lookup tool?
- Am I treating a fluent answer as a checked one?
- Do I know which application and tools I am using, and what they can access?
- Am I allowed to share this information with this application?
The next article shows how to write the request itself so the model has what it needs: How to give AI a useful brief.