Generative AI Explained: What LLMs Actually Do

Picture someone you know — maybe a colleague, maybe yourself. They have a 40-page contract (or a lab report, a policy document, a textbook chapter) and a deadline. On a whim, they paste the whole thing into a chatbot and type: "Summarize this for me. What are the key obligations?" Seconds later, a fluent, well-structured summary appears. It reads like a colleague who skimmed the document for them. It even catches a clause they missed.

That moment — surprise that a program can do this at all, mixed with unease about whether to trust it — is the right starting point for understanding generative AI. Because the surprise comes from imagining the program did something it didn't. It didn't "read" the document the way you do, didn't understand obligations the way a lawyer does, and didn't look anything up. It did exactly one thing, extremely well, in a loop: predict what comes next.

This post explains what a large language model (LLM) actually does, in plain English and without any math. One running scenario will carry the whole explanation: a developer named Maya, working on a side project — a small app that helps her team triage customer feedback. She wants to know what these models are, what they're good for, and where they quietly fail. By the end, you'll have a working mental model of the entire pipeline: from your prompt, through tokens and sampling, to the text that comes out — and a clear sense of what the model cannot do.

What an LLM actually does: one loop, repeated

A large language model is a program trained on huge amounts of text to do exactly one thing: predict the next token, given the tokens that came before it. "Token" just means a chunk of text — roughly a word fragment, a short word, or a few characters. Feed the model the tokens for "The capital of France is", and the most likely next token is " Paris".

Generation is that single step, repeated. The model predicts a token, appends it to what it already has, predicts the next one, and keeps going until it decides to stop (or hits a length limit). There is no separate "understanding step," no lookup, no planning phase bolted on — every fluent paragraph you've ever seen from one of these models is the output of that loop running a few hundred or a few thousand times. The astonishing part isn't the loop. The astonishing part is that this loop, done well enough, produces text that reads like thinking.

Here's the honest caveat to that: "done well enough" hides an enormous amount of training. The model got good at next-token prediction by adjusting its internal numbers (its parameters) across vast quantities of text, gradually getting better at guessing the next chunk. That process — training — is slow and expensive, and once it's done, the model is frozen. What you use day to day is inference: running the finished model to generate text. Training builds the model; inference is the model doing its job. As a rule of thumb, training is where the cost goes in, and inference is where the cost comes out, per token generated.

Why does this matter to Maya? Because the moment you picture a loop that predicts plausible continuations, two confusions clear up at once. First, the model is not answering you from knowledge — it is producing the text that its training taught it to expect next. Second, anything the model "knows" came frozen from training. It cannot know what happened after its training data was gathered, and it cannot know your private data, unless someone puts that information into the prompt at inference time. Both facts have consequences we'll return to.

Tokens: the model's unit of text

Models don't see words the way you do; they see tokens. A rough rule of thumb is that a token is around three to four characters of English text — a short word like "the" might be one token, while a longer word might be split into two or three. Different models split text differently (their tokenizers differ), so any token-to-word conversion you see is approximate.

Three reasons tokens are worth caring about in practice:

• Cost is measured in tokens. Most providers charge per thousand tokens, for both the input you send and the text the model generates. So "how long is my prompt?" is really "how much does this call cost?"
• Limits are measured in tokens. The context window (next section) is a token budget. You can't reason about whether your document "fits" without thinking in tokens.
• Splitting is invisible but real. The model predicts tokens, not words. Rarely this matters — a model can stumble counting the letters in a word, for example, because it never "saw" the word as letters. When output looks oddly mechanical, tokenization is one of the usual suspects.

The context window: the model's working memory

The context window is how many tokens the model can consider in a single request — your input plus its own generated output combined. Everything the model "sees" lives inside this window: your instructions, any documents you paste, the conversation so far, and the text it's generated up to now. The moment Maya pasted that 40-page contract into the chat, the whole thing had to fit inside the window for the model to summarize all of it.

A few practical consequences follow from this one idea:

• What falls outside the window might as well not exist. In a long conversation, the oldest messages can slide out of scope. If the model suddenly seems to have forgotten something you said an hour ago, it probably has — that text is no longer in the window.
• Window sizes are a moving target. They've grown substantially over recent years, and they differ between models and offerings. Treat any specific number you read as a snapshot: check the provider's current documentation rather than memorizing one.
• Bigger windows cost more, not just in money. Long contexts are more expensive to process, and there is evidence from independent evaluations that models can be less reliable at using information buried in the middle of very long inputs. As a rule of thumb: give the model what it needs and not much more.

Also note the model's memory boundary: the context window is per request. The model itself is stateless — it remembers nothing between your calls. When a chat app shows you a conversation history, the app is resending all the earlier messages with every new request, quietly, on your behalf. Your tokens, your cost, every time.

The pipeline, drawn

Here's the whole thing as a picture:

Two boxes in that diagram deserve special attention, because they control what comes out: the sampling control and the window budget. Let's take each in turn.

Why the answer changes: sampling and temperature

Ask a chatbot the same question twice and you'll often get two differently worded answers. That isn't a bug; it's how generation works. At each step, the model doesn't produce a single "correct" next token — it assigns likelihoods across many candidate tokens and samples one. Think of it as rolling weighted dice where the good answers get more faces. Run the loop again and the dice land differently.

Temperature is the knob that controls how freely the dice roll. A lower temperature makes the sampling more conservative — the model favors the most likely tokens, so output becomes more repeatable (but not guaranteed-identical). A higher temperature lets less-likely tokens win more often, so output becomes more varied and creative-feeling (and more likely to wander off).

The important subtlety — and a rule of thumb worth memorizing — is that temperature is a sampling control, not a reliability dial. Turning it down makes wording more consistent; it does not make the content more true. A model can state a confident, identically-worded falsehood at low temperature just as easily as at high. If your use case needs the same answer every time (an exact figure, a yes/no), that exactness has to come from your own code: parse the output, check it against a schema, retry when it doesn't fit. Never trust sameness of wording as evidence of correctness.

Training vs inference: learning the world vs using the model

We've touched on this, but it's worth stating plainly because the confusion is so common. Training is the months-long process of adjusting the model's parameters across huge text collections until next-token prediction is good. Inference is running the finished, frozen model on your prompt to generate text — the part you actually use.

This split explains several behaviors that otherwise feel mysterious:

• The model's "knowledge" has a cutoff. It learned from text gathered up to some point in the past. Anything that happened after that isn't in the parameters. If you need current information — today's prices, this quarter's policy — someone must supply it in the prompt (or the model must be given a tool that fetches it, which we'll cover in a later post on agents and tools).
• The model can't learn from your conversation. Tell it a new fact mid-chat and it will happily use that fact — but only inside the current conversation, where the fact sits in the context window. Start a new chat and it's gone. Nothing you say changes the model's parameters.
• "Fine-tuning" is just more training. When you hear that a model was fine-tuned for a domain, it means someone ran additional training on specialized text. The result is still a frozen model; the distinction between training and inference stays exactly the same.

For Maya's feedback-triage app, this split is the whole design problem. The model knows language; it does not know her company's products or this week's customer complaints. Whatever it needs to know, she must put in the prompt herself — which is exactly what prompt design and retrieval techniques are for (later posts in this track).

Chat vs completions: messages in, text out

Under the hood there are two classic ways to talk to a model, and the chat shape is the one you'll meet everywhere. The idea is simple: you send a list of messages, each tagged with a role, and you get generated text back. The roles you'll see in documentation are roughly:

• System (or developer) message: the standing instructions — "You are a helpful assistant that summarizes customer feedback in plain language." Set once, it steers the whole conversation.
• User message: what the person is asking right now — this is Maya's paste, her question, her feedback ticket.
• Assistant message: the model's own previous replies, included in the history so the conversation has continuity (remember: the app resends them; the model isn't remembering).

That's the whole contract: messages in, text out. The older "completions" style — a single text prompt with no roles, just a continuation — still exists in some APIs, but for practical purposes everything modern is chat-shaped. When you see a request body with a messages array, you're looking at the prompt: everything the model sees, exactly as it sees it. Which means the quality of your messages is the quality of your prompt, and improving an AI feature almost always starts with reading that array.

Worked example: the summary that surprised us

Let's make the opening scenario concrete. Maya pastes a short customer-feedback thread into a chat app and types the following prompt:

You are an assistant that summarizes customer feedback for a product team.
Summarize the feedback below in three bullet points: what customers like,
what they complain about, and what they ask for. Keep each point to one sentence.

Feedback:
---
"Love the new dashboard colors, finally looks modern. But it takes forever
to load on my old laptop and I gave up waiting twice this week."
---
"The dark mode is great. Please add a way to export my reports to CSV —
I keep having to copy-paste into spreadsheets."
---
"Loading is painfully slow since the update. Colors are nice though."
---

The honest illustrative output (the kind of response a model would plausibly produce — labeled illustrative because outputs vary from run to run):

- What customers like: the modern dashboard colors and dark mode.
- What they complain about: the dashboard loads painfully slowly since the update.
- What they ask for: a CSV export for reports.

Now let's walk through what happened, box by box in the pipeline:

• Prompt → tokens. Maya's instructions and the three feedback quotes were converted into a few hundred tokens. Her explicit structure ("three bullet points", "one sentence each") is now part of the sequence the model will continue — and that matters enormously, as we'll see.
• Next-token loop. The model predicts one token at a time: after "What customers like:" the most likely continuation, given similar patterns seen in training, is something about the colors — because the feedback mentions colors twice and the instruction asks for "what customers like."
• Why this structure helps. Prompts that show the desired shape of the answer get that shape back. The model isn't following instructions like a rule engine; it's continuing a pattern. A prompt that says "three bullet points" makes bullet-point text the most likely continuation. This is the core craft of prompt writing, and the next post in this track is devoted to it.
• Where the danger hides. Notice the second complaint: "I gave up waiting twice this week." The summary dropped it. And notice what's not in the output: any citation of who said what, any confidence level. A busy PM reading that summary would never know a customer abandoned the product twice. The model optimized for a fluent three-bullet summary, not for fidelity to every detail. If the lost detail mattered — and for triage, it might — Maya's code would need to verify the summary against the source text, not trust it.

This is the pattern of every LLM success story and every LLM incident: the output looks finished, so humans treat it as finished. It is not. It is a plausible continuation that needs checking before anything depends on it.

What the model does NOT do

Four "nots" prevent the most expensive mistakes. Each comes with a decision rule — the sentence you'll repeat when a design review goes sideways.

1. It is not a database. A model doesn't look things up; it generates text that resembles answers. Ask it for today's exchange rate or the status of your support ticket and it will produce something that looks like an answer — plausibly worded, confidently formatted, and possibly invented. This is what people mean by "hallucination": the loop has no step that checks truth; it only predicts likely text. Decision rule: if a fact lives in your database, the model must never be its source of truth. Fetch the fact yourself and hand it to the model inside the prompt — "The ticket status is SHIPPED; draft a reply saying so" — and the model stops guessing.

2. It is not deterministic. Same prompt, different wording, different runs. Temperature makes output more or less consistent, but as the rule of thumb above says: sampling control, not reliability. Decision rule: if your feature needs a stable answer, build the stability in your code — parse the output, validate it against an expected shape, retry or fall back when it doesn't fit. The model gives you text; your code decides whether the text is acceptable.

3. It does not browse the web — unless given tools. On its own, a model produces text and stops. It doesn't open pages, run queries, or click buttons. When a chatbot cites a live source or looks something up, that's the surrounding application doing it — fetching pages, running searches, feeding results back into the prompt — not the model reaching out. The model never touches your systems or the internet directly; every action happens because code you wrote ran it. Decision rule: draw the trust boundary around the model's tools. Anything that reads private data or changes the world needs your code's permission, not just the model's suggestion.

4. It does not understand consequences. A model can draft a refund approval, a legal summary, or a medical explanation without any notion of what acting on that text costs. Fluency is not judgment. Decision rule: the higher the stakes of acting on the output, the stronger the verification before acting. Low stakes (a draft title) can go straight to a human glance; high stakes (anything sent, published, billed, or enforced) need checks, and usually a human in the loop.

There's a fifth "not" that's subtler and worth carrying: it does not know what it doesn't know. The loop always predicts something; it has no built-in way to stay silent when the evidence in the prompt is insufficient. This is why good prompts tell the model what to do when information is missing — "say you don't know" — and why your code should treat hedging as a signal to verify, not as a minor stylistic choice.

The mental model to carry

Distill the whole post into one picture and you get this: a brilliant, confident, frozen autocomplete with a short-term memory window. It predicts the next token, one at a time. It works within a token budget. Its wording varies because generation is sampling, not calculation. Everything it "knows" came from training and is already old. It looks nothing up, clicks nothing, and means nothing by its fluency — but handed clear instructions and the facts it needs, it can produce remarkably useful drafts.

Notice what that picture implies for everyone building with these models. The skill isn't in finding the magic prompt; it's in owning everything around the model: choosing what goes into the prompt, validating what comes out, deciding which facts the model is allowed to touch, and putting a human in the loop wherever the stakes demand it. The model is the fluent middle. Your judgment is the system.

For Maya, this changes the design of her feedback-triage app. She won't ask the model "what are customers complaining about?" and paste the answer into a dashboard. She'll fetch the week's feedback herself, put it in the prompt with explicit structure, ask for a draft summary, and show it to her team alongside the source quotes — with the numbers (counts, dates) computed by her code, never quoted from the model. The summary is a draft. The data is hers.

Field check before you move on: open any chat app and ask it the same factual-ish question twice — something like "Summarize the pros and cons of electric scooters in one paragraph." Compare the two answers: same structure or different? Any facts you can verify, and do they match? Then paste in one sentence of context it couldn't know ("My city banned scooters last month") and ask again — watch how the new fact only appears because you supplied it. If you can point at each behavior and name the box in the pipeline responsible, the mental model has done its job.

This post is the foundation of the General AI track on the AI hub page. The next posts build on it: writing prompts that hold up against real users, giving the model tools and data it can actually trust, and testing AI features like the software they are. For the full learning order, see the Java Learning Roadmap 2026.