Life Tech Hack

Step 4: How Do AI Models Work? The 3 People Behind the Answers

Step 4: How Do AI Models Work? The 3 People Behind the Answers

Welcome back to the AI Essentials series! In our last lesson, we discovered that AI models such as ChatGPT and Claude generate text one token at a time; in simplified terms, they predict likely next tokens using learned patterns. — built on the Transformer architecture and trained at such a staggering scale that prediction started to look like creativity.

We also left two questions hanging — the ones that keep beginners up at night: if it’s all prediction, is the AI actually thinking, or just rolling very sophisticated dice? And why does it sometimes state complete nonsense with total confidence?

Today, we answer both. We are going to open the hood and watch an answer being built in slow motion, one token at a time.

By the end of this lesson, you will understand three things almost nobody using AI can actually explain: what a token is, how a probability engine chooses every single word, and the three-stage training pipeline that turned a text-completion savant into a helpful assistant.

Still no math degree required. Grab your safety goggles.

1. Meet the Token: AI Models Don’t Read Words — It Reads LEGO Bricks

Here is the first surprise: An LLM does not process text as whole words in the way people do. A tokenizer converts text into tokens, which may represent a whole word, part of a word, punctuation, or a short character sequence. It sees tokens — chunks of text that sit somewhere between a letter and a word.

💡 The LEGO Analogy: A LEGO set doesn’t contain a “castle piece” or a “spaceship piece.” It contains a finite set of standard bricks that can build anything. Tokens work the same way.

Token vocabularies vary by model. They contain reusable text fragments that represent common words, parts of rare words, punctuation, code, and unfamiliar text. A tokenizer does not need a separate stored entry for every possible word, and different models can split the same text differently.

In practice, common words like “the,” “cat,” and “house” are usually one token each, while rare or long words get snapped apart into several bricks (something like “electroencephalography” becomes a small pile of them). A useful rule of thumb: 1,000 tokens ≈ 750 English words.

Once you know about tokens, several everyday mysteries solve themselves:

  • Why AI Models’ pricing is “per token”: you are literally paying per LEGO brick.
  • What a “context window” is: the maximum number of bricks the model can hold in mind at once.
  • Why AI Models famously stumbled on “How many R’s are in strawberry?”: the model sees chunks, not individual letters — one big reason letter-counting is unexpectedly hard for a machine that writes essays.

2. The Probability Engine: Watching One Word Get Chosen in Slow Motion

Now for the main event. Type this prompt: “The capital of France is” — and here is what actually happens inside:

  1. Your text is converted into tokens.
  2. The neural network (the Transformer from Step 3) processes them and produces a score for every single token in its vocabulary.
  3. Those scores become a giant probability list: ” Paris” → 96.1%, ” the” → 1.2%, ” a” → 0.6%, ” located” → 0.5% … all the way down to ” banana” → 0.0000…%. Illustrative example: These percentages are simplified for explanation. Actual probabilities vary depending on the model, tokenizer, prompt, and decoding settings.
  4. The model picks one token, appends it to the text, and then runs the entire loop again for the next token.

A text response is produced token by token; the number of tokens is not the same as the number of words and varies by language and tokenizer. In a plain generation request without browsing or retrieval, the model is not querying a live factual database. Tool-enabled systems may search or retrieve sources, and generated text still requires verification.

This idea is much older than ChatGPT. In 1951, Claude Shannon — the father of information theory — ran a famous experiment where he had people guess the next letter of English text, one letter at a time, and measured how predictable language really is.

His conclusion: human language has deep statistical structure, and that structure can be exploited by machines.

📜 The Dice Roll: Temperature, and Why the Same Question Gets Different Answers

Selecting the highest-probability token at every step is called greedy decoding. It can be useful for some constrained tasks but may produce repetitive or low-diversity text in open-ended generation. Unrestricted random sampling can become incoherent. Modern systems use different decoding and reasoning methods, so no single setting is best for every task.

  • Temperature controls how adventurous the dice are. Low temperature reduces variation and can help produce more consistent formatting or extraction. It does not guarantee factual accuracy, so important claims still need verification. High temperature → underdog tokens get a real chance (great for brainstorming and stories).

  • Top-p (“nucleus”) sampling trims the table first: only tokens that together cover, say, the top 90% of probability stay in the game — Top-p sampling keeps a selected group of higher-probability tokens and excludes the less likely tail before sampling. The excluded tokens are not necessarily nonsense; they are simply less probable in that context.

  • Source: Holtzman, A., et al. (2020). The Curious Case of Neural Text Degeneration. International Conference on Learning Representations (ICLR).

  • Why it matters: This paper showed why controlled randomness produces the most human-like text — and it explains a daily experience millions of users misread as a glitch: ask the same question twice, get two different (both sensible) answers. Not a bug. A design choice.

3. From Autocomplete Savant to Assistant: The Three-Stage Training Pipeline

In Step 3, we promised to dissect how a raw prediction engine becomes a polite assistant. Here is the twist: a freshly trained ai model is not a chatbot. Ask a raw, pretrained model “How do I bake sourdough?” and it might reply “How do I keep my starter active? How long should I proof the dough?” — because on internet forums, questions are usually followed by more questions. It is completing text, not helping you.

A common training approach for turning a raw language model into an assistant uses a three-stage pipeline, although modern AI systems may use different methods. Think of training a chef:

  1. Pretraining (reading every cookbook on Earth): The model learns next-token prediction across a vast slice of the internet. Result: encyclopedic pattern knowledge — and zero table manners.
  2. Instruction tuning (culinary school): The model is shown many thousands of examples of instruction → ideal response, and learns the format of being helpful: answer the question, follow the requested style, stop when done.
  3. Human feedback training / RLHF (customer feedback cards): The ai model writes several candidate answers; human reviewers rank them; a “reward model” learns those preferences; the chatbot is then tuned to produce the kind of answers humans consistently rated higher.
  • Source: Ouyang, L., et al. (2022). Training Language Models to Follow Instructions with Human Feedback . Advances in Neural Information Processing Systems (NeurIPS).
  • Why it matters: This is the famous InstructGPT paper — and it delivered a stunning result: human judges preferred a 1.3-billion-parameter model trained with feedback over the raw 175-billion-parameter GPT-3. In other words, how you train can beat how big you build. Alignment with human preferences turned out to be its own superpower.

4. So… Does It Actually Think? (The Honest Answer)

Time to face the question in this lesson’s title. The honest answer has three parts.

Scope of this explanation: a plain language-model generation step predicts tokens rather than consulting a live answer database. A tool-enabled assistant can also search or retrieve information. Generated text can reproduce memorized material, so token-by-token generation does not guarantee originality.

What the model computes: a distribution over possible next tokens, conditioned on available context and learned parameters. Implementations can cache earlier computations. The selected continuation depends on the decoding method and is not necessarily the single most probable complete answer.

The model is optimised to generate plausible continuations. Fluent output is common but not guaranteed, and fluency is not evidence of truth. The gap between convincing language and verified facts is one reason Step 7 focuses on hallucinations and independent checking.

🙋 5. Reader Q&A: Three People, Three Very Different Questions

(Representative questions beginners commonly ask — answered without the jargon.)

Q1 — from a stay-at-home parent building a blogging side hustle: “I asked the exact same question twice and got two different answers. Is it broken? And which answer am I supposed to trust?”

Different answers can result from sampling, changed context, tool results, model updates or other implementation details. Two responses alone do not identify the cause. Compare factual claims with reliable sources instead of assuming variation is harmless.

For creative drafting, generating several alternatives can be useful: compare them, check originality and factual claims, and keep only material you are authorised to use.

For facts, treat two differing answers as a yellow warning light: ask the model to list its checkable claims, then verify names, numbers, and dates yourself before anything goes on your blog. The rule of thumb: variety in style is a feature; variety in facts is an alarm.

Q2 — from an AI-major university student: “In the API playground I keep seeing temperature and top_p. What do they actually do to the probability distribution?”

Answer: After the network produces its scores over the whole vocabulary, temperature rescales the distribution before the sample is drawn: low values sharpen it so nearly all probability piles onto the top tokens (approaching greedy decoding as it nears zero), while high values flatten it so tail tokens get real odds.

Top-p truncates instead of reshaping: sort tokens by probability, keep the smallest set whose cumulative probability reaches or exceeds p (say, 0.9), renormalize, and sample only within that nucleus. Holtzman et al. (2020) is the paper to read on why this beats both extremes — greedy/beam search collapses into repetition loops, naive sampling drifts into incoherence, and nucleus sampling lands closest to human-like text.

Check which sampling parameters the selected model supports before setting them. Where available, compare settings on representative tasks and record the results. Low temperature is not a factuality guarantee, and temperature zero does not guarantee identical outputs across models, servers or runs.

Q3 — from a solo startup founder who does everything alone: “I’m a one-person company. I don’t have time to fact-check every sentence. Where do I absolutely HAVE to verify?”

Answer: Verify wherever an error carries a price tag. Red zone (always check): anything with names, numbers, dates, prices, legal or medical claims, statistics, and citations — plus anything that leaves the building: published posts, client emails, contracts, investor decks.

Verification effort should follow the consequence of an error. Brainstormed ideas may need little checking, but summaries can omit, distort or invent source details. Compare important summary claims with the original; a model-generated checklist or two spot checks cannot certify an entire answer.

Treat the model like a brilliant unpaid intern: endless ideas and tireless drafts, but you sign everything that leaves the building.

📊 6. Myth vs. Reality

AI model token prediction, probability sampling, and training pipeline explained
AI-assisted educational illustration.

⚡ 7. Try It Today: Watch the Machine Think (Three 2-Minute Experiments)

Privacy first: Use sample text instead of your real name, private emails, passwords, client information, or confidential business content. Check the privacy policy of any online tokenizer or AI tool before entering text.

Theory becomes intuition the moment you see it move. Run these three quick experiments:

  1. See the bricks: Search for a free “LLM tokenizer” playground online, paste a non-sensitive sample phrase—not your real name or confidential text—and the longest word you know, and watch the text snap into colored token chunks. (Notice how rare words shatter into pieces.)
  2. Catch the dice: Open two fresh chats and ask the identical creative question in both — “Write a two-line poem about Mondays.” Compare. Then ask a factual question twice: the phrasing varies, but the core fact should hold still. You have observed output variation; isolating temperature requires controlling the model, context and other settings.
  3. Grade the intern: Ask AI Models about a niche topic you know deeply — your hometown, your hobby, your industry — and score the answer like a teacher. You will feel, not just read, the gap between fluent and accurate.

📝 8. Recap & What’s Next?

Today you looked inside the machine, and five ideas came out with you:

  1. Tokens are the LEGO bricks of language — ai models read chunks, not words or letters.
  2. The loop: score every token in the vocabulary → pick one → append → repeat, hundreds of times per answer.
  3. The dice: sampling settings can contribute to variation; context, model and system changes can contribute too.
  4. The pipeline: pretraining → instruction tuning → human feedback; a small, well-aligned model can beat a giant raw one.
  5. The mindset: treat the system as a powerful prediction and generation tool. Fluent language is not guaranteed and does not establish truth; consequential claims require independent evidence.

So now you know how the engine works. But that raises the practical question every reader eventually asks: which car(AI Models) should you actually drive? ChatGPT, Claude, and Gemini are built on the same science, yet they behave very differently at the wheel.

In the next lesson, we put the big three head-to-head — task by task, strength by strength — so you stop paying for the wrong subscription.

⏮️ Previous Lesson: [Step 3] Understanding Generative AI and Large Language Models (LLMs) Explained

⏭️ Next Lesson: [Step 5] ChatGPT vs. Claude vs. Gemini: Which AI Should You Use?