How does a chatbot decide what to say next?

Answer first

A chatbot decides what to say one word at a time. It looks at all the words so far and scores each possible next word, using patterns learned from a huge amount of text. Then it picks one and repeats. Its words are predictions, not checked facts, so it can sound right and still be wrong.

In one line: A program built on a language model that writes a reply by predicting one likely next word after another.

Last reviewed: 25 October 2026 3 min read

What decision does it make?

A chatbot writes its reply one small piece at a time. Before each piece, it makes one choice: which word should come next? It adds that word, then makes the same choice again, until the reply is finished.

Decision	Possible answers
Which word comes next?	Any word it knows. It usually picks one of the words with the highest scores, adds it, then decides again.

What does it look at?

Words so far. It looks at your message and everything it has written so far, not only the last word.
Patterns from text. It learned from a huge amount of writing, so it has scores for which words tend to follow which.
Your prompt. The way you ask changes the words so far, so it changes the scores too.
A little chance. It does not always take the top word. Sometimes it picks at random, with likely words more likely to be chosen, so the same question can get different answers.

How does it learn the rule?

Collect lots of text. The program is given a huge amount of writing.
Find the patterns. It looks for which words tend to come after which other words in that writing.
Turn patterns into scores. From those patterns it learns a score for each possible next word, given the words before it.
Write one word at a time. To reply, it scores every possible next word, picks one, adds it and scores again.

Where can it go wrong?

It sounds sure and is wrong. It picks words that fit well together, not facts it has checked. A smooth sentence can still be false.
It is not thinking the way you do. It is scoring which word fits next. Looking clever and being correct are not the same thing.
Common can beat correct. Care only about which words often come next and an odd word can win. Try it below.

Try it

A toy next-word machine has read "The cat sat on the". Move the sliders to change how much it cares about each clue, and watch which word it picks next.

A toy example: five possible next words after "The cat sat on the", each scored from 0 to 5 on two clues. The numbers are made up for practice.

Item	Values	Score at starting weights
mat	Fits the words before (0 to 5): 5; Often comes after "the" (0 to 5): 3	18
floor	Fits the words before (0 to 5): 4; Often comes after "the" (0 to 5): 4	16
chair	Fits the words before (0 to 5): 4; Often comes after "the" (0 to 5): 3	15
moon	Fits the words before (0 to 5): 1; Often comes after "the" (0 to 5): 5	8
banana	Fits the words before (0 to 5): 0; Often comes after "the" (0 to 5): 1	1

Check it

How does a chatbot write a reply?

Answer: It picks one likely next word at a time. It scores possible next words, picks one, adds it and repeats.

Why can a chatbot say something false?

Answer: It picks words that fit, not facts it has checked. A sentence can fit together well and still be wrong, so check important facts somewhere else.

You ask the same question twice and get different answers. Why?

Answer: It sometimes picks at random, with likely words more likely. It does not always take the top word, so the reply can change.

For older readers

The engine inside is a large language model. It works on tokens, not whole words: a token can be a word, part of a word or a single character. At each step it estimates a probability for every possible next token, based on the tokens before it, which is why it is called auto-regressive. A setting called temperature reshapes those probabilities: a low temperature makes the most likely tokens even more likely, and a high temperature flattens the probabilities so less likely tokens get picked more often. Fluent but false output is often called a hallucination.

Questions

It finds patterns in words and picks likely next words. That can look like understanding, but it is not checking whether what it writes is true, so check important facts somewhere else.

It does not always pick the top word. It sometimes chooses at random, with likely words more likely to be chosen, so the reply can change.

They are cousins. Both predict words. Autocorrect picks the one word you most likely meant. A chatbot keeps predicting next words until it has written a whole reply.