How Do AI Chatbots Work? Large Language Models Explained
A chatbot answer is built one small piece of text at a time. Here is how language models are trained, why they sound sure of themselves, and where they go wrong.
AI chatbots such as ChatGPT, Claude and Gemini run on large language models: neural networks trained to predict the next small piece of text, called a token, given everything that came before. They learn patterns from enormous collections of text, then are tuned with human feedback to follow instructions. They generate answers one token at a time, which is why they can sound fluent and confident while being wrong.
The basic recipe has three stages: break text into tokens, pretrain on prediction, then fine-tune for helpful conversation.
What is a token?
A token is the unit of text a language model reads and writes. It is often a whole common word, a piece of a longer word, a number or a punctuation mark. A word like “unbelievable” might be split into several tokens, while “the” is usually one. Each model has a fixed vocabulary of tokens, and each token is turned into a list of numbers the network can work with.
When you type a question, the model converts it into tokens, processes them, and produces a probability for every possible next token. It picks one, adds it to the text, and repeats. A long answer is simply that loop running hundreds or thousands of times.
How is a language model trained?
The first stage, pretraining, teaches the model to predict the next token across a huge body of text such as books, articles, websites and code. Each time it guesses wrong, its internal settings, called parameters, are adjusted slightly so the correct token becomes a little more likely next time. Modern models have billions of parameters, and training takes large clusters of specialized chips running for weeks or months.
Most current models use a neural network design called the transformer, introduced in a 2017 paper titled “Attention Is All You Need.” Its key idea, attention, lets the model weigh how much each earlier token matters when predicting the next one, so a pronoun can be linked to the right noun many sentences back. The design also trains efficiently on modern hardware, which made very large models practical.
Prediction sounds simple, but doing it well across so much text forces the model to pick up grammar, facts that appear often, styles of writing and some reasoning patterns. How much of this counts as understanding is a subject of active debate among researchers.
How do chatbots learn to follow instructions?
A pretrained model continues text; it does not naturally answer questions or refuse harmful requests. Developers add a second stage of training to turn it into an assistant. A widely cited 2022 paper, “Training language models to follow instructions with human feedback,” describes one approach in three steps.
- People write example answers to prompts, and the model is fine-tuned on those demonstrations.
- People rank several model answers from best to worst, and a separate “reward model” learns to predict their preferences.
- The language model is further trained with reinforcement learning to produce answers the reward model scores highly.
The paper reported that human raters preferred answers from a 1.3-billion-parameter model trained this way over those from the original 175-billion-parameter GPT-3, despite the tuned model having about 100 times fewer parameters. This approach is known as reinforcement learning from human feedback, or RLHF. Companies now use many variations, including AI-generated feedback and written rules.
| Stage | What the model learns | Training data |
|---|---|---|
| Tokenization | How to split text into units | Large text sample |
| Pretraining | To predict the next token | Huge collections of text and code |
| Instruction tuning | To follow requests in a helpful format | Human-written example answers |
| Feedback training (RLHF and variants) | To prefer answers people rate highly | Human or AI rankings of answers |
Why do AI chatbots make things up?
A language model generates text that is likely given its training, not text checked against a database of facts. When the training data is thin, conflicting or missing on a topic, the model can still produce a smooth, plausible answer, and it may present an invented citation, quote or number with the same confident tone as a correct one.
The US National Institute of Standards and Technology lists this among the risks of generative AI in its 2024 Generative AI Profile (NIST AI 600-1). NIST calls it “confabulation,” defined as the production of confidently stated but erroneous or false content, known colloquially as “hallucinations” or “fabrications.”
Developers reduce the problem with more careful training, by teaching models to say when they are unsure, and by connecting them to search or documents. None of these removes it completely, so important facts from a chatbot are worth checking against the original source.
Do chatbots look things up on the internet?
Not on their own. A base language model only knows what was in its training data, up to a cutoff date. It cannot see today’s news or check a web page unless the product connects it to a search tool or gives it documents to read.
Many chatbot products now add those tools. When a chatbot searches the web, it reads the pages it finds and writes an answer based on them, often with links. The quality of that answer then depends on both the model and the sources it picked.
What can’t language models do?
Language models do not have senses or direct experience of the world, and they do not remember past conversations unless a product stores them and feeds them back. They can make arithmetic slips, misread a question, or follow a pattern in the prompt that leads them astray. They also reflect the biases and gaps of their training text.
Researchers also use machine learning, a broader family of techniques, in science well beyond chatbots. Biologists apply it to sort huge sets of animal recordings, as described in how whales communicate. And comparisons between artificial and animal intelligence remain tricky, a theme in how smart octopuses are.
Bottom line
An AI chatbot runs on a large language model that splits text into tokens and predicts the next one, over and over. It is pretrained on huge text collections, usually with a transformer network, then tuned with human examples and feedback to act as an assistant. Because it predicts likely text rather than checking facts, it can confabulate, and it only knows recent events if connected to search.
FAQ
Does ChatGPT understand what it says?
Researchers disagree on how to describe it. The model has learned statistical patterns that let it use language very well, but it does not experience the world the way people do, and it can produce confident errors.
Where does an AI chatbot get its information?
From patterns in the text it was trained on, up to a cutoff date, plus any search results or documents the product gives it while answering. Without those tools it cannot check current facts.
What is a hallucination in AI?
It is when a model states something false with confidence, such as a made-up study or quote. NIST calls this confabulation in its 2024 Generative AI Profile.
Why are they called large language models?
They model language, meaning they estimate how likely different pieces of text are, and they are large in the number of parameters and in the amount of training text. Modern models have billions of parameters.
