What Is an LLM? How Large Language Models Work

Published:

15 minute read

Acar Diveroli
Written by: Acar Diveroli
Faint text streams flow into a wireframe sphere; a blue card with the next word, rates, leaves it above two dashed options

A marketing manager pastes fifty customer reviews into a chat window and asks for the three most common complaints. A developer asks for a regular expression that matches order numbers. A student asks why the sky is blue. All three get a fluent answer within seconds from the same kind of software: a large language model, or LLM, which writes its reply one small piece of text at a time.

This post explains what an LLM is and what "large" means, how it turns a prompt into an answer, how it is trained and on what data, how it differs from a chatbot, an AI agent and a search engine, where it fails, and how open-weight and closed models differ.

What is a large language model?

A large language model is a neural network trained on a very large collection of text to predict which piece of text comes next, and it uses that single skill to write, summarise, translate, answer questions and produce code. A language model in general is any model that assigns probabilities to sequences of words; the next-word suggestions on a phone keyboard rest on a similar idea at a tiny scale.

"Large" has two meanings. The first is the number of parameters. A parameter is one number inside the model, a weight that training adjusts again and again until the model's guesses improve; everything the model has learned is stored in these numbers. The second is the amount of training text. GPT-3, described by OpenAI researchers in 2020, had 175 billion parameters, which its authors called ten times more than any previous non-sparse language model, and it was trained on 300 billion tokens.

An LLM does not contain a database of facts that it looks up. What it "knows" exists only as patterns spread across its parameters, which is why it can explain a concept well and still get a date, a price or a quotation wrong.

How does an LLM work?

An LLM generates text one token at a time: it turns your words into numbers, passes them through many layers that work out how the words relate to each other, scores every possible next token, picks one, adds it to the text and repeats. In order:

  1. Split the text into tokens. A token is a word or a piece of a word; a common word is often one token, while a rare one is cut into several pieces. What Are Tokens in AI? shows how the split works and why it decides cost and limits.
  2. Turn each token into numbers. Every token is mapped to a long list of numbers, a vector, that places it in a space where words used in similar ways sit close together.
  3. Pass the vectors through transformer layers. Most current LLMs are built on the transformer, the architecture introduced in the 2017 paper Attention Is All You Need. Its core is attention: each token looks at the other tokens in the text and weighs which ones matter for its meaning. In "the bank raised its rates", attention lets "rates" tell the model that "bank" is a lender, not a riverside.
  4. Score every possible next token. The last layer outputs a probability for each token in the model's vocabulary, the fixed list of tokens it knows.
  5. Pick one. The model takes the most likely token or samples from the likely ones. A setting called temperature controls how much randomness is allowed, which is why the same question can get two differently worded answers.
  6. Append and repeat. The chosen token is added to the text and the whole loop runs again, until the model produces a stop token or reaches its length limit.

This loop explains the model's strengths and its weak spots. It writes fluent, well-structured text because fluent text is exactly what it was trained to continue. It does not stop to check whether a sentence is true, because nothing in the loop asks that question.

How is an LLM trained?

An LLM is trained in stages: pretraining on a huge text corpus teaches it language and general knowledge, then instruction tuning and preference tuning teach it to follow requests and give the kind of answers people want.

Pretraining is self-supervised, which means the text supplies its own answers. The model reads a passage, guesses the next token, compares its guess with the real one and adjusts its parameters slightly so the right token becomes more likely. Repeated across hundreds of billions of tokens, this stage takes most of the computing work. The result is a base model: it continues text well, but asked a question it may simply write more questions.

Instruction tuning trains the base model further on examples of prompts paired with good responses, often written by people. This is also called supervised fine-tuning, and it turns a text continuer into an assistant that answers.

Preference tuning uses rankings instead of written answers. People compare several responses to the same prompt and mark the better ones, a separate reward model learns from those rankings, and reinforcement learning nudges the LLM toward answers that score well. This is reinforcement learning from human feedback (RLHF). In the InstructGPT paper from 2022, human evaluators preferred the answers of a 1.3-billion-parameter model tuned this way to those of the 175-billion-parameter GPT-3, so size alone does not make a model useful.

Companies can repeat a smaller version of the second stage on their own examples to teach a model a format, a tone or a narrow task. That process, and when it beats a well-written prompt, is covered in What Is Fine-Tuning?

Where does the training data come from?

Most pretraining text comes from public web pages, together with books, code, encyclopedias and other curated or licensed sources, filtered for quality and cleared of duplicates. The GPT-3 paper is one of the clearest public descriptions of a training mix: 60% of the examples seen during training came from a filtered copy of Common Crawl, with the rest drawn from a web-text set, two book collections and English Wikipedia.

Common Crawl is an archive of web pages that describes itself as a free, open repository of web crawl data that anyone can use. The GPT-3 authors did not use it raw: they kept the pages most similar to high-quality reference texts and removed near-duplicates, because unfiltered crawl data lowered quality.

Web data matters after pretraining too. Teams tune models on domain examples and build test sets that check whether an assistant answers correctly about prices, products or rules in different markets. Many products also use retrieval-augmented generation (RAG): the app fetches relevant pages at question time and hands them to the model with the question.

Those pages often differ by country, so a check run from one office sees only one version. A Residential Proxy sends each request through a home connection in the country and city you choose, so you collect the public page as a local visitor sees it.

For teams that need public pages from many countries at scale, our data scraping solutions page covers the proxy pools built for collection jobs, and our sales team can discuss managed page extraction. Whichever route you take, collect public pages only, follow robots.txt and the site's terms, keep request rates low, use an official API where one exists and leave out personal data you do not need.

What is the difference between an LLM, a chatbot, an AI agent and a search engine?

An LLM is the model itself; a chatbot is an application built around a model; an AI agent is a system that lets a model use tools and work through steps toward a goal; a search engine finds and ranks pages that already exist.

TermWhat it isWhat it doesExample
LLMA trained modelPredicts the next tokengpt-oss-20b
ChatbotAn app built on a modelHolds a conversationChatGPT, a help-desk bot
AI agentA model plus tools and a loopPlans and acts in stepsA travel-booking agent
Search engineAn index of web pagesFinds and ranks pagesGoogle Search

The lines blur in daily use: chatbots call a search engine before answering, search engines show AI-written summaries, and agents run on the same models. The split still helps when something goes wrong, because a wrong fact usually comes from the model, a missing source from the retrieval step and a wrong action from the agent's tools.

What are the limits of LLMs?

An LLM produces likely text, not verified text, and most of its limits follow from that:

  • Hallucination. The model can state false facts, invent sources or misquote a document with complete confidence; What Is AI Hallucination? explains why and how to reduce it.
  • Knowledge cutoff. Its training data ends at a date, so without a search tool it knows nothing about later events, prices or laws.
  • Context window. The model can consider only a limited number of tokens at once, counting your prompt, any pasted documents and its own answer; beyond that, text has to be cut or summarised.
  • Cost per token. API providers bill by the token, usually with separate prices for input and output, so long documents and long answers cost more.
  • Bias. Patterns in the training data, including stereotypes and the dominance of some languages, carry over into the answers.
  • Privacy. Text you paste into an AI tool goes to its provider; customer or employee details in a prompt count as personal data under the GDPR in the EU and KVKK in Türkiye, so check the provider's terms first.

What is the difference between open-weight and closed models?

An open-weight model has its parameters published, so anyone can download it, run it on their own hardware and fine-tune it within the licence; a closed model is available only through the vendor's app or API. OpenAI's gpt-oss models are an example of the first kind, released under the Apache 2.0 licence, and Mistral publishes both open-weight and commercial models.

Open-weight is not the same as open source. The weights may be free to use while the training data and the training code stay private, and licences range from permissive to restrictive. The practical trade-off is control against convenience: a model on your own servers keeps data in-house and can be changed freely, while a closed model needs no hardware and is updated by its vendor.

Two more terms often appear next to LLM. A mixture-of-experts (MoE) model splits some layers into many smaller sub-networks and sends each token to only a few of them, so a very large model does less work per token. A multimodal model accepts or produces more than text, such as images and audio, in the same model.

What are LLMs used for?

LLMs are used wherever work starts or ends with text. In offices they draft emails, reports and product descriptions and summarise long documents, meeting notes and support tickets. Customer service teams use them to answer routine questions and route the rest to a person, translators use them for first drafts, and marketers use them to sort open-ended survey answers into themes.

Developers use them to write and explain code and to turn plain-language requests into database queries. Analysts use them to pull names, dates and amounts out of unstructured text. Search products use them to write answers on top of retrieved pages, and agents use them as the part that decides what to do next.

Common mistakes

  • Treating the answer as a fact. Fluency is not evidence; check names, figures, dates and quotations against a primary source.
  • Asking about recent events without a search tool. A model without retrieval answers from its training data and may not say that it is out of date.
  • Pasting confidential or personal data without checking the terms. Read the provider's data policy and remove what the task does not need.
  • Choosing a model by parameter count alone. Tuning, training data and the task itself matter as much as size.
  • Expecting the same answer every time. Sampling makes outputs vary; fix the format with clear instructions and examples instead.
  • Asking for exact calculations or counts in plain text. Give the model a calculator or code tool, or check the arithmetic yourself.

Decision guide

NeedUse
Draft, rewrite or summarise textA chat assistant, then edit the result
Answers about current events or pricesA model with web search, showing sources
Answers from your own documentsRetrieval over those documents (RAG)
One narrow task in a fixed formatBetter prompts first, fine-tuning if they fail
Data that must stay on your serversAn open-weight model you host
Broad tasks with no hardwareA closed model through an app or API
Legal, medical or financial factsPrimary sources; the LLM only for drafting

Frequently asked questions

Is ChatGPT an LLM?

Not exactly: ChatGPT is a chatbot, an application, and the engines behind it are OpenAI's GPT large language models. The app adds what the model lacks on its own, such as the chat interface, the conversation history, safety layers and tools.

What is the difference between an LLM and generative AI?

Generative AI is the broad category of AI that creates new content, whether text, images, audio, video or code, and an LLM is the part of that category that works with language. Image generators are generative AI but not LLMs. Many newer LLMs also read images or audio, which blurs the boundary.

What does LLM stand for?

In AI, LLM stands for large language model. The same letters, written LL.M., also stand for Master of Laws, a postgraduate law degree that has nothing to do with AI.

How is an LLM trained?

It is trained in three broad stages. Pretraining teaches it to predict the next token across a very large body of text, instruction tuning trains it on examples of good answers, and preference tuning uses people's rankings of answers, often through RLHF, to make its replies more helpful.

Can an LLM browse the internet?

Not by itself: the model knows only its training data and whatever is in the prompt. An application or API can give it a web search tool. Anthropic's documentation, for example, describes a tool that developers add to a request so the model can answer with up-to-date information beyond its knowledge cutoff, with citations to the pages it used. The model then reads results that software fetched for it; clicking through pages the way a person does is what AI agents and AI browsers add on top.

Does an LLM understand what it says?

There is no settled answer. An LLM learns the statistical patterns of text, and researchers Emily Bender and Alexander Koller argued in a 2020 ACL paper that a system trained only on the form of language has no way to learn its meaning. Others argue that predicting text this well requires some internal representation of meaning. Either way, the practical point is the same: the model has no built-in check that what it says is true.

Summary

A large language model is a neural network trained on a very large amount of text to predict the next token, and it builds every answer by repeating that prediction. It is pretrained mainly on public web pages, books and code, then tuned with examples and human rankings so it follows instructions. That makes it good at drafting, summarising, translating and coding, and unreliable as a source of facts: it can hallucinate, it stops learning at its training cutoff and it sees only what fits in its context window. Use it for fast first drafts, ground it in current sources when facts matter, and keep a person in the loop for anything that has to be right.

Ask ChatGPTAsk Claude