A customer types "my order never came" into a shop's help search, and the article that answers it is called "Tracking a missing delivery". A keyword search returns nothing useful, because the two share no word that matters. A semantic search puts that article first, because both sentences are about the same problem.
This post explains what semantic search is, how it works from text to ranked results, what an embedding is, how it compares with keyword and hybrid search, what a vector database does, what it changes for SEO and where it fails.
What is semantic search?
Semantic search is a way of retrieving documents by what a query means rather than by which words it contains. "Semantic" simply means "relating to meaning": a semantic engine treats "cheap flights to Rome" and "low-cost airline tickets to Rome" as the same request, although they share only one word.
The older approach is keyword search, also called lexical search. The engine looks the query's words up in an inverted index, a list of which documents contain which words, and scores the documents that contain them. It is fast and exact but blind to meaning: a page that says "automobile" does not match a search for "car" unless someone added that synonym by hand.
Today "semantic search" almost always means dense retrieval: a neural network turns texts into vectors, lists of numbers, and the engine compares them by distance.
How does semantic search work?
Semantic search works in two phases: ahead of time it turns your documents into vectors and stores them, and for every query it turns the query into a vector and looks for the nearest stored ones.
- Split the documents into passages. One vector for a long page blurs its topics, so most systems index paragraphs or sections, each with its title and source address.
- Turn each passage into an embedding. An embedding model, a neural network trained for this job, reads the passage and outputs a fixed-length list of numbers.
- Store the vectors in an index, together with the passage text and metadata such as date, language and who may see it.
- Embed the query with the same model. A different model, or even a different version of it, produces numbers that cannot be compared.
- Find the nearest neighbours. The index returns the stored vectors closest to the query vector; on large collections this is an approximate search that gives up a little accuracy for a lot of speed.
- Rank and filter. Results are ordered by similarity, filtered by metadata (only this language, only documents this user may open) and cut to the best 10 or 50.
- Rerank, if needed. A reranker is a slower model that reads the query and each candidate passage together. It is too slow for the whole collection, so it only reorders the short list.
What is an embedding?
An embedding is a list of numbers that represents a piece of text so that texts with similar meanings get similar numbers. Google's machine learning course calls it "a vector representation of data in embedding space", in which the distance between two items can be read as a measure of how similar they are; its lesson on embedding space shows this with foods placed along imagined dimensions.
A made-up example makes the idea concrete. Suppose a tiny model described every text with three numbers: how much it is about flying, about price and about hotels. Real models use hundreds or thousands of numbers whose dimensions have no names; the labels here only make the idea visible.
| Text | Illustrative vector | Distance to the query |
|---|---|---|
| Query: "budget air travel Rome" | [0.85, 0.85, 0.05] | n/a |
| "cheap flights to Rome" | [0.90, 0.80, 0.10] | 0.09 |
| "low-cost airline tickets to Rome" | [0.80, 0.90, 0.10] | 0.09 |
| "best hotels in Rome" | [0.10, 0.20, 0.90] | 1.31 |
The query shares only "Rome" with all three texts, so a keyword engine cannot tell them apart. In the vector space the two flight texts sit next to the query and the hotel text sits far away, so semantic search ranks the flights first. "Distance" is the ordinary straight-line distance between two points, in three dimensions instead of two.
Modern models are also contextual: the same word gets different numbers depending on the sentence around it, so "orange juice" and "an orange dress" land in different neighbourhoods.
Sentence embeddings made this practical. In the 2019 Sentence-BERT paper, Nils Reimers and Iryna Gurevych worked out that finding the most similar pair among 10,000 sentences with BERT, which has to read every pair together, would take about 65 hours; with one embedding per sentence compared by cosine similarity, it took about 5 seconds. Every semantic engine relies on that split: embed documents once, compare cheap vectors at query time.
Semantic vs keyword vs hybrid search: what is the difference?
Keyword search matches the words of the query, semantic search matches its meaning, and hybrid search runs both and merges the results. The classic keyword ranking formula is BM25, from the Okapi system presented at the TREC-3 conference in 1994. It ranks a document higher when the query words appear in it often and are rare elsewhere in the collection, and it corrects for document length.
| Keyword (BM25) | Semantic (vectors) | Hybrid | |
|---|---|---|---|
| Matches on | Shared words | Similar meaning | Both |
| Synonyms and paraphrases | Only if listed | Yes | Yes |
| Exact codes and names | Strong | Weak | Strong |
| Why a result matched | Visible | Hard to explain | Partly visible |
| What it needs | An inverted index | A model and a vector index | Both |
On questions written in natural language, dense retrieval wins clearly. In the Dense Passage Retrieval paper from 2020, a retriever trained on question and passage pairs found a passage containing the answer among its top 20 results 9 to 19 percentage points more often than a strong BM25 system, depending on the dataset.
Hybrid search exists because each method fails where the other works. The usual way to merge the two lists is reciprocal rank fusion (RRF), from a 2009 SIGIR paper by Gordon Cormack and colleagues: a document receives 1/(60 + its rank) from each list it appears in, and the sums set the final order. A passage ranked first by keywords and third by vectors scores about 0.0323 and beats one that only a single list ranked first, at about 0.0164. Because RRF uses only positions, the two systems' incompatible scores never have to be compared.
What is a vector database?
A vector database is a database built to store embeddings and quickly find the ones nearest to a given vector. For each item it keeps the vector, the original text or a pointer to it, and metadata such as address, date, language or access rights, so results can be filtered and shown.
It is needed because comparing a query with every stored vector, called exact or brute-force search, gets slow as a collection grows into millions. Vector databases build an approximate nearest neighbour (ANN) index instead. A common one is HNSW (hierarchical navigable small world), a layered graph in which a search hops from neighbour to neighbour towards the query and examines only a small part of the collection. "Approximate" means it can occasionally miss one of the true nearest vectors in exchange for a far faster answer.
Vector search comes in four common forms, and what you already run usually decides between them:
- Dedicated vector databases: systems built only for this job, with ANN indexes, metadata filters and replication.
- Vector support in a database you already use: pgvector, for example, adds vector similarity search to PostgreSQL. It searches exactly by default and switches to approximate search when you add an HNSW or IVFFlat index.
- Search engines with vector fields: keyword search engines that also store vectors, so a hybrid query runs in one system.
- Libraries inside your program: Faiss, an MIT-licensed library developed mainly at Meta's AI research group, builds and searches vector indexes in memory, without a server.
An index is only as current as the documents behind it. Pages change, so they have to be fetched again and re-embedded on a schedule, which is the job of a web crawler when the documents live on the web.
Where do you meet semantic search?
- Google Search: Google says its systems match the meaning of queries and pages, not only their words.
- Site and help-centre search: "my order never came" finds the article about missing deliveries.
- Chatbots and AI assistants: in RAG (retrieval-augmented generation), semantic search finds the passages the model answers from.
- Online shops: "shoes for standing all day" finds work shoes whose titles never use that phrase.
Semantic search over web content needs the pages first. Your own help centre is easy to index; an assistant that answers from public sources, such as product documentation or prices across many shops, has to collect those pages, extract the readable text and fetch it again when it changes. Some pages also differ by the visitor's country, so the copy you index should come from the market your users are in, for example through a proxy in that country. The collection side is covered in Web Scraping for RAG.
What does semantic search change for SEO?
For SEO, semantic search means Google matches the meaning of a query with the meaning of a page, so a page can rank for phrasings it never uses and repeating keyword variants adds nothing. Google described the shift in 2019, when it said BERT helped it understand about one in 10 English searches in the US; in one example, the word "to" in "2019 brazil traveler to usa need a visa" decides which way the person is travelling.
Google's guide to its ranking systems describes three such systems. BERT understands "how combinations of words express different meanings and intent"; neural matching compares "representations of concepts in queries and pages"; RankBrain returns relevant content "even if it doesn't contain all the exact words used in a search". Google does not publish how they are built, so treat claims about "Google's vectors" as guesses.
The SEO Starter Guide draws the practical conclusion: Google's language matching "can understand how your page relates to many queries", and you need not anticipate every variation of a search. Keywords have not disappeared, though: Google still calls a page containing the query's words "the most basic signal" of relevance.
- Write one complete page per topic and intent, not one thin page per keyword variant.
- Answer the question early and plainly, the way readers ask it, then cover their follow-up questions.
- Use the exact names readers use for products, error codes and places; matching by meaning does not replace an exact match for a code.
- Skip the tricks. Keyword stuffing, hidden text and pages mass-produced to cover phrasings are listed in Google's spam policies.
Where does semantic search fail?
Semantic search fails where the exact string matters more than the meaning: product codes, part numbers, error codes, names and numbers. To an embedding model, "error 0x80070005" and "error 0x80070057" can sit very close together because they are the same kind of thing, but a user who typed one wants exactly that one.
- Rare names and new words: a model trained before a product launched has never seen its name.
- Negation and limits: "laptops without a touchscreen" can rank touchscreen laptops high, because the text is about touchscreens. Prices, dates and sizes work better as filters.
- No empty result: nearest neighbours always exist, so a question your documents cannot answer still returns the "closest" passages. Without a similarity threshold, an assistant answers from something irrelevant.
- Languages and jargon: a model trained mostly on general English may do worse elsewhere; test it with your own queries.
The usual fix for the first three is hybrid search: a keyword index for exact matches and filters, vectors for paraphrases.
Common mistakes
- Mixing embedding models. Queries and documents embedded with different models or versions cannot be compared; a new model means re-embedding everything.
- One vector per long page. An average of ten topics matches none of them well. Index passages.
- Dropping keyword search. Users still paste order numbers, error codes and names.
- No test set. Keep a few dozen real queries with the result each should return, and check the top 10 after every change.
- Leaving permissions out of the index. Unless access rights are filtered with each passage, the index hands internal documents to the wrong people.
- Indexing what you may not use. Pages whose terms forbid reuse, or pages full of personal data covered by the GDPR or Türkiye's KVKK, do not become fair game because they were turned into vectors.
Decision guide
| Your situation | What to use |
|---|---|
| Users search by product code, SKU or name | Keyword search with a synonym list |
| Users ask questions in their own words | Semantic search with an embedding model |
| Both kinds of queries | Hybrid search merged with rank fusion |
| A few thousand passages | Exact search in memory or your current database |
| Millions of passages with filters | An ANN index in a vector database or search engine |
| A chatbot that must cite sources | Hybrid retrieval, a reranker, a source address per passage |
| A page that misses related phrasings in Google | One complete page per topic and intent |
| Answers drawn from public web pages | Collect and refresh the pages before indexing |
Frequently asked questions
Is semantic search the same as vector search?
Not quite. Semantic search is the goal, finding results by meaning; vector search is the usual technique for it, finding the nearest vectors in an index. Vector search can also compare images or sounds, and a rough semantic search can be built without vectors, from synonym lists and query rewriting.
What is the difference between semantic search and RAG?
Semantic search finds and ranks passages. RAG places the passages a search returned in a language model's prompt and has the model write an answer from them, so semantic search is usually the retrieval step inside RAG. AI research agents run many such searches in a loop, as our deep research guide explains.
What is cosine similarity in semantic search?
Cosine similarity measures the angle between two vectors: 1 means they point the same way, and values near 0 mean the texts have little in common. It ignores length, so a long and a short text on the same subject can still score close to 1. In the example above, both flight texts score 0.997 against the query and the hotel text 0.269.
What is the difference between a vector database and a relational database?
A relational database finds rows that meet an exact condition, such as a price below 50, and returns nothing when no row qualifies. A vector database finds the items most similar to a given vector and always returns the closest ones. Many systems use both: relational filters, vector ranking.
Do I need a vector database for semantic search?
Not always. For a few thousand passages, exact search in memory or a vector extension such as pgvector in a database you already run is usually enough. A dedicated vector database pays off with millions of vectors, frequent updates, strict filters or several applications sharing one index.
What is semantic SEO?
Semantic SEO is an industry name for writing pages around topics, entities and search intent rather than single keywords. It follows from how Google says it matches meaning: answer the question completely, use your readers' words, and let one good page cover the variants of a query.
Summary
Semantic search finds results by meaning: a model turns passages and queries into embeddings, an index finds the passages nearest to the query, and an optional reranker refines the order. It beats keyword search on paraphrases and natural questions and loses on exact codes, names and numbers, which is why hybrid search with rank fusion is a common production choice. For SEO, Google's own guidance points to complete pages for topics and intents rather than keyword variants. When the index is built from public web pages, collecting them is the hard part; our Web Scraping API, a managed page extraction service available through the sales team, is described on the data scraping page.




