A support team turns 100,000 help-desk answers into embeddings and builds a "find similar tickets" button. The prototype compares each new ticket with every stored vector, and it feels instant. Then the archive grows to 20 million passages, every search reads all 20 million vectors, and the button slows down with every import. Closing that gap between "compare with everything" and "find the closest few in milliseconds" is the job of a vector database.
This post explains what a vector database stores, how a search runs, which similarity measures and index types it uses (HNSW, IVF and product quantization in plain words), how filters and hybrid search fit in, when an extension in your current database is enough and what it costs in memory.
What is a vector database?
A vector database is a database built to store vectors and answer one kind of question quickly: which stored items are closest to this one? A relational database answers "which rows match this condition"; a vector database answers "which items are most similar", and it always returns the nearest ones, even when none of them is a good match.
Each record usually holds four things:
- An ID, so the result can be traced back to a document, product or image.
- The vector, a fixed-length list of numbers produced by an embedding model.
- Metadata, such as language, date, country, source address or who may see the item, used to filter results.
- The content or a pointer to it, so the application can show the passage or product the vector stands for.
{
"id": "faq-0412#2",
"vector": [0.021, -0.113, 0.087, "... 381 more numbers"],
"metadata": { "lang": "en", "country": "DE", "updated": "2026-11-02" },
"text": "Orders to Germany usually arrive within three working days."
}What is a vector embedding?
A vector embedding is a list of numbers that an embedding model, a neural network trained for this job, produces for a piece of data so that similar meanings end up close together. "Cheap flights to Rome" and "low-cost airline tickets to Rome" get nearby vectors although they share almost no words; how that matching by meaning works for search is covered in What Is Semantic Search?.
Real embeddings have hundreds to a few thousand numbers, called dimensions, and the model fixes that count. Two rules follow: every vector in one index has the same number of dimensions, and queries must be embedded with the same model and version as the stored items. Vectors from two different models are not comparable, even when they have the same length.
How does a vector database work?
A vector database works in two phases: it ingests and indexes items ahead of time, then answers each query by searching that index.
- Prepare the items. Long documents are split into passages, each with its source address and other metadata.
- Embed them. The embedding model turns every passage, image or product into a vector, usually in your application or pipeline rather than inside the database.
- Store the records. The database writes ID, vector, metadata and content.
- Build the index. An ANN index organises the vectors into a graph or into clusters so that a search can skip most of them.
- Embed the query. The user's question goes through the same model and becomes a query vector.
- Search and filter. The index returns candidates close to the query, and metadata conditions such as "language is German" remove the ones that do not qualify.
- Return the top k. The k nearest items come back with their similarity scores, ready to show, rerank or hand to a language model.
Updates follow the same path: a changed document is re-embedded and its old vector replaced, otherwise the index keeps answering from text that no longer exists.
Which similarity measures does a vector database use?
Vector databases rank results with one of three common measures: cosine similarity, dot product or Euclidean distance. The right one is the measure the embedding model was trained for, which its documentation states.
| Measure | What it compares | Higher means |
|---|---|---|
| Cosine similarity | The angle between two vectors | More similar |
| Dot product | Angle and length together | More similar |
| Euclidean (L2) distance | Straight-line distance | Less similar |
When all vectors are normalised to length 1, which many embedding models do, the three measures put results in the same order: cosine equals the dot product, and a smaller distance means a larger dot product. That is why the pgvector documentation recommends inner product for normalised vectors: same ranking, least arithmetic. On vectors that are not normalised, indexing with one measure and querying with another silently changes the ranking.
What is the difference between exact and approximate nearest neighbour search?
Exact search compares the query with every stored vector and always finds the true nearest ones; approximate search checks only a small, well-chosen part of the collection and usually finds almost all of them. Exact search, also called brute force or flat search, costs time in direct proportion to the number of vectors.
Approximate nearest neighbour (ANN) search trades a little accuracy for a lot of speed. The accuracy is measured as recall: if the index returns 9 of the true 10 nearest items, recall at 10 is 0.9. Every ANN index has settings that move along this line, so tune it on your data instead of trusting the defaults.
We ran a small version of this trade-off ourselves, in plain JavaScript and on synthetic data. On our test machine, scanning 100,000 vectors of 384 numbers took about 40 milliseconds per query; a minimal IVF index with 256 clusters answered in about half a millisecond when it searched one cluster, but found only about 7 of the true 10 nearest neighbours, and searching 32 clusters took about 10 milliseconds and found about 8.5 of them. Real indexes are far more refined, yet the pattern is the same: more work per query, higher recall.
What are the main vector index types?
The main index types are flat (no index), graph indexes such as HNSW, cluster indexes such as IVF, and compression with product quantization, which is often combined with IVF. Each one balances speed, memory and recall differently.
| Index | How a search runs | Main cost |
|---|---|---|
| Flat | Compares the query with every vector | Slow on large sets |
| HNSW | Hops through a layered graph of neighbours | Memory, slow build |
| IVF | Scans only the closest clusters | Lower recall at equal speed |
| PQ | Compares short compressed codes | Approximate distances |
HNSW (hierarchical navigable small world) was described by Yury Malkov and Dmitry Yashunin in a 2016 paper. Every vector is linked to some of its neighbours, and a few randomly chosen vectors also appear in sparser upper layers. A search starts at the top, takes long jumps across the collection, then moves down layer by layer to finer neighbourhoods, so the work grows roughly with the logarithm of the collection size rather than with the size itself. The price is memory for all those links and a slower build.
IVF (inverted file index) groups the vectors into clusters with k-means when the index is built, as the Faiss library paper describes. A query is compared with the cluster centres first and then only with the vectors in the few closest clusters; the number of clusters searched, often called nprobe, is the main recall dial.
Product quantization (PQ), introduced by Hervé Jégou, Matthijs Douze and Cordelia Schmid in a 2011 paper, compresses the vectors themselves. It splits each vector into short sub-vectors and replaces each sub-vector with the number of the closest entry in a small learned codebook. A vector of 768 numbers stored as 4-byte floats takes 3,072 bytes; as 64 one-byte codes it takes 64 bytes, at the cost of approximate distances. Systems often keep the full vectors on disk and use them to re-rank the short list.
How does metadata filtering work in a vector database?
Metadata filtering limits a vector search to items that meet exact conditions, such as language, country, date, price or access rights. It turns "the 10 nearest passages" into "the 10 nearest passages in German, updated this year, that this user may read".
Where the filter runs matters. If the database first finds the nearest candidates and then removes the ones that fail the filter, a strict filter can leave very few results: pgvector's documentation shows an HNSW search with default settings and a condition matched by 10% of rows returning only about 4 rows on average. Its version 0.8.0 added iterative index scans, which keep searching until enough rows pass; other systems filter during the graph or cluster search itself. Either way, test your strictest real filters.
What is hybrid search in a vector database?
Hybrid search runs a keyword search and a vector search on the same collection and merges the two result lists. Vectors find paraphrases; keywords find exact strings such as order numbers, error codes and product names, which embeddings tend to blur. Many vector databases store a keyword index or sparse vectors (long vectors of mostly zeros, one position per word) next to the embeddings so one query can use both; otherwise the application merges the two lists itself.
Do you need a dedicated vector database or a vector extension?
You need a dedicated vector database when vectors are a central, large and busy part of your system; when they are one more column next to data you already keep, a vector extension or a library is usually enough. The choice is about operations, not about which product is "best".
- A vector extension in your current database. pgvector, for example, adds a vector type, distance operators and HNSW and IVFFlat indexes to PostgreSQL. Its search is exact by default, a plain
vectorcolumn stores up to 16,000 dimensions and can be indexed up to 2,000, and filters, joins, backups and access rules stay the ones you already have. - A dedicated vector database. A separate system built around ANN search, with its own scaling, replication, filtering and hybrid features. It pays off with very large collections, high query rates, frequent updates or several applications sharing one index.
- A library inside your program. Faiss, for example, builds and searches indexes in memory without a server. It fits batch jobs and offline duplicate detection, but you handle storage, updates and filtering yourself.
Sometimes you need none of them. Up to a few hundred thousand vectors can be searched exactly; our 100,000-vector scan took about 40 milliseconds in plain JavaScript. A search by SKU or order number, or "orders over 500 units last month", is a keyword or SQL query and should never be "almost right". Start where your data already lives and move only when measured recall and latency tell you to.
How much memory does a vector database need?
Raw vectors need number of vectors × dimensions × bytes per number, before any index. With the common 4-byte floats, one million vectors of 768 dimensions take about 3.1 GB, and one million vectors of 1,536 dimensions about 6.1 GB; ten million of the latter take about 61 GB.
The index adds to that: an HNSW graph stores a list of neighbour IDs for every vector, and fast search expects the graph and usually the vectors to sit in RAM. Half-precision numbers (2 bytes instead of 4), product quantization codes and a model with fewer dimensions all shrink the bill, and each can cost some recall, so measure before and after.
Re-embedding is the hidden cost. A new embedding model makes every stored vector incomparable with new queries, so the whole collection has to be embedded again and the index rebuilt.
Where are vector databases used?
Vector databases are used wherever an application has to find "things like this one" among many items.
RAG and AI assistants. In retrieval-augmented generation (RAG), the vector database finds the passages most related to a user's question and hands them to a large language model, which writes the answer from them and can cite their sources.
Search, recommendations and duplicates. A help centre or shop search finds pages by meaning; "similar products" are the nearest neighbours of the item just viewed; two tickets or listings that say the same thing in other words sit so close together that a distance threshold flags them as duplicates. Models that embed images and text in the same space also let a text query find matching photos.
Where do the vectors' source pages come from?
A vector index is only as good as the content behind it. When an assistant answers from public web pages, such as product documentation, prices or local regulations, those pages have to be collected, cleaned and fetched again when they change; turning HTML into clean, source-tagged chunks is covered in Web Scraping for RAG.
Many public pages also differ by the visitor's country: prices, stock, delivery times and even the language of the text. If your users are in Germany and the index holds the copy a server in another country saw, the nearest neighbours will be confidently wrong. Collecting the public pages through Residential Proxy IPs in the target country, with country and city targeting, keeps the indexed copy close to what local visitors see.
Teams that would rather not run that collection themselves can ask about our Web Scraping API, a managed page extraction service available through the sales team and described on the data scraping page. Whichever way you collect, respect each site's robots.txt, terms and rate limits, and keep personal data out of the index unless you have a legal basis to process it.
Common mistakes
- Mixing embedding models or versions. Vectors from different models cannot be compared; a model change means re-embedding everything.
- Using the wrong similarity measure. Use the one the model was trained for, and the same one for indexing and querying.
- Trusting default index settings. Measure recall at 10 against an exact search on a sample of real queries, then tune the graph or cluster parameters.
- Testing without filters. A strict metadata filter can shrink the result list to a handful of items; test the filters users really apply.
- No similarity threshold. Nearest neighbours always exist, so without a minimum score an assistant answers from passages that are merely the least irrelevant.
- Forgetting deletes and access rights. Deleted or restricted documents stay searchable until their vectors are removed or filtered.
Decision guide
| Your situation | What to use |
|---|---|
| Under a few hundred thousand vectors | Exact search or a vector extension |
| Data already in PostgreSQL | pgvector with an HNSW index when needed |
| Millions of vectors, high query rate | A dedicated vector database |
| Offline batch jobs or research | An in-memory library such as Faiss |
| Users search codes and names too | Hybrid search, keyword plus vectors |
| Strict filters on most queries | A system that filters during the search |
| Memory is the limit | Half precision or product quantization |
| Answers drawn from public web pages | Collect and refresh pages per market |
Frequently asked questions
What is the difference between a vector database and a vector store?
"Vector store" is a looser term for anything that keeps embeddings and returns the nearest ones, from an in-memory list to a full database. A vector database adds the database side: persistence, updates and deletes, filters, access control, backups and scaling. Many AI frameworks call the interface in front of either a "vector store".
Do I need a vector database for RAG?
Not always. RAG needs a retrieval step, and for a modest collection that can be exact search in memory, a vector extension in your existing database or a keyword index. A dedicated vector database becomes useful when the collection is large, changes often or serves many users with strict filters.
HNSW vs IVF: which index is better?
Neither is better everywhere. HNSW usually gives higher recall at the same speed but uses more memory and builds more slowly; IVF builds faster and uses less memory, and combined with product quantization it fits very large collections into limited RAM. Test both on your own data and filters.
What is the difference between a vector database and a graph database?
A graph database stores explicit relationships, such as "this customer bought this product", and follows them. A vector database stores implicit similarity: items are "related" because their vectors are close. Some systems combine both, using vectors to find a starting point and the graph to follow known connections.
What is an example of a vector database?
pgvector is an open-source example of the extension route: it adds vector search to PostgreSQL. Faiss is an example of the library route, and several open-source and managed systems are dedicated vector databases. Which one fits depends on your data size, your filters and what you already run.
How many dimensions should my vectors have?
The embedding model decides the dimension count, typically a few hundred to a few thousand. More dimensions can hold more detail but cost proportionally more memory, so compare a smaller model against a larger one on your own test queries before paying for the bigger index.
Summary
A vector database stores embeddings with an ID and metadata and finds the items most similar to a query vector. Exact search is fine for small collections; at scale, HNSW graphs, IVF clusters and product quantization trade a little recall for much faster answers, so tune them on your own queries and filters. Start with the database you already run, add hybrid search for exact strings, and budget memory as vectors × dimensions × bytes. When the index is built from public web pages, keeping those pages fresh and country-correct is half the work.




