A vector database stores content such as text, images or audio as arrays of numbers that capture its meaning (vectors, or embeddings) and returns the records closest in meaning to a query very quickly. It searches by semantic similarity rather than keyword matching, and it is usually the retrieval layer behind enterprise AI assistants.
Where the vector database question comes from
The question usually surfaces halfway through an AI project. A team has built a document assistant or a smarter search pilot, early tests look promising, and then, as the document count grows, answers slow down and irrelevant paragraphs start appearing. Someone says, "We need a vector database," and the follow-ups begin: can our existing database do this, do we need a separate product, where will the data live?
The confusion comes from treating a vector database as a solution in its own right. It is an infrastructure component. Users never see it; they only notice whether the assistant found the right document. We cover the full answer-from-your-documents architecture in our guide to RAG for enterprise LLMs. This article zooms in on the retrieval layer: how vectors are stored, indexed and searched, and how to choose the store. What users actually see on the search screen is covered in our article on AI-powered enterprise search.
How a vector database works: embeddings and semantic search
An embedding model turns a sentence or paragraph into a list of hundreds, sometimes thousands, of numbers. Texts with similar meanings land close together in that space. "How many days of annual leave do I get?" and a paragraph headed "Paid holiday entitlement" share almost no words, yet their vectors sit near each other.
The vector database stores those points and finds the nearest neighbours to an incoming query vector. Comparing a query against millions of vectors one by one would be far too slow, so approximate nearest neighbour (ANN) indexes are used. The common approaches are:
- HNSW (hierarchical navigable small world graphs): fast and accurate, but memory-hungry.
- IVF (inverted file, cluster-based): groups vectors into clusters, uses less memory and needs more careful tuning.
- Quantisation: compresses vectors to cut memory and disk use, usually at the cost of some accuracy.
The index is only half the story. Each vector is stored alongside metadata such as document name, version, section, date and permission group, and the query is filtered with rules like "current versions only, and only what this user may see". Language matters too: Turkish, with its agglutinative word forms, is a good example of why a model's multilingual performance must be tested; see our piece on Turkish natural language processing.
What a badly built vector layer costs
A vector store is effectively a semantic copy of your company's documents. That is why the OWASP Top 10 for LLM Applications 2025 lists "Vector and Embedding Weaknesses" (LLM08:2025) as a risk category of its own. A store without permission filtering, one that mixes sources in a single pool, or one that can be poisoned with outside content will lead the assistant to hand the wrong document to the wrong person.
According to IBM's Cost of a Data Breach Report 2025, 13% of organisations reported breaches of AI models or applications, and 97% of those lacked proper AI access controls.
The problem did not stay in 2025. In IBM's 2026 Cost of a Data Breach announcement, more than 20% of organisations reported a breach targeting AI models or applications. If access control is not enforced in the vector layer, the assistant bypasses your file-server permissions, however carefully they were set.
Quality carries a price as well. The Stanford HAI AI Index Report 2026 found hallucination rates across 26 leading models ranging from 22% to 94% on the AA-Omniscience knowledge benchmark. That is why grounding answers in company documents, rather than in the model's own knowledge, matters; but when retrieval surfaces the wrong or an outdated paragraph, the model still writes a convincing but incorrect answer on top of it; our guide to reducing AI hallucinations covers the other safeguards.
Dedicated vector database or an extension to what you already run?
This is the decision most enterprise teams debate. There are three broad options, and none is right in every case:
| Criterion | Vector extension to an existing relational database (e.g. pgvector for PostgreSQL) | Vector features of a search engine | Dedicated (purpose-built) vector database |
|---|---|---|---|
| Suitable scale | Small to medium document collections | Medium to large, especially if text search is already in use | Very large volumes and heavy query loads |
| Hybrid search (keyword + semantic) | Needs extra configuration | Strong, supported natively | Varies by product |
| Permission and metadata filtering | Joins your existing tables and roles | Good | Good; the permission model is designed separately |
| Operational overhead | Low; the team already knows the database | Moderate | A new component to back up, monitor and patch |
| On-premises hosting | Straightforward | Possible | Possible with open-source options; some products are cloud-only |
A practical rule of thumb: if the vector count stays within a few million and the team already runs a relational database, starting with an extension is often enough. A dedicated vector database earns its place when tens of millions of vectors, high concurrency and strict latency targets arrive together. These thresholds are not hard limits; they shift with hardware, index settings and filtering needs, so let pilot measurements settle the decision.
Nor does a vector database replace your reporting stack. A data warehouse answers numerical questions ("sales by region last quarter"); a vector store answers semantic ones ("what do we tell customers during a return?"). We compare raw-data layers in data lake vs data warehouse and cover sensor-specific stores in our time-series database guide.
Seven questions to answer before choosing a vector database
- What will be searched? Procedures, contracts, technical manuals, emails, product catalogues or images. The content type drives the chunking method and the choice of embedding model.
- How large is it, and how fast is it growing? Estimate today's document count and annual growth. The vector count is a multiple of the document count, because documents are split into chunks; a long manual alone can produce dozens.
- Do exact terms matter? Product codes, clause numbers and part numbers can slip through semantic search. If they matter, hybrid search (keyword plus vector) is essential.
- What is the permission model? Each chunk's access group must be stored with the vector and filtered at query time. Our article on file-sharing permissions explains how to structure the folder rights this should mirror.
- Where must the data live? Embeddings of documents containing personal data can themselves be personal data. In Türkiye, the Personal Data Protection Law (KVKK) sets rules on processing and cross-border transfer, so cloud versus on-premises is a compliance decision; see AI and KVKK for the framework.
- How will updates flow? When a document changes, its old chunks must be removed and new ones added automatically. Otherwise the assistant keeps quoting a withdrawn procedure.
- How will you measure success? Build a test set from real user questions and track whether the right chunk appears in the top five results. Rerun it whenever the model or index settings change.
Most of these questions are about how your data is organised before they are about technology. If three versions of the same procedure exist, even the best index may retrieve the wrong one; our piece on duplicate records and data quality covers the clean-up.
Running a vector store securely
The risks grouped under OWASP's LLM08 can be reduced with concrete controls. We recommend this checklist:
- Enforce permissions in the query. Filtering must happen during retrieval, not after results come back; otherwise the model has already seen the restricted chunk.
- Separate sources. Customer data, HR files and general procedures should not share one index; use separate collections where needed.
- Control ingestion. Only approved sources should feed the store; uploads from outside carry a poisoning risk.
- Prove deletion. When a document reaches the end of its retention period or an erasure request arrives, its vectors must go too, and the deletion should be logged.
- Log access. It should be possible to see who asked what and which chunks they reached.
Staff pasting company data into public AI tools is a separate risk, covered in our article on ChatGPT and company data security. We test the configuration and exposure of the vector layer as part of our cyber security consultancy.
How we approach this at Digital Bridge
We do not sell a vector database as a standalone product. We build it as one layer of an enterprise LLM assistant or semantic search project. Our approach:
- Discovery and needs analysis. Together we map which documents will be searched, their volume, the permission structure and where the data must reside. At the end of this stage we put the extension-versus-dedicated decision in writing, with the reasoning.
- Data preparation. We weed out duplicates, settle versions and define the metadata schema, applying the same discipline as our data governance and quality work.
- Pilot. In a single domain, such as quality procedures, we build a test set from real user questions and compare embedding models and index settings. The decision follows the measurements.
- Integration. We connect the vector layer to your ERP, document management system, intranet or help desk through AI integration, and make sure the index refreshes automatically when documents change.
- On-premises option. Where data cannot leave the organisation, we can run the embedding model, vector store and language model entirely on your own servers.
In fields with highly specialised terminology, the embedding or language model may need adapting; our custom AI model guide explains when that is worth it. After the needs analysis we prepare a written proposal setting out scope and phases.
Next step
A vector database decision starts not with a product shortlist but with three facts: the document types to be searched, the approximate volume, and who may see what. Write those on one page and add the 30 to 50 questions your team asks most often. If you are not sure where AI fits yet, start with our guide on where to start with AI in business.
Once your list is ready, get in touch. We will review it with you, work out whether an extension or a dedicated database makes more sense, and scope a pilot. For more use cases, browse our full Artificial Intelligence hub.