Dedicated Vector DB vs pgvector for Custom AI Apps

The Marketing Pitch and the Empirical Counterweight

The conventional wisdom circulating through vendor decks, conference talks, and integration tutorials insists that any production-grade retrieval-augmented system — a custom AI app serving document search, conversational retrieval, or semantic product discovery — needs a dedicated vector database. Pinecone, Qdrant, and Milvus are the names that usually anchor that narrative.

The empirical record tells a more measured story. For datasets under roughly 10 million vectors, pgvector with HNSW indexing now delivers p50 query latencies around 5 milliseconds, matching or outperforming dedicated managed stores like Pinecone Serverless in standardized 1M vector benchmarks. The pivot to a dedicated engine is rarely an architectural inevitability; for most growing applications, it is a probabilistic forecast about a workload that has not yet arrived — a forecast made on the vendor's behalf, not yours.

The 10 Million Vector Threshold: When PostgreSQL Hits Its Limit

The number that quietly governs this entire decision sits at the boundary of operational sanity: roughly 10 million vectors. Below that line, modern PostgreSQL configurations with the pgvector extension handle production traffic with latencies that are, for most user-facing inference workloads, indistinguishable from specialized systems. Above it, the conversation changes — not because pgvector collapses, but because the resource profile of the database starts to dominate the application's cost structure in ways that demand different trade-offs.

This is not a hard cliff. It is a probabilistic inflection point shaped by vector dimensions, query patterns, and how aggressively the index fits into available RAM. A 10 million vector collection at 384 dimensions behaves very differently from the same count at 1536 dimensions; the latter consumes four times the memory per vector and reshapes the entire tuning conversation. The 10 million figure, however, is the threshold that recurs across recent benchmark reports and production migrations as the point at which teams begin evaluating dedicated infrastructure in earnest.

It is also the point at which the "one database for everything" argument for PostgreSQL begins to fray at the edges. Once an AI app's vector index demands dedicated memory partitions, dedicated backup policies, and dedicated scaling strategies, the operational benefits of co-location start to give way to the operational benefits of separation. The question stops being "can PostgreSQL do this?" and becomes "should it?"

Dedicated vector stores are not a precondition for shipping an AI app. They are an answer to a question most teams have not yet been asked.

Performance Parity: How pgvector HNSW Matches Dedicated Engines

The single most important development in this space over the last two years is the maturation of HNSW — Hierarchical Navigable Small World — indexing inside pgvector. HNSW support landed in pgvector 0.5.0 in 2023, and subsequent versions have steadily closed the performance gap with purpose-built engines. HNSW is a graph-based index that organizes vectors into layers of navigable proximity, allowing nearest-neighbor queries to traverse the structure logarithmically rather than scanning the full distance matrix — a probabilistic approach that trades a small amount of recall fidelity for substantial speed.

In controlled benchmarks at the 1 million vector mark using 1536-dimensional embeddings — the canonical size for OpenAI's text-embedding-3-small and similar models — pgvector with HNSW delivers p50 query latencies around 5ms. Qdrant, a dedicated vector store widely respected for retrieval speed, lands at roughly 4ms in the same workload. That is a margin narrower than the typical network jitter between a custom AI app's edge and its database tier, and well within the noise floor of real production environments.

The more interesting comparison is against Pinecone Serverless, the managed offering that has become a default choice for teams avoiding infrastructure work. In the same 1M vector benchmarks, pgvector matches or outperforms Pinecone Serverless on p50 latency while keeping the entire dataset inside the same transactional system that holds the application's relational state. The result is not a victory for PostgreSQL on raw speed; it is evidence that the conventional gap between the two architectures is smaller than the marketing suggests.

A second performance lever arrived with pgvector 0.8.0: Iterative Scans. Pre-filtering vector queries — where a relational constraint narrows the candidate set before similarity search — had been a known weakness of pgvector, because the post-filter approach degraded recall and latency simultaneously. Iterative Scans resolve this by running the HNSW traversal in segments that respect the filter, restoring the kind of hybrid retrieval patterns that real AI apps depend on. For custom applications that combine semantic search with metadata constraints — customer tenant, document category, time window — this is not a minor optimization. It is the difference between a feature that ships and a feature that quietly loses precision under load.

Storage Efficiency and Quantization: Reducing Footprint by 32x

Latency is the metric engineers benchmark, but storage is the metric finance teams remember. The case for keeping vectors inside PostgreSQL strengthens considerably when the index footprint shrinks.

pgvector's binary vector type (bit) and half-precision floating point type (halfvec) enable aggressive quantization at the index layer. In practice, real-world document search workloads see index storage reductions of up to 32x when vectors are stored in binary form versus their full precision equivalents. The recall loss in these configurations is, for typical RAG patterns, near zero — often indistinguishable from full-precision retrieval in head-to-head evaluations against held-out query sets.

The economic consequence is not subtle. A custom AI app with 5 million document vectors at 1536 dimensions will, at full precision, occupy a sizeable fraction of a PostgreSQL instance's storage budget and push the working set uncomfortably close to RAM limits. Compress that by 32x with binary quantization and the same workload fits comfortably on infrastructure that is already paying for itself. The marginal cost of adding semantic search becomes a function of compute, not storage, which is exactly the inversion most teams want when scaling inference workloads.

Dedicated vector databases have their own quantization strategies — Qdrant's scalar and product quantization, Pinecone's pod-based isolation tiers — but they do not have the same compositional advantage. They are not co-located with the relational data, which means the cost reduction is paid twice: once in the vector store, once in the synchronization layer that keeps it coherent with the primary database. That synchronization layer is not free, and at scale it becomes a meaningful line item.

Operational Complexity vs. Throughput: The Case for Dedicated Engines

The honest version of this comparison requires acknowledging where dedicated vector stores genuinely outperform.

Milvus, designed from the outset for large-scale similarity search, supports bulk throughput on the order of 100,000 queries per second against 100 million vector datasets. That is not a figure pgvector can match at the same dataset size, and it is the figure that matters for AI apps serving very large user bases with concurrent inference — multi-tenant SaaS platforms, semantic recommendation engines operating at e-commerce scale, or retrieval layers behind autonomous agent systems where hundreds of requests fan out per user action.

Qdrant offers native multi-tenant filtering capabilities that are still awkward to reproduce in PostgreSQL. For applications where tenant isolation is enforced at the vector layer — and the filter selectivity is extreme, narrowing candidate sets to small slices of the index — this matters more than raw p50 latency. The dedicated engines also tend to expose richer observability primitives: per-segment statistics, recall diagnostics, and sharding controls that map cleanly onto infrastructure-as-code workflows.

The trade-off is operational complexity. Two databases mean two backup strategies, two patching cadences, two sets of access controls, and a synchronization layer that becomes its own availability risk. Change data capture pipelines, dual-write patterns, and eventual consistency models all introduce edge cases that have to be reasoned about separately from the retrieval logic itself. Recent production migrations reflect this calculus from both directions. Notion publicly reported an approximately 60% reduction in search engine cost when moving parts of its workload to Turbopuffer, while GlassDollar described a 40% reduction in infrastructure costs after migrating from Elasticsearch to a PostgreSQL vector workflow. Both teams reached the right answer for their specific scale and edge cases; the answers were not the same answer.

The architectural choice is rarely between a fast database and a slow database. It is between a coherent system and a partitioned one.

Architectural Decision Matrix for Custom AI Workflows

Choosing between pgvector and a dedicated vector store comes down to the workload profile, not the brand name. The matrix below summarizes the decision space as it stands today, with the caveat that the thresholds are descriptive rather than universal.

The matrix points in one direction for prototypes and small-scale production systems, and another for applications that have crossed into sustained high-throughput territory. The inflection sits somewhere between 5 million and 10 million vectors, modulated by dimension count and concurrent query volume.

The Real Decision: When to Stop Asking

Most of the comparisons circulating in 2026 treat this as a permanent architectural commitment. It is not. The honest pattern across production migrations is that teams start with pgvector, instrument the workload honestly, and revisit the question when latency budgets tighten or cost curves bend in ways the index cannot absorb.

That instrumentation matters more than the choice itself. A custom AI app shipping without query logs, without p95 latency tracking on its retrieval path, and without recall sampling against a held-out evaluation set is making this decision blind. Neither pgvector nor Qdrant can save an application that cannot measure its own accuracy, and the difference between a probabilistic retrieval system and a broken one usually shows up first in recall metrics that nobody is collecting.

The question worth holding open is not "which engine is faster" but "what does our workload look like at 10 million vectors, and do we have the observability to notice when we get there." Until that question has an empirical answer, the most rigorous move a researcher can make is to keep the architecture coherent, run the benchmarks locally against the actual embedding model in use, and let the data — not the pitch deck — decide when the threshold has been crossed.