Number Research
← Back to Explore
Information Systems Software Development Vector Database Architecture

The Bottleneck Was Never the Vector

Why filtering, not nearest-neighbor speed, should decide your pgvector-vs-Qdrant-vs-Pinecone call — and why benchmarks hide it

S.J. Nam 7 min read
The Bottleneck Was Never the Vector

Set hnsw.ef_search to its default of 40, ask pgvector for the ten nearest neighbors, then add a WHERE clause that only 10% of your rows satisfy. You will get back roughly four results — not ten, not an error, just a quietly wrong answer that looks right. Amazon's database team published that exact scenario in 2024 while explaining what pgvector 0.8.0 fixed. It is the most important fact in the entire vector-database debate, and almost no comparison article mentions it, because it points somewhere uncomfortable: the thing that breaks production retrieval isn't nearest-neighbor speed. It's filtering. And filtering is a database problem, not a vector problem.

That reframes the whole "pgvector vs Qdrant vs Pinecone" question. The standard explainer sorts these three onto a tidy spectrum — Postgres extension for hobbyists, Rust engine for control freaks, managed cloud for people with money — and tells you to pick by scale. That advice is mostly wrong, or at least aimed at the wrong axis. The real decision is whether vector search is a feature of the database you already run or a separate system you now have to operate and keep in sync. For most teams shipping retrieval-augmented generation today, the honest answer is the first one, and the benchmarks marshaled to argue otherwise are quietly rigged.

The Benchmark Everyone Quotes Is Theater

Start with the numbers that migration decks lean on. On the standard glove-100-angular dataset under the public ann-benchmarks harness, Qdrant clocks around 292 requests per second while Milvus posts 1,751 — a beating so lopsided it looks like a category error. It is a category error. As Qdrant's own maintainers pointed out in a GitHub discussion, the Milvus entry in that benchmark isn't the Milvus you'd deploy: it's a stripped-down library called Nowhere with no storage layer and no REST overhead — pure in-memory index math with the whole database sawed off. You are comparing a race car to a photograph of a race car.

This is the dirty secret of vector benchmarks. Almost all of them measure the approximate-nearest-neighbor kernel in isolation — the one part every serious engine has already optimized to within a rounding error of the others. The costs that actually dominate a production request live everywhere else: serialization, network hops, the storage engine, concurrent write pressure, and above all filtering against live business data. Qdrant, to its credit, split its 2024 benchmarks into separate latency and throughput regimes precisely because a single "it's fast" number is meaningless. When a vendor waves a 6x speedup at you, ask what they amputated to get it.

Filtering Is the Whole Game

Here's the counterintuitive part. The moment your retrieval query stops being "find me similar vectors" and becomes "find me similar vectors from in-stock products under $50 in the electronics category," raw ANN speed stops mattering and joins start mattering. Combine an HNSW graph with a selective filter and the graph traversal keeps handing back neighbors that fail the filter, so you either over-scan or under-return. pgvector's 0.8.0 release, which shipped on October 30, 2024, exists almost entirely to solve this — its iterative index scans keep walking the graph until enough filtered rows survive. Amazon measured the payoff on Aurora at up to 9x faster query processing and, more tellingly, up to 100x more relevant results on filtered searches. The headline there isn't speed. It's that the old behavior was returning garbage.

Now notice who wins that game structurally. Your filter predicates — price, inventory, tenant ID, permissions, freshness — already live in your relational tables. With pgvector the filtered vector search is one SQL statement against data that is already consistent, already transactional, already yours. With a dedicated engine you must copy those attributes into the vector store, keep them synchronized as they change, and hope your "in stock" flag in Qdrant hasn't drifted from the truth in Postgres. The dedicated-database pitch quietly assumes the hard part is the vector. In production, the hard part is the join.

The Ceiling Is Real, Just Higher Than You Think

None of this is a claim that pgvector is limitless — it plainly isn't. It caps indexed vectors at 2,000 dimensions, offers only HNSW and IVFFlat, and ships no DiskANN and no GPU acceleration, so genuinely enormous or latency-brutal workloads outgrow it. Under about five million vectors, HNSW queries return in single-digit milliseconds, which quietly covers the overwhelming majority of RAG systems, recommendation features, and semantic search that ever get built. The interesting move is what happens above that line. Timescale's pgvectorscale extension, benchmarked on 50 million vectors, reported 28x lower p95 latency and 16x higher throughput than stock pgvector — meaning the Postgres ecosystem is now growing its own answer to scale rather than conceding it. The ceiling everyone warns you about keeps getting a new floor built on top of it.

So when do you actually need to leave? When you're at hundreds of millions to billions of vectors, or you need GPU search, or your query pattern is so vector-dominant that the relational baggage is pure overhead. That is a real population of companies. It is not most of them.

Two Very Different Bets on What a Vector Store Is

Which brings us to why Qdrant and Pinecone exist as separate businesses, and why they're betting in opposite directions. Qdrant is written in Rust from the ground up, and its founders are explicit that they see themselves building durable low-level infrastructure — "Linux kernel, not SaaS wrapper," as they put it announcing a $50 million Series B in 2026, led by AVP with Bosch Ventures, Unusual Ventures, Spark Capital, and 42CAP. That's a wager that vector search becomes a permanent, ownable primitive you self-host and tune — the anti-lock-in play for teams who want the control pgvector can't give them and refuse to rent it.

Pinecone bets the other way: that nobody should operate this at all. Its serverless product sells "no infrastructure or tuning required," and the market has rewarded the pitch — a $100 million Series B at a $750 million valuation, around 4,000 customers, a workforce tripled since 2019. The catch shows up on the invoice. Pinecone's serverless pricing is opaque enough that third parties have built calculators just to guess at it, decomposing usage into read units, write units, and storage in ways that resist a napkin estimate. You are trading operational simplicity for cost predictability, and for a workload whose volume you can't yet forecast, that is not a trade to make casually.

The Question Behind the Question

Strip away the feature tables and the migration is really a claim about your own future: that vector search will become a workload big enough, hot enough, or specialized enough to deserve its own system, its own on-call rotation, and its own copy of your data. For the team indexing a few million documents to answer support tickets, that claim is speculative fiction, and pgvector's boring co-location with the source of truth is the correct, slightly embarrassing answer. For the team genuinely at a billion vectors, the specialized engines earn their keep and the only real fight is control versus convenience — Qdrant's kernel or Pinecone's abstraction.

The mistake isn't picking the wrong one of the three. It's picking any of them before you've asked whether your bottleneck was ever the vector at all.

References