Oct 18, 2026 · 16 min · 8 segments
In this episode of Scaling Postgres, we review two companies battling one another for text search dominance in Postgres, faster vector search with pg\_vectorsearch, pushing Citus scaling with query…
Creston JamisonHost
And from what they were looking at, they got about twice as fast depending upon the type of query, whether it was a mixed conjunction, disjunction, or phrase, although disjunction was not quite as fast.

Now, how did they improve this count performance? The first optimization they did is segment partitioning, because what they realized when doing profiling is that query visibility checking accounted for 80% of the page accesses.

so basically how can we reduce these visibility checks and with their lsm tree implementation data gets partitioned into immutable segments that are merged together at a particular size threshold and that this merging is relatively naive it's oblivious to the data contained within a segment So each segment is basically random data.

But with their new implementation, they're actually splitting up segments based upon the values of the columns.

So there's some correlation between them, and they are partitioning merging strategy based upon the spooky LSM compaction paper.

And they said they can partition by CTID, which gives each segment a range of Postgres page numbers.

And this enhancement brought, quote, visibility-related page accesses down from 1,382 to just 30.

This allows them to do more efficient visibility checks when they need to do that.

So basically, it's a compressed column that maps Postgres page numbers to the first doc ID found on that page.

And by doing this, it reduced buffer accesses from over 7,500 to just under 400, which is a 95% reduction.

So for the indexes posting list, it uses BP128 encoding, and that's fine for most terms, but very common terms had super dense posting lists.

and that intersecting and unioning large B128 arrays incurred significant overhead.

And they were able to get efficiencies by representing them as bitmaps instead.


So in this example, looking at a count of Hacker News items for the term database with a score greater than 10, when doing a filtered count like this, paradeDB was 10 times faster than 10.

And then doing a filtered topK by BM25 relevance, where you're adding in the ordering, paradeDB was 11 times faster than 10. then looking at the top case search again ordered by time was 21.7 times faster than 10. now this also depends on what version is being used because you'll see in the next blog post that planet scale has actually upped their game as well but these are the results with the versions that they're using right now And they're making the argument that when you're combining filtering and ordering, that in these use cases, paradeDB might be superior because everything's in one index, whereas the 10 solution relies on btree indexes for filtering and perhaps sorting, and then 10 just takes care of the text queries.

And they show an example of that here where you're just creating one index for per ADB, whereas you have to apply your 10 index and then two other indexes to be able to order by time or do scoring.

And from what they were looking at, they got about twice as fast depending upon the type of query, whether it was a mixed conjunction, disjunction, or phrase, although disjunction was not quite as fast.

Now, how did they improve this count performance? The first optimization they did is segment partitioning, because what they realized when doing profiling is that query visibility checking accounted for 80% of the page accesses.

so basically how can we reduce these visibility checks and with their lsm tree implementation data gets partitioned into immutable segments that are merged together at a particular size threshold and that this merging is relatively naive it's oblivious to the data contained within a segment So each segment is basically random data.

But with their new implementation, they're actually splitting up segments based upon the values of the columns.

So there's some correlation between them, and they are partitioning merging strategy based upon the spooky LSM compaction paper.

And they said they can partition by CTID, which gives each segment a range of Postgres page numbers.

And this enhancement brought, quote, visibility-related page accesses down from 1,382 to just 30.

This allows them to do more efficient visibility checks when they need to do that.

So basically, it's a compressed column that maps Postgres page numbers to the first doc ID found on that page.

And by doing this, it reduced buffer accesses from over 7,500 to just under 400, which is a 95% reduction.

So for the indexes posting list, it uses BP128 encoding, and that's fine for most terms, but very common terms had super dense posting lists.

and that intersecting and unioning large B128 arrays incurred significant overhead.

And they were able to get efficiencies by representing them as bitmaps instead.


So in this example, looking at a count of Hacker News items for the term database with a score greater than 10, when doing a filtered count like this, paradeDB was 10 times faster than 10.

And then doing a filtered topK by BM25 relevance, where you're adding in the ordering, paradeDB was 11 times faster than 10. then looking at the top case search again ordered by time was 21.7 times faster than 10. now this also depends on what version is being used because you'll see in the next blog post that planet scale has actually upped their game as well but these are the results with the versions that they're using right now And they're making the argument that when you're combining filtering and ordering, that in these use cases, paradeDB might be superior because everything's in one index, whereas the 10 solution relies on btree indexes for filtering and perhaps sorting, and then 10 just takes care of the text queries.

And they show an example of that here where you're just creating one index for per ADB, whereas you have to apply your 10 index and then two other indexes to be able to order by time or do scoring.
The rest of this transcript — segmented and speaker-labeled, so you land on the exact moment something was said
Search every transcript — by keyword, by phrase, or by meaning, across every show Radar indexes
Trends — what is surging across podcasts, measured against its own baseline
Alerts — when a name you follow appears in a newly indexed episode
No account is needed to search Radar.