RIP, vector database
- Architectural Transition in turbopuffer v3:
- turbopuffer is overhauling its core storage layout by demoting Approximate Nearest Neighbor (ANN) search from the primary index to "just another" secondary index.
- This change aims to accelerate full-text, regex, and vector search, while providing the underlying architecture to efficiently support broader SQL operations (such as
GROUP BYand aggregations).
- Evolution from v1 to v2:
- v1 (Vector-Only): Focused strictly on low-cost, fast vector search backed by object storage with NVMe/RAM caching. It used hierarchical clustering indexes (SPANN, later SPFresh) keyed by an "ANN address" (ClusterId + LocalId) as the primary index.
- v2 (Attribute & Full-Text Search): Introduced inverted indexes for attribute filtering and BM25 full-text search, along with regex, aggregations, and sparse vectors—all built around the legacy vector-primary layout.
- Limitations of a Vector-Primary Architecture:
- Storage Amplification: Because document data is stored directly under the vector's ANN address, multi-vector representations (e.g., late-interaction models, nested docs) duplicate non-vector data across each vector.
- Write Amplification: Cluster rebalancing in SPFresh cascades heavily; moving an ANN address requires relocating all document contents, inverted indexes, and full-text metadata associated with it.
- Limited Vectorization: Query execution engines perform best with specific block sizes (e.g., DuckDB at ~2,048 rows, ClickHouse at ~65k). A vector-primary index constrains non-vector scans and aggregations to ANN cluster sizes (100–200 documents), preventing optimal SIMD and CPU pipeline saturation.
- The v3 Solution:
- By dropping the ANN address as the primary key and separating document storage from vector clustering, v3 eliminates these amplification bottlenecks and allows independent optimization for vector, full-text, and relational query plans.
Hacker News Discussion
- Parallels with Relational Database Storage Engines (Postgres vs. MySQL):
- Commenters highlighted that turbopuffer's shift closely resembles the architectural contrast between PostgreSQL and MySQL (InnoDB).
- PostgreSQL ties secondary indexes directly to physical tuple addresses (ctid/row-id), leading to write amplification on updates. In contrast, MySQL points secondary indexes to a stable primary key, introducing an indirection layer during lookups but substantially lowering re-indexing costs during updates.
- Users drew comparisons to Uber’s classic migration from Postgres to MySQL, which was motivated by similar index-write amplification issues.
- Trade-offs of Indirection and Primary Clustered Indexes:
- Participants debated the lookup overhead of stable-ID indirection versus direct pointers. Several noted that because primary key B-trees are typically kept in memory or cache, the extra traversal penalty is negligible compared to reading pages off disk.
- Commenters discussed how clustered primary key indexes (like InnoDB) enable cleaner MVCC and undo logging strategies compared to heap-based storage with vacuuming.
- Discussion on AI and Comment Quality:
- A tangential discussion emerged regarding automated or LLM-generated commentary on tech forums, debating the effectiveness of gated registrations (like Lobste.rs) versus open registration platforms.