Question
Couchbase Server 7.2+ Vector Index Creation Fails Due to Dimension Mismatch
Ira Nova
0 reputation · 30 Jan 2024, 11:43 UTC
143.4K views1
Loading…
0 reputation · 30 Jan 2024, 11:43 UTC
Couchbase Server 7.2+ introduces vector search indexes that must be defined with exact dimension specifications matching the query vectors.
Log entries report 'vector dimension mismatch' when the vector dimension of the indexed documents differs from the dimension declared in the CREATE INDEX ... WITH {"type":"vector"} clause, and memory pressure can trigger indexer termination.
What configuration steps guarantee that the declared vector dimension aligns with the actual document vectors? How can the indexer.memoryQuota setting be validated to prevent memory‑related indexer kills?
35775 reputation · 30 Jan 2024, 19:26 UTC
Couchbase Server 7.2+ vector indexes require the dimension specified in the CREATE INDEX ... WITH {"type":"vector"} clause to be an exact match for the dimension of every vector within the documents being indexed. Inconsistent document vector dimensions are the direct cause of the reported mismatch error.
couchbaseCREATE INDEX statement precisely reflects this confirmed, consistent dimension.indexer.memoryQuota configuration against the anticipated memory footprint of the index to mitigate termination due to memory pressure.The primary issue is data consistency. Memory pressure is a secondary concern. To determine if the memory quota needs adjustment, please provide the current value of indexer.memoryQuota.
Use comments to ask for clarification. Post a solution as an answer.
6,690 reputation · 30 Jan 2024, 22:25 UTC
The most reliable way to prevent dimension mismatch is to validate vector dimensions across your dataset before index creation. Run this CBQ query against a representative sample:
SELECT META().id, ARRAY_DIM(v.embedding) AS dim FROM bucket WHERE v.embedding IS NOT NULL LIMIT 100
This confirms all vectors share the same dimensionality. Mixed dimensions will cause indexing to fail for all documents, not just the mismatched ones.
The indexer.memoryQuota must accommodate both vector data and index metadata. For a 10GB vector index, allocate at least 12-15GB to account for overhead. Monitor actual usage via the Web Console's Data Service → Indexer section after creation.
If you encounter memory-related termination, use ALTER INDEX to adjust dynamically:
ALTER INDEX idx_name SET {"memoryQuota": 8388608}Note that dimension changes require dropping and recreating the index—there is no online ALTER support for this in 7.2+.