You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: fern/pages/choosing_and_configuring_indexes.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,7 +26,7 @@ The best index depends mostly on dataset size, vector dimensionality, recall tar
26
26
|[IVF-Flat](neighbors/ivfflat.md)| Medium | Fast when probing a subset of partitions; exact within scanned partitions | Partitions the dataset into coarse clusters, allowing the index to scale to larger datasets by searching only relevant partitions. Stores full-precision vectors. |
27
27
| IVF-SQ | Medium | Often faster than IVF-Flat due to lower memory bandwidth | IVF with scalar quantization. Vectors are compressed to reduce memory use and improve throughput with some recall tradeoff. |
28
28
|[IVF-PQ](neighbors/ivfpq.md)| Medium-high | Very memory-efficient; search speed depends on compression, probing, and refinement | IVF with product quantization. Splits vectors into subvectors and stores compact codes, enabling much smaller indexes with a larger accuracy tradeoff than IVF-SQ. |
29
-
| ScaNN | Medium-high | Strong recall and speed tradeoff | Combines partitioning, quantization, and reranking to prune the search space while preserving result quality. |
29
+
|[ScaNN](neighbors/scann.md)| Medium-high | Strong recall and speed tradeoff | Combines partitioning, quantization, and reranking to prune the search space while preserving result quality. |
30
30
|[Vamana/DiskANN](neighbors/vamana.md)| High | Excellent at very large scale, including SSD-backed search | Builds a graph designed for large or disk-backed indexes, especially when the full dataset cannot fit comfortably in memory. |
31
31
32
32
## Quantization
@@ -47,7 +47,7 @@ This is especially useful with quantized indexes. Quantization makes search fast
47
47
48
48
For example, an IVF-SQ or [IVF-PQ](neighbors/ivfpq.md) index may use compressed vectors to scan selected partitions quickly. After it finds the best candidate IDs, a refinement step can load the original vectors for those candidates and compute exact distances. This usually improves recall while keeping most of the speed and memory benefits of quantized search.
49
49
50
-
ScaNN also relies heavily on this idea: it prunes the dataset quickly using partitioning and quantization, then reranks a smaller candidate set to improve final result quality.
50
+
[ScaNN](neighbors/scann.md) also relies heavily on this idea: it prunes the dataset quickly using partitioning and quantization, then reranks a smaller candidate set to improve final result quality.
51
51
52
52
## IVF vs. graph-based indexes
53
53
@@ -67,7 +67,7 @@ For quantized indexes, tune compression together with refinement. Stronger compr
67
67
68
68
For graph-based indexes, tune search breadth before increasing build complexity. If search-time tuning cannot reach the recall target within the latency budget, then increase graph quality, graph degree, or construction breadth.
69
69
70
-
For multi-stage indexes such as ScaNN, tune candidate count and reranking together. The goal is to keep the first stage broad enough to preserve recall and the reranking stage small enough to stay fast.
70
+
For multi-stage indexes such as [ScaNN](neighbors/scann.md), tune candidate count and reranking together. The goal is to keep the first stage broad enough to preserve recall and the reranking stage small enough to stay fast.
71
71
72
72
## When to use each
73
73
@@ -83,6 +83,6 @@ Use HNSW for high-quality CPU search when the index fits in memory.
83
83
84
84
Use [CAGRA](neighbors/cagra.md) for high-throughput GPU search, fast GPU index construction, or hybrid workflows where a GPU-built graph is converted to HNSW for CPU search.
85
85
86
-
Use ScaNN when you want a tuned combination of partitioning, quantization, and reranking.
86
+
Use [ScaNN](neighbors/scann.md) when you want a tuned combination of partitioning, quantization, and reranking.
87
87
88
88
Use [Vamana/DiskANN](neighbors/vamana.md) for very large datasets, especially when SSD-backed search is important, or hybrid workflows where a GPU-built graph is converted to CPU for DiskANN search.
0 commit comments