Optimizing IVF-PQ Performance with RAPIDS cuVS: Key Tuning Techniques

Within the first a part of the collection, an outline of the IVF-PQ algorithm was offered, explaining its basis on the IVF-Flat algorithm and the usage of Product Quantization (PQ) to compress the index and help bigger datasets. Partly two, the main focus shifts to the sensible features of tuning IVF-PQ efficiency, which is essential for reaching optimum outcomes, particularly when coping with billion-scale datasets.

Tuning Parameters for Index Constructing

IVF-PQ shares some parameters with IVF-Flat, akin to coarse-level indexing and search hyper-parameters. Nonetheless, IVF-PQ introduces further parameters that management compression. One of many crucial parameters is n_lists, which determines the variety of partitions (inverted lists) into which the enter dataset is clustered. The efficiency is influenced by the variety of lists probed and their sizes. Experiments counsel that n_lists within the vary of 10K to 50K yield good efficiency throughout recall ranges, although this will fluctuate relying on the dataset.

One other essential parameter is pq_dim, which controls compression. Beginning with one fourth the variety of options within the dataset and rising in steps is an effective approach for tuning this parameter. Determine 2 within the unique weblog put up illustrates important drops in QPS, which will be attributed to elements akin to elevated compute work and shared reminiscence necessities per CUDA block.

The pq_bits parameter, starting from 4 to eight, controls the variety of bits utilized in every particular person PQ code, affecting the codebook dimension and recall. Decreasing pq_bits can enhance search velocity by becoming the look-up desk (LUT) in shared reminiscence, though this comes at the price of recall.

Further Parameters

The codebook_kind parameter determines how the codebooks for the second-level quantizer are constructed, both for every subspace or for every cluster. The selection between these choices can influence coaching time, GPU shared reminiscence utilization, and recall. Parameters akin to kmeans_n_iters and kmeans_trainset_fraction are additionally vital, although they not often want adjustment.

Tuning Parameters for Search

The n_probes parameter, mentioned within the earlier weblog put up on IVF-Flat, is crucial for search accuracy and throughput. IVF-PQ offers further parameters like internal_distance_dtype and lut_dtype, which management the illustration of distance or similarity throughout the search and the datatype used to retailer the LUT, respectively. Adjusting these parameters can considerably influence efficiency, particularly for datasets with giant dimensionality.

Bettering Recall with Refinement

When tuning parameters just isn’t sufficient to attain the specified recall, refinement presents a promising different. This separate operation, carried out after the ANN search, recomputes precise distances for chosen candidates and reranks them. The refinement operation can considerably enhance recall, as demonstrated in Determine 4 of the unique weblog put up, although it requires entry to the supply dataset.

Abstract

The collection on accelerating vector search with inverted-file indexes covers two cuVS algorithms: IVF-Flat and IVF-PQ. IVF-PQ extends IVF-Flat with PQ compression, enabling sooner searches and the power to deal with billion-scale datasets with restricted GPU reminiscence. By fine-tuning parameters for index constructing and search, information practitioners can obtain the most effective outcomes effectively. The RAPIDS cuVS library presents a variety of vector search algorithms to cater to varied use instances, from precise searches to low-accuracy-high-QPS ANN strategies.

For sensible tuning of IVF-PQ parameters, confer with the IVF-PQ pocket book on GitHub. For extra particulars on the offered APIs, see the cuVS documentation.

Picture supply: Shutterstock