Tony Kim
Oct 08, 2024 06:23
Microsoft Bing Visible Search achieves a 5.13x speedup utilizing NVIDIA’s TensorRT, CV-CUDA, and nvImageCodec, enhancing effectivity and decreasing prices.
Microsoft Bing Visible Search, a instrument enabling customers worldwide to go looking utilizing pictures, has been considerably optimized via a collaboration with NVIDIA, leading to a exceptional efficiency increase. Based on NVIDIA Technical Weblog, the combination of NVIDIA’s TensorRT, CV-CUDA, and nvImageCodec into Bing’s TuringMM visible embedding mannequin has led to a 5.13x improve in throughput for offline indexing pipelines, decreasing each power consumption and prices.
Multimodal AI and Visible Search
Multimodal AI applied sciences, like Microsoft’s TuringMM, are important for functions that require seamless interplay between completely different information varieties reminiscent of textual content and pictures. A preferred mannequin for joint image-text understanding is CLIP, which makes use of a twin encoder structure to course of a whole lot of tens of millions of image-caption pairs. These superior fashions are vital for duties reminiscent of text-based visible search, zero-shot picture classification, and picture captioning.
Optimization Efforts
The optimization of Bing’s visible embedding pipeline was achieved by leveraging NVIDIA’s GPU acceleration applied sciences. The trouble centered on enhancing the efficiency of the TuringMM pipeline through the use of NVIDIA’s TensorRT for mannequin execution, which improved the effectivity of computationally costly layers in transformer architectures. Moreover, the usage of nvImageCodec and CV-CUDA accelerated the picture decoding and preprocessing levels, resulting in a big discount in latency for picture processing duties.
Implementation and Outcomes
Previous to optimization, Bing’s visible embedding mannequin operated on a GPU server cluster that dealt with inference duties for numerous deep studying companies throughout Microsoft. The unique implementation, utilizing ONNXRuntime with CUDA Execution Supplier, confronted bottlenecks on account of picture decoding processes dealt with by OpenCV. By integrating NVIDIA’s libraries, the pipeline’s throughput elevated from 88 queries per second (QPS) to 452 QPS, showcasing a 5.14x speedup.
These enhancements not solely improved processing velocity but in addition lowered the computational load on CPUs by offloading duties to GPUs, thus maximizing energy effectivity. The NVIDIA TensorRT contributed most to the efficiency good points, whereas the nvImageCodec and CV-CUDA libraries added an extra 27% enchancment.
Conclusion
The profitable optimization of Microsoft Bing Visible Search highlights the potential of NVIDIA’s accelerated libraries in enhancing AI-driven functions. The collaboration demonstrates how GPU assets could be successfully utilized to speed up deep studying and picture processing workloads, even when baseline methods already make use of GPU acceleration. These developments pave the best way for extra environment friendly and responsive visible search capabilities, benefiting each customers and repair suppliers.
For extra detailed insights into the optimization course of, go to the unique NVIDIA Technical Weblog.
Picture supply: Shutterstock
