FAISS: Facebook Just Released A New Open Source Library For Clustering Big Data

Share on twitter
Share on whatsapp
Share on facebook
facebook open source

Short Bytes: FAISS is an open-source library released by Facebook for similarity search and clustering high-dimensional data. This library finds application in complex datasets like images and videos which could not fit in RAM all at once.

With the advent of highly successful Machine Learning methods, there has been a boom in big datasets across varied domains. With these huge datasets, hardware becomes a bottleneck. Processing these datasets requires high memory bandwidth and processor capabilities. Furthermore, indexing the data points, clustering and search become highly demanding.

Researchers at Facebook AI Research or FAIR recently published a research paper describing an efficient design for clustering and similarity search. Their new algorithmic structure performs much faster than the previous state-of-art algorithms and utilises GPU for higher memory bandwidth and computational throughput.

Recommended: Top 15 Facebook Open Source Projects You Must Know

Based on their research, they have created a library called FAISS and open-sourced it. Although the algorithms for clustering and similarity search are well-known, this library optimizes those algorithms to perform efficiently on GPUs. Some the algorithms implemented in the library include –

  • Fast K-Nearest Neighbour
  • QuickSelect
  • Warpselect
  • K-Means clustering

As a test of how the library performs, in the following figure, the first and the last image are given and the algorithm computes the intermediate transitional images from a collection of 95 million images.

FAISS Similarity Search

Top Features of FAISS Library –

  • Written in C++ with complete Python wrappers
  • Supports single/multiple GPUs
  • Highly Scalable, typically supports up to 100 dimensions
  • Built on BLAS and CUDA libraries
  • 8.5x faster performance than current state-of-art libraries

Here is the GitHub repo of the FAISS library. So what do you think about the new library? Share your thoughts with us in comments.

Anand Krish

Anand Krish

Research Student working on Robotics and Deep Learning

New on Fossbytes

Scroll to Top