A library for efficient similarity search and clustering of dense vectors.

Faiss contains several methods for similarity search. It assumes that the instances are represented as vectors and are identified by an integer, and that the vectors can be compared with L2 (Euclidean) distances or dot products. Vectors that are similar to a query vector are those that have the lowest L2 distance or the highest dot product with the query vector. It also supports cosine similarity, since this is a dot product on normalized vectors. Most of the methods, like those based on binary vectors and compact quantization codes, solely use a compressed representation of the vectors and do not require to keep the original vectors. This generally comes at the cost of a less precise search but these methods can scale to billions of vectors in main memory on a single server. The GPU implementation can accept input from either CPU or GPU memory. On a server with GPUs, the GPU indexes can be used a drop-in replacement for the CPU indexes (e.g., replace IndexFlatL2 with GpuIndexFlatL2) and copies to/from GPU memory are handled automatically. Results will be faster however if both input and output remain resident on the GPU. Both single and multi-GPU usage is supported. 

💫 Industrial-strength Natural Language Processing (NLP) in Python

spaCy is a library for advanced Natural Language Processing in Python and Cython. It's built on the very latest research, and was designed from day one to be used in real products. spaCy comes with pretrained pipelines and currently supports tokenization and training for 60+ languages. It features state-of-the-art speed and neural network models for tagging, parsing, named entity recognition, text classification and more, multi-task learning with pretrained transformers like BERT, as well as a production-ready training system and easy model packaging, deployment and workflow management. spaCy is commercial open-source software, released under the MIT license. Support for 60+ languages Trained pipelines for different languages and tasks Multi-task learning with pretrained transformers like BERT Support for pretrained word vectors and embeddings State-of-the-art speed Production-ready training system Linguistically-motivated tokenization Components for named entity recognition, part-of-speech-tagging, dependency parsing, sentence segmentation, text classification, lemmatization, morphological analysis, entity linking and more Easily extensible with custom components and attributes Support for custom models in PyTorch, TensorFlow and other frameworks Built in visualizers for syntax and NER Easy model packaging, deployment and workflow management Robust, rigorously evaluated accuracy 

Fine-tune BERT to generate sentence embedding for cosine similarity

The Python micro framework for building web applications.

Students Studio

NLP

scikit-learnby scikit-learn

scikit-learnby scikit-learn

faissby facebookresearch

faissby facebookresearch

spaCyby explosion

spaCyby explosion

bert-cosine-simby beekbin

bert-cosine-simby beekbin

Webservices

flaskby pallets

flaskby pallets