Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

pandas is a Python package that provides fast, flexible, and expressive data structures designed to make working with "relational" or "labeled" data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. It is already well on its way towards this goal. Easy handling of missing data (represented as NaN, NA, or NaT) in floating point as well as non-floating point data Size mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can simply ignore the labels and let Series, DataFrame, etc. automatically align the data for you in computations Powerful, flexible group by functionality to perform split-apply-combine operations on data sets, for both aggregating and transforming data Make it easy to convert ragged, differently-indexed data in other Python and NumPy data structures into DataFrame objects Intelligent label-based slicing, fancy indexing, and subsetting of large data sets Intuitive merging and joining data sets Flexible reshaping and pivoting of data sets Hierarchical labeling of axes (possible to have multiple labels per tick) Robust IO tools for loading data from flat files (CSV and delimited), Excel files, databases, and saving/loading data from the ultrafast HDF5 format Time series-specific functionality: date range generation and frequency conversion, moving window statistics, date shifting and lagging 

Multilingual Sentence & Image Embeddings with BERT

This framework provides an easy method to compute dense vector representations for sentences, paragraphs, and images. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various task. Text is embedding in vector space such that similar text is close and can efficiently be found using cosine similarity. We provide an increasing number of state-of-the-art pretrained models for more than 100 languages, fine-tuned for various use-cases. Further, this framework allows an easy fine-tuning of custom embeddings models, to achieve maximal performance on your specific task. For the full documentation, see www.SBERT.net. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (EMNLP 2019) Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation (EMNLP 2020) Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks (NAACL 2021) The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes (arXiv 2020) TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning (arXiv 2021) BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models (arXiv 2021) 

Fine-tune BERT to generate sentence embedding for cosine similarity

A text analytics library with support for codemixed data

Lingualytics is a Python library for dealing with indic text. Lingualytics is powered by powerful libraries like Pytorch, Transformers, Texthero, NLTK and Scikit-learn. Preprocessing  Remove stopwords Remove punctuations, with an option to add punctuations of your own language Remove words less than a character limit Representation  Find n-grams from given text NLP  Classification using PyTorch  Train a classifier on your data to perform tasks like Sentiment Analysis Evaluate the classifier with metrics like accuracy, f1 score, precision and recall Use the trained tokenizer to tokenize text 