BERTweet | trained language model for English Tweets | Natural Language Processing library

BERTweet is a Python library typically used in Artificial Intelligence, Natural Language Processing, Pytorch, Bert, Neural Network, Transformer applications. BERTweet has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. However BERTweet build file is not available. You can download it from GitHub.

The general architecture and experimental results of BERTweet can be found in our paper:. Please CITE our paper when BERTweet is used to help produce published results or is incorporated into other software. 845M English Tweets (cased). 23M COVID-19 English Tweets (cased). 23M COVID-19 English Tweets (uncased). As of 09/2020, we have collected a corpus of about 23M "cased" COVID-19 English Tweets, and also generate an "uncased" version of this corpus. Then we continue pre-training from vinai/bertweet-base on each of the "cased" and "uncased" corpora of 23M Tweets for 40 additional epochs, resulting in two BERTweet variants vinai/bertweet-covid19-base-cased and vinai/bertweet-covid19-base-uncased, respectively. Before applying fastBPE to the pre-training corpus of 850M English Tweets, we tokenized these Tweets using TweetTokenizer from the NLTK toolkit and used the emoji package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens @USER and HTTPURL, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets. BERTweet provides this pre-processing step by enabling the normalization argument.