Transformer Libraries - Page 1

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Python 104111Updated: 10 mo ago License: Permissive (Apache-2.0)

TensorFlow code and pre-trained models for BERT

Python 34473Updated: 10 mo ago License: Permissive (Apache-2.0)

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Python 14382Updated: 10 mo ago License: Permissive (MIT)

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Python 12771Updated: 10 mo ago License: Permissive (MIT)

Multilingual Sentence & Image Embeddings with BERT

Python 10938Updated: 10 mo ago License: Permissive (Apache-2.0)

👑 Easy-to-use and powerful NLP library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis and 🖼 Diffusion AIGC system etc.

Python 9417Updated: 10 mo ago License: Permissive (Apache-2.0)

Mapping a variable-length sentence to a fixed-length vector using BERT model

Python 9373Updated: 3 y ago License: Permissive (MIT)

Pre-Training with Whole Word Masking for Chinese BERT（中文BERT-wwm系列模型）

Python 8439Updated: 10 mo ago License: Permissive (Apache-2.0)

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

Python 7866Updated: 10 mo ago License: Permissive (MIT)

A PyTorch implementation of the Transformer model in "Attention is All You Need".

Python 7565Updated: 10 mo ago License: Permissive (MIT)

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Rust 7111Updated: 10 mo ago License: Permissive (Apache-2.0)

Chinese version of GPT2 training code, using BERT tokenizer.

Python 6977Updated: 10 mo ago License: Permissive (MIT)

Code for the paper "Jukebox: A Generative Model for Music"

Python 6973Updated: 10 mo ago License: Proprietary (Proprietary)

TensorFlow Neural Machine Translation Tutorial

Python 6232Updated: 10 mo ago License: Permissive (Apache-2.0)

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

Python 5633Updated: 10 mo ago License: Permissive (Apache-2.0)

Google AI 2018 BERT pytorch implementation

Python 5545Updated: 10 mo ago License: Permissive (Apache-2.0)

Ongoing research training transformer models at scale

Python 5415Updated: 10 mo ago License: Proprietary (Proprietary)

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

Python 5282Updated: 10 mo ago License: Permissive (Apache-2.0)

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Python 5230Updated: 10 mo ago License: Permissive (MIT)

PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO

Python 4892Updated: 10 mo ago License: Permissive (Apache-2.0)

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

Python 4881Updated: 10 mo ago License: Permissive (MIT)

Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.

Jupyter Notebook 4534Updated: 11 mo ago License: Permissive (MIT)

中文文本分类，TextCNN，TextRNN，FastText，TextRCNN，BiLSTM_Attention，DPCNN，Transformer，基于pytorch，开箱即用。

Python 4459Updated: 10 mo ago License: Permissive (MIT)

Hackable and optimized Transformers building blocks, supporting a composable construction.

Python 4418Updated: 10 mo ago License: Proprietary (Proprietary)

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

Python 4329Updated: 10 mo ago License: Permissive (MIT)

pycorrector is a toolkit for text error correction. 文本纠错，Kenlm，ConvSeq2Seq，BERT，MacBERT，ELECTRA，ERNIE，Transformer，T5等模型实现，开箱即用。

Python 4230Updated: 10 mo ago License: Permissive (Apache-2.0)

A TensorFlow Implementation of the Transformer: Attention Is All You Need

Python 3954Updated: 10 mo ago License: Permissive (Apache-2.0)

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

Python 3823Updated: 10 mo ago License: No License (No License)

Transformers for Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI

Python 3698Updated: 10 mo ago License: Permissive (Apache-2.0)

Official DeiT repository

Python 3417Updated: 11 mo ago License: Permissive (Apache-2.0)

State of the Art Natural Language Processing

Scala 3279Updated: 10 mo ago License: Permissive (Apache-2.0)

Production First and Production Ready End-to-End Speech Recognition Toolkit

C++ 3072Updated: 10 mo ago License: Permissive (Apache-2.0)

viewpager with parallax pages, together with vertical sliding (or click) and activity transition

Java 3035Updated: 1 y ago License: No License (No License)

A simple but complete full-attention transformer with a set of promising experimental features from various papers

Python 3021Updated: 10 mo ago License: Permissive (MIT)

A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Python 2897Updated: 1 y ago License: Permissive (Apache-2.0)

GPT2 for Chinese chitchat/用于中文闲聊的GPT2模型(实现了DialoGPT的MMI思想)

Python 2752Updated: 10 mo ago License: No License (No License)

A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks

Python 2485Updated: 4 y ago License: Permissive (Apache-2.0)

Implementation of BERT that could load official pre-trained models for feature extraction and prediction

Python 2411Updated: 10 mo ago License: Permissive (MIT)

AI 写小说，生成玄幻和言情网文等等。中文预训练生成模型。采用我的 RWKV 模型，类似 GPT-2 。AI写作。RWKV for Chinese novel generation.

Python 2190Updated: 10 mo ago License: Permissive (Apache-2.0)

Transform PHP data to JavaScript.

PHP 2188Updated: 1 y ago License: Permissive (MIT)

Large-scale pretraining for dialogue

Python 2181Updated: 10 mo ago License: Permissive (MIT)

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Python 2179Updated: 11 mo ago License: Permissive (Apache-2.0)

Evolutionary Scale Modeling (esm): Pretrained language models for proteins

Python 2078Updated: 10 mo ago License: Permissive (MIT)

Elegant PyTorch implementation of paper Model-Agnostic Meta-Learning (MAML)

Python 2014Updated: 10 mo ago License: Permissive (MIT)

Reformer, the efficient Transformer, in Pytorch

Python 1911Updated: 10 mo ago License: Permissive (MIT)

Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

Rust 1876Updated: 10 mo ago License: Permissive (Apache-2.0)

Official PyTorch code for "BAM: Bottleneck Attention Module (BMVC2018)" and "CBAM: Convolutional Block Attention Module (ECCV2018)"

Python 1814Updated: 10 mo ago License: Permissive (MIT)

Super easy library for BERT based NLP models

Python 1793Updated: 11 mo ago License: Permissive (Apache-2.0)

Longformer: The Long-Document Transformer

Python 1778Updated: 10 mo ago License: Permissive (Apache-2.0)

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).

Jupyter Notebook 1719Updated: 10 mo ago License: Permissive (BSD-3-Clause)