MPNet | MPNet : Masked and Permuted Pre-training for Language | Natural Language Processing library

by microsoft Python Version: Current License: MIT

X-Ray Key Features Code Snippets Community Discussions(3)Vulnerabilities Install Support

kandi X-RAY | MPNet Summary

MPNet is a Python library typically used in Artificial Intelligence, Natural Language Processing, Deep Learning, Pytorch, Tensorflow, Bert, Transformer applications. MPNet has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. However MPNet build file is not available. You can download it from GitHub.

MPNet: Masked and Permuted Pre-training for Language Understanding, by Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu, is a novel pre-training method for language understanding tasks. It solves the problems of MLM (masked language modeling) in BERT and PLM (permuted language modeling) in XLNet and achieves better accuracy. News: We have updated the pre-trained models now.

Support

Quality

Security

License

Reuse

Support

MPNet has a low active ecosystem.

It has 164 star(s) with 18 fork(s). There are 10 watchers for this library.

It had no major release in the last 6 months.

There are 3 open issues and 9 have been closed. On average issues are closed in 19 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of MPNet is current.

Quality

MPNet has 0 bugs and 0 code smells.

Security

MPNet has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

MPNet code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

MPNet is licensed under the MIT License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

MPNet releases are not available. You will need to build from source code and install.

MPNet has no build file. You will be need to create the build yourself to build the component from source.

Installation instructions, examples and code snippets are available.

MPNet saves you 14432 person hours of effort in developing the same functionality from scratch.

It has 28880 lines of code, 2213 functions and 281 files.

It has high code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed MPNet and discovered the below as its top functions. This is intended to give you an instant insight into MPNet implemented functionality, and help decide if they suit your requirements.

Generate the encoder for the given model .
Generate and re - process nbest results .
Generate gradient .
Score target hypo .
Run lm scoring .
An iterator that parses a JSONL file .
Register gradient hooks .
Calculate a single step .
Collate the given samples into a dict .
Generate a forward pass .

Get all kandi verified functions for this library.

MPNet Key Features

No Key Features are available at this moment for MPNet.

MPNet Examples and Code Snippets

No Code Snippets are available at this moment for MPNet.

Community Discussions

Trending Discussions on MPNet

How to use metadata for document retrieval using Sentence Transformers?

Using sentence transformers with limited access to internet

ValueError: Unrecognized model in ./MRPC/. Should have a `model_type` key in its config.json, or contain one of the following strings in its name

QUESTION

How to use metadata for document retrieval using Sentence Transformers?

Asked 2022-Mar-26 at 10:57

I'm trying to use Sentence Transformers and Haystack for document retrieval, focusing on searching documents on other metadata beside document text.

I'm using a dataset of academic publication titles, and I've appended a fake publication year (which I want to use as a search term). From reading around I've combined the columns and just added a separator between the title and publication year, and included the column titles since I thought maybe this could add context. An example input looks like:

title Sparsity-certifying Graph Decompositions [SEP] published year 1980

I have a document store and method of retrieving here, based on this:

...

ANSWER

Answered 2022-Mar-26 at 10:57

It sounds like you need metadata filtering rather than placing the year within the query itself. The FaissDocumentStore doesn't support filtering, I'd recommend switching to the PineconeDocumentStore which Haystack introduced in the v1.3 release a few days ago. It supports the strongest filter functionality in the current set of document stores.

You will need to make sure you have the latest version of Haystack installed, and it needs an additional pinecone-client library too:

Source https://stackoverflow.com/questions/71617889

QUESTION

Using sentence transformers with limited access to internet

Asked 2022-Jan-19 at 13:27

I have access to the latest packages but I cannot access internet from my python enviroment.

Package versions that I have are as below

...

ANSWER

Answered 2022-Jan-19 at 13:27

Based on the things you mentioned, I checked the source code of sentence-transformers on Google Colab. After running the model and getting the files, I check the directory and I saw the pytorch_model.bin there.

And according to sentence-transformers code: Link

the flax_model.msgpack , rust_model.ot, tf_model.h5 are getting ignored when the it is trying to download.

and these are the files that it downloads :

Source https://stackoverflow.com/questions/70716702

QUESTION

ValueError: Unrecognized model in ./MRPC/. Should have a `model_type` key in its config.json, or contain one of the following strings in its name

Asked 2022-Jan-13 at 14:10

Goal: Amend this Notebook to work with Albert and Distilbert models

Kernel: conda_pytorch_p36. I did Restart & Run All, and refreshed file view in working directory.

Error occurs in Section 1.2, only for these 2 new models.

For filenames etc., I've created a variable used everywhere:

...

ANSWER

Answered 2022-Jan-13 at 14:10

Explanation:

When instantiating AutoModel, you must specify a model_type parameter in ./MRPC/config.json file (downloaded during Notebook runtime).

List of model_types can be found here.

Solution:

Code that appends model_type to config.json, in the same format:

Source https://stackoverflow.com/questions/70697470

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install MPNet

We implement MPNet and this pre-training toolkit based on the codebase of fairseq. The installation is as follow:.

Support

A unified view and implementation of several pre-training models including BERT, XLNet, MPNet, etc.Code for pre-training and fine-tuning for a variety of language understanding (GLUE, SQuAD, RACE, etc) tasks.

Find more information at: