fastBPE | Neural Machine Translation of Rare Words | Natural Language Processing library
kandi X-RAY | fastBPE Summary
kandi X-RAY | fastBPE Summary
C++ implementation of Neural Machine Translation of Rare Words with Subword Units, with Python API.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of fastBPE
fastBPE Key Features
fastBPE Examples and Code Snippets
Community Discussions
Trending Discussions on fastBPE
QUESTION
I'm trying to run the code 'transformers' version of this code to use the new pre-trained BERTweet model and I'm getting an error.
The following lines of code ran successfully in my Google Colab notebook:
...ANSWER
Answered 2020-Jun-16 at 12:15First of all you have to download the proper package as described in the github readme:
QUESTION
I am trying to make a transalation chatbot with the help of a github repository. Initial step is to install fastBPE package. When running "pip install fastBPE" command in terminal, I am getting following error:
...ANSWER
Answered 2020-Oct-13 at 06:47As far as i know it is not really made for windows. You could try to install it manually by following the instructions from this post.https://github.com/pytorch/fairseq/issues/1224
QUESTION
I am trying to deploy a Python server to Heroku, and I need to execute a "g++" command on one of the libraries to install it on the server.
I want to create a gunicorn and Flask server hosting facebook's XLM model from cross-lingual model pretaining : https://github.com/facebookresearch/XLM
The model requires the "fastBPE" library (https://github.com/glample/fastBPE), which requires to be installed with the command : g++ -std=c++11 -pthread -O3 fastBPE/main.cc -IfastBPE -o fast
However, since the Heroku server is configured for Python, it doesn't recognize the "g++" command.
Here is what I tried so far : - adding the buildpack “heroku-buildpack-apt” in Heroku and creating an "Aptfile" in my source file, to write "g++" inside of it, as well as "build-essential" - inside the main python file, I create a subprocess to launch "apt-get install g++" :
...ANSWER
Answered 2019-Jun-17 at 09:16I managed to figure it out eventually.
For posterity, there are 2 solutions that worked for me :
1 - The not-so-good-one was to execute the g++ command on a Linux computer with exactly the same environment as the Heroku server, push it to Heroku and make sure to never modify it afterwards. You can then call fastBPE with a subprocess like above ==> it works, but it's more of a DIY unstable solution. The associated GitHub main file is https://github.com/Tony4469/xlm-agir/blob/master/mlm_tlm_prod.py
2 - The best solution was to precompile everything on a Docker container with a Miniconda environment, you can install and run all the necessary commands and then push it easily to heroku. You can find the Dockerfile I used here : https://github.com/Tony4469/laser-agir/blob/master/Dockerfile
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install fastBPE
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page