pytorch-seq2seq | An open source framework for seq2seq models in PyTorch | Machine Learning library

 by   IBM Python Version: 0.1.6 License: Apache-2.0

kandi X-RAY | pytorch-seq2seq Summary

kandi X-RAY | pytorch-seq2seq Summary

pytorch-seq2seq is a Python library typically used in Artificial Intelligence, Machine Learning, Deep Learning, Pytorch, Neural Network applications. pytorch-seq2seq has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. You can download it from GitHub.

An open source framework for seq2seq models in PyTorch.

            kandi-support Support

              pytorch-seq2seq has a medium active ecosystem.
              It has 1421 star(s) with 376 fork(s). There are 59 watchers for this library.
              It had no major release in the last 12 months.
              There are 34 open issues and 80 have been closed. On average issues are closed in 47 days. There are 3 open pull requests and 0 closed requests.
              It has a neutral sentiment in the developer community.
              The latest version of pytorch-seq2seq is 0.1.6

            kandi-Quality Quality

              pytorch-seq2seq has 0 bugs and 0 code smells.

            kandi-Security Security

              pytorch-seq2seq has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              pytorch-seq2seq code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              pytorch-seq2seq is licensed under the Apache-2.0 License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              pytorch-seq2seq releases are available to install and integrate.
              Build file is available. You can build the component from source.
              Installation instructions, examples and code snippets are available.

            Top functions reviewed by kandi - BETA

            kandi has reviewed pytorch-seq2seq and discovered the below as its top functions. This is intended to give you an instant insight into pytorch-seq2seq implemented functionality, and help decide if they suit your requirements.
            • Forward computation
            • Inflate tensor
            • Validate inputs
            • Compute predicted softmax
            • Update the cuda
            • Backtrack decoding
            • Predict n features from src_seq
            • Get features from src_seq
            • Load model
            • Flattens the parameters
            • Evaluate the model
            • Reset acc_loss
            • Generate a dataset
            • Run the decoder
            • Predict sequence from src_seq
            • Builds the vocab
            • Draw the cuda
            Get all kandi verified functions for this library.

            pytorch-seq2seq Key Features

            No Key Features are available at this moment for pytorch-seq2seq.

            pytorch-seq2seq Examples and Code Snippets

            Naver AI Hackathon Speech - Team Kai.Lib,Model
            Pythondot img1Lines of Code : 32dot img1License : Permissive (Apache-2.0)
            copy iconCopy
              (encoder): EncoderRNN(
                (input_dropout): Dropout(p=0.5, inplace=False)
                (conv): Sequential(
                  (0): Conv2d(1, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                  (1): BatchNorm2d(16, eps=1e-05, momentum=0.1, a  
            copy iconCopy
            make download-datasets
            make normalize-datasets
            make apply-transforms-sri-py150
            make apply-transforms-c2s-java-small
            make extract-transformed-tokens
            Sequence MNIST,Requirements
            Pythondot img3Lines of Code : 4dot img3no licencesLicense : No License
            copy iconCopy

            Community Discussions


            RuntimeError: The size of tensor a (1024) must match the size of tensor b (512) at non-singleton dimension 3
            Asked 2020-Aug-27 at 06:07

            I am doing the following operation,



            Answered 2020-Aug-27 at 06:07

            I took a look at your code (which by the way, didnt run with seq_len = 10) and the problem is that you hard coded the batch_size to be equal 1 (line 143) in your code.

            It looks like the example you are trying to run the model on has batch_size = 2.

            Just uncomment the previous line where you wrote batch_size = query.shape[0] and everything runs fine.



            Implementing Attention
            Asked 2020-Jun-18 at 07:53

            I'm implementing the Attention in PyTorch. I got questions during implementing the attention mechanism.

            1. What is the initial state of the decoder $s_0$? Some post represents it as zero vector and some implements it as the final hidden state of the encoder. So what is real $s_0$? The original paper doesn't mention it.

            2. Do I alternate the maxout layer to dropout layer? The original paper uses maxout layer of Goodfellow.

            3. Is there any differences between encoder's dropout probability and decoder's? Some implementation sets different probabilities of dropouts for encoder and decoder.

            4. When calculating $a_{ij}$ in the alignment model (concat), there are two trainable weights $W$ and $U$ . I think the better way to implement it is using two linear layers. If I use a linear layer, should I remove bias term in the linear layers?

            5. The dimension of the output of the encoder(=$H$) doesn't fit the decoder's hidden state. $H$ is concatenated, so it has to be 2000 (for the original paper). However, the decoder's hidden dimension is also 1000. Do I need to add a linear layer after the encoder to fit the encoder's dimension and the decoder's dimension?



            Answered 2020-Jun-18 at 07:53

            In general, many answers are: it is different in different implementations. The original implementation from the paper is at For later implementations that reached better translation quality, you can check:

            Now to your points:

            1. In the original paper, it was a zero vector. Later implementations use a projection of either of the encoder final state or the average of the encoder states. The argument for using average is that it propagates the gradients more directly into the encoder states. However, this decision does not seem to influence the translation quality much.

            2. Maxout layer is a variant of a non-linear layer. It is sort of two ReLU layers in one: you do two independent linear projections and take the maximum of them. You can happily replace Maxout with ReLU (modern implementations do so), but you still should use dropout.

            3. I don't know about any meaningful use case in MT when I would set the dropout rates differently. Note, however, that seq2seq models are used in many wild scenarios when it might make sense.

            4. Most implementations do use bias when computing attention energies. If you use two linear layers, you will have the bias split into two variables. Biases are usually zero-initialized, they will thus get the same gradients and the same updates. However, you can always disable the bias in a linear layer.

            5. Yes, if you want to initialize s0 with the decoder states. In the attention mechanism, matrix U takes care of it.


            Community Discussions, Code Snippets contain sources that include Stack Exchange Network


            No vulnerabilities reported

            Install pytorch-seq2seq

            This package requires Python 2.7 or 3.6. We recommend creating a new virtual environment for this project (using virtualenv or conda).
            Currently we only support installation from source code using setuptools. Checkout the source code and run the following commands:. If you already had a version of PyTorch installed on your system, please verify that the active torch package is at least version 0.1.11.


            If you have any questions, bug reports, and feature requests, please open an issue on Github. For live discussions, please go to our Gitter lobby. We appreciate any kind of feedback or contribution. Feel free to proceed with small issues like bug fixes, documentation improvement. For major contributions and new features, please discuss with the collaborators in corresponding issues.
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
          • HTTPS


          • CLI

            gh repo clone IBM/pytorch-seq2seq

          • sshUrl


          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link