tacotron | Audio samples accompanying publications related to Tacotron | Speech library

I'm trying to run tacotron2 on docker within Ubuntu WSL2 (v.20.04) on Win10 2004 build. Docker is installed and running and I can run hello world successfully.

(There's a nearly identical question here, but nobody has answered it.)

When I try to run docker build -t tacotron-2_image docker/ I get the error:

unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /home/nate/docker/Dockerfile: no such file or directory

So then I navigated in bash to where docker is installed (/var/lib/docker) and tried to run it there, and got the same error. In both cases I created a docker directory, but kept getting that error in all cases.

How can I get this to work?

I was earlier able to browse the github repo at https://github.com/r9y9/Tacotron-2/blob/master/wavenet_vocoder/models/wavenet.py easily in browser, so that when I put cursor on top of jResidualConv1dGLU at Line84, it'd highlight and let me click on "Definition" and "References" of class ResidualConv1dGLU.

But I used the same repo in the same browser today, and it doesn't do anything. It doesn't highlight ResidualConv1dGLU or show links for Definition/References of it. It's as if it doesn't know that it's a class.

Is there some default setting needed to enable that? What am I missing?

PS: (It was working a few days ago, so I am not sure what changed in just a few days)

I am trying to implement Tacotron speech synthesis with Tensorflow in Google Colab using this code form a repo in Github, below is my code and working good till the step of using localhost server, how I can to run a localhost server in a notebook in Google Colab?

My code:

For example I have a wav file with speech.

I can create nice spectrogram visualization with sox:

My folder structure:

I want to use a Linear, Fully-Connected Layer as one of the input layers in my network. The input has shape (batch_size, in_channels, num_samples). It is based on the Tacotron paper: https://arxiv.org/pdf/1703.10135.pdf, the Enocder prenet part. It feels to me as if Chainer and PyTorch have different implementations of the Linear layer - are they really performing the same operations or am I misunderstanding something?

In PyTorch, behavior of the Linear layer follows the documentations: https://pytorch.org/docs/0.3.1/nn.html#torch.nn.Linear according to which, the shape of the input and output data are as follows:

Input: (N,∗,in_features) where * means any number of additional dimensions

Output: (N,∗,out_features) where all but the last dimension are the same shape as the input.

Now, let's try creating a linear layer in pytorch and performing the operation. I want an output with 8 channels, and the input data will have 3 channels.

I am trying to find databases like the LJ Speech Dataset made by Keith Ito. I need to use these datasets in TacoTron 2 (Link), so I think datasets need to be structured in a certain way. the LJ database is linked directly into the tacotron 2 github page, so I think it's safe to assume it's made to work with it. So I think Databases should have the same structure as the LJ. I downloaded the Dataset and I found out that it's structured like this:

I want to ask you how we can effectively re-train a trained seq2seq model to remove/mitigate a specific observed error output. I'm going to give an example about Speech Synthesis, but any idea from different domains, such as Machine Translation and Speech Recognition, using seq2seq model will be appreciated.

I learned the basics of seq2seq with attention model, especially for Speech Synthesis such as Tacotron-2. Using a distributed well-trained model showed me how naturally our computer could speak with the seq2seq (end-to-end) model (you can listen to some audio samples here). But still, the model fails to read some words properly, e.g., it fails to read "obey [əˈbā]" in multiple ways like [əˈbī] and [əˈbē].

The reason is obvious because the word "obey" appears too little, only three times out of 225,715 words, in our dataset (LJ Speech), and the model had no luck.

So, how can we re-train the model to overcome the error? Adding extra audio clips containing the "obey" pronunciation sounds impractical, but reusing the three audio clips has the danger of overfitting. And also, I suppose we use a well-trained model and "simply training more" is not an effective solution.

Now, this is one of the drawbacks of seq2seq model, which is not talked much. The model successfully simplified the pipelines of the traditional models, e.g., for Speech Synthesis, it replaced an acoustic model and a text analysis frontend etc by a single neural network. But we lost the controllability of our model at all. It's impossible to make the system read in a specific way.

Again, if you use a seq2seq model in any field and get an undesirable output, how do you fix that? Is there a data-scientific workaround to this problem, or maybe a cutting-edge Neural Network mechanism to gain more controllability in seq2seq model?

Thanks.

sorry if my question is so dump but i spent a lot of time trying to understand the reason of the problem but i couldn't so here it is

i'm training tacotron model on google cloud ML i have trained it before on floyd hub and it was pretty fast so i configured my project to be able to run on google ML

this is the major changes that i made to my project

original