FastSpeech | The Implementation of FastSpeech based on pytorch | Speech library

Denote the hidden states of the phoneme sequence as Hpho = [h1, h2, ..., hn], where n is the length of the sequence. Denote the phoneme duration sequence as D = [d1, d2, ..., dn], where sum of di = m and m is the length of the mel-spectrogram sequence. We denote the length regulator LR as Hmel = LR(Hpho, D, α), (1) where α is a hyperparameter to determine the length of the expanded sequence Hmel, thereby controlling the voice speed. For example, given Hpho = [h1, h2, h3, h4] and the corresponding phoneme duration sequence D = [2, 2, 3, 1], the expanded sequence Hmel based on Equation 1 becomes [h1, h1, h2, h2, h3, h3, h3, h4] if α = 1 (normal speed). When α = 1.3 (slow speed) and 0.5 (fast speed), the duration sequences become Dα=1.3 = [2.6, 2.6, 3.9, 1.3] ≈ [3, 3, 4, 1] and Dα=0.5 = [1, 1, 1.5, 0.5] ≈ [1, 1, 2, 1], and the expanded sequences become [h1, h1, h1, h2, h2, h2, h3, h3, h3, h3, h4] and [h1, h2, h3, h3, h4] respectively.

above text is from a paper FastSpeech TTS model. Here the Hpho sequence is a 3D tensor [batch_size, text_length, word_dim], the D sequence is 1D tensor [N]. how to imlement the target tensor Hmel? Hmel is also a 3D tensor [N, mel_length, word_dim]