scripts_python | Python 脚本 | Crawler library

I'm trying to understand RNNs (not a specific one) with the Reber Grammar inputs (not embedded for now). You can find the jupyter notebook on this link (please disregard markdowns because I failed on the first version with output and it's not up-to-date :) ).

For every timestep, I provide the input and expected output for the training (so it's a many-to-many model).

• Input/output are "OneHotEncoded" (based on the string "BTSXPVE") so for example

  • B is [1, 0, 0, 0, 0, 0, 0]
  • V is [0, 0, 0, 0, 0, 1, 0]

• For the timesteps, I have string with unknown lenght (not encoded here to make it clearer) for example:

  • BPVVE
  • BPVPXVPXVPXVVE

so I decided to pad them to 20 timesteps.

• For the batch, I'm free. I've generated 2048 encoded strings for training and 256 for test.

So my input tensor is (2048, 20, 7). My output tensor is also (2048, 20, 7) because for every timestep I would like to get the prediction.

So I trained 3 many-to-many models (Simple RNN, GRU and LSTM) like the following code.