LSTM | Playing around with various LSTM architectures | Machine Learning library

An LSTM cell in Keras gives you three outputs:

• an output state o_t (1st output)
• a hidden state h_t (2nd output)
• a cell state c_t (3rd output)

and you can see an LSTM cell here:

The output state is generally passed to any upper layers, but not to any layers to the right. You would use this state when predicting your final output.

The cell state is information that is transported from previous LSTM cells to the current LSTM cell. When it arrives in the LSTM cell, the cell decides whether information from the cell state should be deleted, i.e. we will "forget" some states. This is done by a forget gate: This gate takes the current features x_t as an input and the hidden state from the previous cell h_{t-1}. It outputs a vector of probabilities that we multiply with the last cell state c_{t-1}. After determining what information we want to forget, we update the cell state with the input gate. This gate takes the current features x_t as an input and the hidden state from the previous cell h_{t-1} and produces an input which is added to the last cell state (from which we have already forgotten information). This sum is the new cell state c_t. To get the new hidden state, we combine the cell state with a hidden state vector, which is again a vector of probabilities that determines which information from the cell state should be kept and which should be discarded.

As you have correctly interpreted, the first tensor is the output of all hidden states.

The second tensor is the hidden output, i.e. $h_t$, which acts as the short-term memory of the neural network The third tensor is the cell output, i.e. $c_t$, which acts as the long-term memory of the neural network

In the keras-documentation it is written that

I made a mistake in the code itself while executing the Model part of in the functional API version.

It seems that you want to understand that how could you use your dataset and apply LSTMs over it to get some meaningful out of your data.

Now here you can reframe your data set to create more features from your present data set for eg.

Features That could be derived out of Data

1. Take Out day of the month (which day is it 1-31)
2. Week of the month (which week of month it is 1-4)
3. Day of the week (Monday - Saturday)
4. what is the time ( you can have any of the value out of 188)

Features that could be added from opensource data

1. What is the wheather of the day
2. Is there any holiday nearby(days remaining for next holiday/function etc.)

Now let's Assume for each row you have K features in your data and you have a target that you have to predict which is what is the availability of parking. P(#parking_space|X)

Now just just keep your timesteps as a variable while creating your model and reshape your data from X.shape-->(Examples, Features) to the format X.shape-->(examples,Timesteps,Features). You can use below code and define your own look_back

Here your architecture will be many to many with Tx=Ty

You need to pass x_train and y_train into the fit statement.

I found the problem and so I'm answering my own question.

There is a setting in Keras that specifies the way of working with (and supossedly affecting only) image data.

• Channels Last. Image data is represented in a three-dimensional array where the last channel represents the color channels, e.g. [rows][cols][channels].

• Channels First. Image data is represented in a three-dimensional array where the first channel represents the color channels, e.g. [channels][rows][cols].

Keras keeps this setting differently for different backends, and this is supossedly set as Channels Last for Tensorflow, BUT it looks like in our planets it is set as Channels First.

Thankfully, this can be set manually and I managed to fix it with:

You can use pandas pd.concat() to combine multiple dataframes with the same columns (pandas docs).

You can iterate through that directory to create a list of csv file names, read each csv using pd.read_csv(), and then concatenate into a final dataframe with something like this:

Simply pass "sample weights", which will be 1/(eps**2) for each sample.

Your generator should just output x, y, sample_weights and that's all.

Your loss can be:

Add your input layer at the beginning. Try this

This error indicates that, you have defined an activation function that is not interpretable. In your definition of a dense layer you have passed two argument as layers[i] and layers[i+1].

Based on the docs here for the Dense function: The first argument is number of units (neurons) and the second parameter is activation function. So, it considers layers[i+1] as an activation function that could not be recognized by the Dense function.

Inference: You do not need to pass next layer neurons to your dense layer. So remove layers[i+1] argument.

Furthermore, you have to define an input layer for your model and pass the input shape to it for your model.

Therefore, modified code should be like this: