LSTM-Neural-Network-for-Time-Series-Prediction | LSTM built using Keras Python package to predict time | Machine Learning library

A few questions:

• Is this graph for training data or validation data?
• Do you consider it better because:
  • The graph seems cool?
  • You actually have a better "loss" value?
    • If so, was it training loss?
    • Or validation loss?

The early graph seems interesting, indeed, but take a close look at it:

I clearly see huge predicted valleys where the expected data should be a peak

Is this really better? It sounds like a random wave that is completely out of phase, meaning that a straight line would indeed represent a better loss than this.

Take a look a the "training loss", this is what can surely tell you if your model is better or not.

If this is the case and your model isn't reaching the desired output, then you should probably make a more capable model (more layers, more units, a different method, etc.). But be aware that many datasets are simply too random to be learned, no matter how good the model.

In case you actually have a better training loss. Ok, so your model is indeed getting better.

• Are you plotting training data? - Then this straight line is actually better than a wave out of phase
• Are you plotting validation data?
  • What is happening with the validation loss? Better or worse?

If your "validation" loss is getting worse, your model is overfitting. It's memorizing the training data instead of learning generally. You need a less capable model, or a lot of "dropout".

Often, there is an optimal point where the validation loss stops going down, while the training loss keeps going down. This is the point to stop training if you're overfitting. Read about the EarlyStopping callback in keras documentation.

If your training loss is going up, then you've got a real problem there, either a bug, a badly prepared calculation somewhere if you're using custom layers, or simply a learning rate that is too big.

Reduce the learning rate (divide it by 10, or 100), create and compile a "new" model and restart training.

Then you need to detail your question properly.

What is currently going on is that the current frame at each iteration of the loop is set to every remaining data except the current one. We can keep the same structure and just flip each row.

At the beginning of your function, you can reverse the data to make the predictions start from the end and go towards the beginning by adding the line data = np.flip(data, axis=0) which reverses each row of data.

I realised what the problem was. I was trying to extract my model weights using Tensorflow session (after model fitting), rather than via Keras methods directly. This resulted in weights matrices that made perfect sense (dimension wise) but contained the values from initialization step.