BackPropNetwork | Backpropagation using Stochastic gradient descent

added the answer for future visitors

• Try converting the grayscale values from 0-255 interval to 0-1 interval. Just divide each pixel with 255. the fact that LeakyRELU performed better than sigmoid or tanh is because the values are too large. large in a sense that they get mistreated by tanh and sigmoid and get rounded by the computer to integers.

• Look carefully about how the neural network weights are initialised if you intend to use tanh or sigmoid.

• Since this is a classification problem, I recommend you use a softmax activation function in your output layer.

after preprocessing the data, @JMC0352 got 88% accuracy only.

the reason why you're getting 88% only is because neural network (alone) is not suitable for image recognition. convolutional neural networks are used for that matter. In order to understand the problem intuitively, you can picture raw neural networks as making sense of all the pixels together, where as conv. nets. make sense of relatively close pixels.