nn-from-scratch | Neural Networks from Scratch Using Python | Machine Learning library

I've implemented a basic neural network from scratch using Tensorflow and trained it on MNIST fashion dataset. It's trained correctly and outputs testing accuracy around ~88-90% over 10 classes.

Now I've written predict() function which predicts the class of given image using trained weights. Here is the code:

Here is my dataset: t.csv
I am looking for a display like this:
The red dot for negative value, grey for 0, and blue for positive.
I tried to refer the example from the : Logistic Regression

I'm trying to train my network on MNIST using a self-made CNN (C++).

It gives enough good results when I use a simple model, like: Convolution (2 feature maps, 5x5) (Tanh) -> MaxPool (2x2) -> Flatten -> Fully-Connected (64) (Tanh) -> Fully-Connected (10) (Sigmoid).

After 4 epochs, it behaves like here 1.
After 16 epochs, it gives ~6,5% error on a test dataset.

But in the case of 4 feature maps in Conv, the MSE value isn't improving, sometimes even increasing 2,5 times 2.

The online training mode is used, with help of Adam optimizer (alpha: 0.01, beta_1: 0.9, beta_2: 0.999, epsilon: 1.0e-8). It is calculated as: