neural-network-from-scratch | Implementation of a neural network from scratch | Machine Learning library

If you have your data in a csv file and images as the target in separate folders, so one of the best ways is to use flow_from_dataframe generator from keras libraries. Here is an example, and a more detailed example on keras library here. It's also the documentations.

Here is some sample code:

I see you use a model from scratch. In this case, you should run this code, as indicated in the notebook, after setting your X and y for your new test set. For more information, see the the notebook as I did not put here everything:

You want to use the derivative with respect to the output. During backpropagation we use the weights only to determine how much of the error belongs to each one of the weights and by doing so we can further propagate the error back through the layers.

In the tutorial, the sigmoid is applied to the last layer:

Your input matrix X suggests that number of samples is 4 and the number of features is 3. The number of neurons in the input layer of a neural network is equal to the number of features*, not number of samples. For example, consider that you have 4 cars and you chose 3 features for each of them: color, number of seats and origin country. For each car sample, you feed these 3 features to the network and train your model. Even if you have 4000 samples, the number of input neurons do not change; it's 3.

So self.weights1 is of shape (3, 4) where 3 is number of features and 4 is the number of hidden neurons (this 4 has nothing to do with the number of samples), as expected.

*: Sometimes the inputs are augmented by 1 (or -1) to account for bias, so number of input neurons would be num_features + 1 in that case; but it's a choice of whether to deal with bias seperately or not.

This can be caused by your training data. Either it is too small or too many diverse labels (What i get from your code from the link you share).

I re-run your code several times and it produce different training performance. Sometimes the loss keeps decreasing until last epoch, some times it keep increasing, in one time it decreased until some point and it increasing. (With minimum loss achieved of 0.5)

I think it is your training data that matters this time. The learning rate is good enough though (Assuming you did the calculation for Linear combination, back propagation, etc right).