neural_network | Some Deep Learning related projects | Machine Learning library

so I checked the Wine Quality dataset, and upon doing:

Probably because the gradient flow graph for NN is destroyed with the gradH step. (check HH.grad_fn vs gradH.grad_fn )

So your pred tensor (and subsequent loss) does not contain the necessary gradient flow through the NN network.

The loss contains gradient flow for the input X, but not for the NN.parameters(). Because the optimizer only take a step() over thoseNN.parameters(), the network NN is not being updated, and since X is neither being updated, loss does not change.

You can check how the loss is sending it's gradients backward by checking loss.grad_fn after loss.backward() and here's a neat function (found on Stackoverflow) to check it:

You are currently passing your preprocessing steps as two separate arguments when calling the add method. You can instead combine them as follows:

As you are using regression, the model will attempt to predict a continuous value, which is how you are training it (note that the output Y is a single value):

• from X = [1,2,3] fit Y = 0
• from X = [2,1,3] fit Y = 1
• from X = [2,1,2] fit Y = 2

The output you are expecting is the one for a classifier, where each class gets a probability as output, i.e. confidence of the prediction. You should use a classification model instead, if that's what you want/need. And training it accordingly (each index in the output represents a class)

• from X = [1,2,3] fit Y = [1, 0, 0]
• from X = [2,1,3] fit Y = [0, 1, 0]
• from X = [2,1,2] fit Y = [0, 0, 1]

This is called Sigmoid Neuron. It is not really a network but a single block of a NN. You can read about it here

How about to use kernel? Kernel is a way of a model to to extract the desirable features from data.

Generally used kernels may not satisfy your requirement. I believe they try to find 'cut' hyperplane between one hyperplane which contains [0, 0] and [1, 1] and another hyperplane which contains [0, 1].

In 2-dimensional space, for example, one hyperplane is y = x and another hyperplane is y = x + 1. Then 'cut' hyperplane could be y = x + 1/2.

So I suggest the following kernel.

Yes, it resets the weights, as you can see here on the documentation

Calling fit() more than once will overwrite what was learned by any previous fit()

That's very normal accuracy for a such network like this. You only have Dense layers which is not sufficient for this dataset. Cifar-10 is an image dataset, so:

• Consider using CNNs

• Use 'relu' activation instead of sigmoid.

• Try to use image augmentation

• To avoid overfitting do not forget to regularize your model.

Also batch size of 500 is high. Consider using 32 - 64 - 128.

Everything looks fine. You have very big data, so you should wait a bit more to fit it.