Perceptron | Perceptron - | Machine Learning library

This is not a direct answer, but an alternative. PyTorch already has functionality for this: torch.nn.functional.one_hot. So, if you have a label tensor label and n classes, just call:

The issue is not on result, it's either on X, W_ih, or torch.where(outputs > 0, outputs, 0.).

If you don't set an argument for the dtype of torch.rand(), it will assign the dtype based on the pytorch's global default value.

The global variable can be changed using torch.set_default_tensor_type().

Or go the easy route:

Here is the problem:

I seem to have fixed it. I made three main changes:

1. I switched the a and o in the output layer error calculation which then looked like this: error = (o - a) * self.activationPrime( self.zCollection[-1] ).

2. When updating the weights and biases I replaced

Two things I noted in your code:

1. When you assign numpy arrays to a new variable, they are treated as the same variable, ie. changes to the new variable will affect the initial. That means your code will change y_train when you change your y_train_01_subset, y_train_02_subset, and y_train_03_subset values. This means that your code will only work for your first subset because y_train will be all -1's when you reach the second set. Use .copy() to get around this.
2. In your y_train_03_subset, you set your values equal to 0 to be 2, and then set values equal to 2 equal to -1, so all of your values will be -1. Rearrange these lines.

To get around these, substitute in the following code:

You simply have to cut your trained network in the middle, wherever you like but paying attention that your network returns 2D data. Only in this way you can fit a tabular model.

In the example below, we fit a dense NN and then apply a RandomForest from sklearn on the hidden extracted features. You can modify the network or change the final tabular model

The one issue that immediately pops out is that your derivative of the sigmoid seems incorrect. The derivative of sigmoid(x) is not equal to (x)*(1-x) but sigmoid(x)(1-sigmoid(x))

You can change your implementation accordingly

So finally I found a solution, actually it is more a workaround. I write it here wishing that it can be useful for some other ML practictioner. The explanation of my problem is simple and can be explained in 3 steps:

1. sklearn do not provide a method to plot the training history of the model. I found something similar to the keras history only in the MLPclassifier that has an attribute loss_
2. tensorflow and keras do not provide crossvalidation and pipelines routines to avoid data-leakage (since usually in deep learning there is not room for CV)
3. wrapping a keras MLP using KerasClassifier and putting it in a sklearn pipeline is not useful because it is not possible to extrapolate the history of the classifier of the pipelin (when calling the fit function).

So finally I used the sklearn function plot_validation_curve to create a plot of the MLP loss function in function of the training epochs. In order to avoid data-leakage I used a pipeline and the cross validation method of sklearn.

After I put the save code into with Session... issue solved.