NeuralNetwork | Neural Network implementation in Numpy and Keras | Machine Learning library

You should be using tf.keras.Sequential() or tf.keras.models.Sequential(). Also, you need to define a valid loss function. Here is a working example:

1. All initial weights must be DIFFERENT numbers, otherwise backpropagation will not work. For example, you can replace 1 with math.random()
2. Increase number of attempts to 10000

With these modifications, your code works fine:

Formalizing your pipeline to get a good idea of the setup:

Properly scaling the inputs is crucial for proper training. Weights are initialized based on some assumptions on the way inputs are scaled. See this part of a lecture on weight initialization and see how critical it is for proper convergence.

More details on the mathematical analysis of the influence of weight initialization can be found in Sec. 2 of this paper:
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification (ICCV 2015).

While there are unsubstantiated claims that powers of 2 help to optimize performance for various parts of a neural network, it is a convenient method of selecting/testing/finding the right order of magnitude to use for various parameters/hyperparameters.

All you need is

After a lot of searching and research I found the solution:

The solution is to train and test a new classifier, in my case it was another Neural Network, with the concatenated probability scores obtained from both data sets (training and test), of the two classifiers, the Linear SVM and the Neural Network.

An example of this of three Linear SVM Late fusion was implemented in python, and can be found in the following link:

https://github.com/JMalhotra7/Learning-image-by-parts-using-early-and-late-fusion-of-auto-encoder-features

Turns out its just documented incorrectly.

In reality the export from brain.js is this:

The error is a bit misleading, if you try running the code from a fresh kernel, the issues are elsewhere...

There are multiple issues with your code:

• You are not using the correct shape for the target tensor y, here it should have a single dimension since the output tensor is two-dimensional.

• The target tensor should be of dtype Long

• When iterating over you data and selecting input (and target) with data[i, :, :] (and y[i, :]), you are essentially removing the batch axis. However all builtin nn.Module work with a batch axis. You can do a slice to avoid that side effect: with data[i:i+1] and y[i:i+1] respectively. Also do note that x[j, :, :] is identical to x[j].

That being said, the usage of the cross-entropy loss is not justified. You are outputting a single logit, so it doesn't make sense to use a cross-entropy loss.

• You can either output two logits on the last layer of your model,

• or switch to another loss function such as a binary cross-entropy loss (either using nn.BCELoss or a nn.BCEWithLogitsLoss which includes a sigmoid activation).

  In this case the target vector should be of dtype float, and its shape should equal that of pred.