neural-network-from-scratch | A Neural Network implemented from scratch | Machine Learning library

When using the chain rule to calculate the slope of the cost function relative to the weights at the layer L , the formula becomes:

d C0 / d W(L) = ... . d a(L) / d z(L) . ...

With :

z (L) being the induced local field : z (L) = w1(L) * a1(L-1) + w2(L) * a2(L-1) * ...

a (L) beeing the ouput : a (L) = & (z (L))

& being the sigmoid function used as an activation function

Note that L is taken as a layer indicator and not as an index

Now:
d a(L) / d z(L) = &' ( z(L) )

With &' being the derivative of the sigmoid function

The problem:

But in this post which is written by James Loy on building a simple neural network from scratch with python,
When doing the backpropagation, he didn't give z (L) as an input to &' to replace d a(L) / d z(L) in the chain rule function. Instead he gave it the output = last activation of the layer (L) as the input the the sigmoid derivative &'

...

I am reading : https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6

I saw following code:

I have been trying to make my own neural networks from scratch. After some time, I made it, but I run into a problem I cannot solve. I have been following a tutorial which shows how to do this. The problem I run into, was how my network updates weights and biases. Well, I know that gradient descent won't be always decreasing loss and for a few epochs it might even increase a bit, bit it still should decrease and work much better than mine does. Sometimes the whole process gets stuck on loss 9 and 13 and it cannot get out of it. I have checked many tutorials, videos and websites, but I couldn't find anything wrong in my code. self.activate, self.dactivate, self.loss and self.dloss:

I've been following a tutorial on applying CNN to classify the MNIST handwritten numbers dataset.

I'm just a bit confused on one point in K-fold Cross-validation. The author of this tutorial mentions in another tutorial that the model should be discarded each time the folds are swapped around. So to quote it:

1. Shuffle the dataset randomly.
2. Split the dataset into k groups
3. For each unique group:
   • Take the group as a hold out or test data set
   • Take the remaining groups as a training data set
   • Fit a model on the training set and evaluate it on the test set
   • Retain the evaluation score and discard the model
4. Summarize the skill of the model using the sample of model evaluation scores

Although in the CNN tutorial this is how the author applies K-fold validation:

The following code that I pulled from here in an effort to better understand how Machine Learning and Neural Networks work, isn't working. It keeps producing an "invalid syntax" error at line 31:

Trying to reproduce a loss function from an example found here. The provided code doesn't show how to plot from a looped class. Here is the original code:

So I’m trying to program a basic Artificial Neural Network using Excel VBA. I’ve been following one example in particular:

https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/

I’ve been basing the code off of the Python example in the article (code example is toward the bottom of the article). I don’t have access to Python unfortunately so I’m trying to do this with VBA. I’ve done my best to convert the code into a useable format for Excel, however there is an issue I'm encountering and I'm not sure how to solve it:

Here is my VBA code:

I'm watching this series https://www.youtube.com/watch?v=wlnx-7cm4Gg&list=PL5tcWHG-UPH2zBfOz40HSzcGUPAVOOnu1 which is about mining tweets with tweepy (python) and the guy stores the tweets with everything ( such as created_at, id, id_str, text) and then he uses Dataframes in pandas to store only the text. Is this way efficient ? How Can I only store the "text" in the Json file instead of all other details ?

The code:

I'm a computer science teacher currently building an introductory course on deep learning. Python and the Keras framework are my tools of choice.

I'd like to show my students what is overfitting by training a model of increasing complexity on some predefined 2D data, just like at the end of this example.

The same idea appears in a programming activity for Andrew Ng's course on neural networks tuning.

However, no matter how hard I try, I can't replicate this behavior with Keras. Using the same dataset and hyperparameters, the decision boundaries are always "smoother" and the model never fits the noisy points in the dataset. See my results below and click here to browse the associated code. Here's the relevant extract: