Neural network tutorial: The back-propagation algorithm (Part 1)
Why Stack Overflow recommends it
Stack Overflow users link it when explaining how weights are updated in a multilayer perceptron and when asking about the math behind backpropagation.
Recommended by 2 people across 2 different months, 2013 to 2017.
What it is
A 13-minute tutorial by Ryan Harris that derives the back-propagation algorithm for neural networks using the sigmoid transfer function. This first part covers the derivative of the sigmoid and carries the derivation through to the delta for the output layer, with slides and a diagram supplied.
What people said
Strong rec jorgenkg · 2013-08 · Stack Overflow · on “When I need to update weights in MultiLayer Perceptron?”
I've appended my answer. Here is a mathmatical proof that the `first method` in your question is a mathmatical error. I will still recommend this youtube video. It taught me more than my professor. [this video]
Recommends TheJD · 2017-09 · Stack Overflow · on “How does one train the input-to-hidden weights in a neural network?”
This video [this video] on YouTube should be a good resource to start with! If you're just interested in the mathematical formula, you can skip to the end of the second part. I recommend watching both videos if you want to know how the weights are updated. Using this video, I developed following algorithm: public void applyBackpropagation(double expectedOutput[]){ for (int i = 0; i < expectedOutput.length; i++) { double d = expectedOutput[i]; if (d < 0 || d > 1) { if (d < 0) expectedOutput[i] = 0 + epsilon; else expectedOutput[i] = 1 - epsilon; } } int i = 0; for(Neuron n : outputLayer){ ArrayList<Connection> connections = n.getAllInConnections(); double Ok = n.getOutput(); double tk = expectedOu
Cites Snate · 2016-03 · Stack Overflow · on “Derivation of the equation of backpropagation algorithm”
Can someone provide a derivation of the equation he gets on 1:15 using quotient rule [this video] Pretty simple question, but I get another equation, so I'm stuck. Thanks in advance.
Cites RaviTej310 · 2016-05 · Stack Overflow · on “Neural network bias for each neuron”
I have been following Andrew NG's videos on neural networks. In these videos, he doesn't associate a bias to each and every neuron. Instead, he adds a bias unit at the head of every layer after their activations have been computed and uses this bias along with the computations to calculate the activations of the next layer (forward propogation). However, in some other blogs on machine learning and videos like this [this video] , there is a bias being associated with each neuron. What and why is this difference and what are it's implications?