• Notes/Tips

  How to use:

  1. Enter your input, weights, and target output in the text fields, or leave as default
     • Hit the "Update Parameters" button if you made any changes
  2. Press the "Forward Pass" button - this will run your input value through the network to produce an output
  3. Press the "Backpropagate" button - this will run a vanilla gradient descent calculation to determine weight updates
  4. Press the "Update Weights" button - this will update the weights using the learning rate in the network
  5. Repeat - and see how the netword output slowly comes to closer to the target you set

  ---

  General notes:

  • This website works best on desktop
  • All numbers are rounded to 4 digits after each operation
  • This is a simple neural network, flowing from left (input node) to right (output node)
  • The "hidden" nodes are the two in the middle, there is a top one, and a bottom one
  • The "Forward Pass", "Backpropagate", and "Update Weights" buttons will appear as you go through the process
  • More details about each operation will also appear as you go through the process

  ---

  Tips:

  Hover To see more info about the calculations

  over underlined text
• Forward Pass Results

  Input value: 0.5

  Input to Top Hidden Node: Input value multiplied by Weight 1

  0.5

  Input to Bottom Hidden Node: Input value multiplied by Weight 2

  0.5

  ---

  Output of Top Hidden Node: Sigmoid function of the input to top hidden node

  0.5

  Output of Bottom Hidden Node: Sigmoid function of the input to bottom hidden node

  0.5

  ---

  Input to Output Node from Top Hidden Neuron: Output of top hidden node multiplied by Weight 3

  0.5

  Input to Output Node from Bottom Hidden Neuron: Output of bottom hidden node multiplied by Weight 4

  0.5

  ---

  Final Input to Output Node: Sum of the weight-adjusted outputs of the hidden nodes

  0.5

  ---

  Output of Output Node: In this case, the output node does not have an activation function

  0.5
• Backpropagation Results
  \(\frac{\partial MSE}{\partial W1}\): Derivative of the MSE w.r.t Weight 1

  0.5

  \(\frac{\partial MSE}{\partial W2}\): Derivative of the MSE w.r.t Weight 2

  0.5

  ---

  \(\frac{\partial MSE}{\partial W3}\): Derivative of the MSE w.r.t Weight 3

  0.5

  \(\frac{\partial MSE}{\partial W4}\): Derivative of the MSE w.r.t Weight 4

  0.5

  ---

  \(\frac{\partial MSE}{\partial HNTO}\): Derivative of the MSE w.r.t Hidden Node Top Output

  0.5

  \(\frac{\partial MSE}{\partial HNBO}\): Derivative of the MSE w.r.t Hidden Node Bottom Output

  0.5

  ---

  \(\frac{\partial MSE}{\partial HNTI}\): Derivative of the MSE w.r.t Hidden Node Top Input

  0.5

  \(\frac{\partial MSE}{\partial HNBI}\): Derivative of the MSE w.r.t Hidden Node Bottom Input

  0.5

  ---

  Error: Error, calculated by target - actual

  0.5

  MSE: Mean Squared Error, calculated by (target - actual)^2

  0.5

  \(\frac{\partial MSE}{\partial Out}\): Derivative of the MSE w.r.t Output

  0.5
• Weight Update Results
  Weight 1: Weight 1 update, calulated with w1 = w1 - ⍺*dMSEdW1

  0.5

  Weight 2: Weight 2 update, calulated with w2 = w2 - ⍺*dMSEdW2

  0.5

  ---

  Weight 3: Weight 3 update, calulated with w3 = w3 - ⍺*dMSEdW3

  0.5

  Weight 4: Weight 4 update, calulated with w4 = w4 - ⍺*dMSEdW4

  0.5

This website was inspired by Matt Mazur, specifically his post about a step-by-step back-propagation example on his website which helped me out a ton when I was learning about neural nets the first time.

When I was learning the ins and outs of how neural nets worked, and their training process, it was extremely hard to find a decent beginner friendly article or video with a numerical demonstration of the backpropagation algorithm. As such, when working out examples with pen and paper it was difficult to know if I got the correct answer, and if my method would work in reality. I thought now that I have some time and I'm passed that point of struggle, I could create a simple website which will help others who might find themselves in a similar position to past me.

For those who want an easy quick-start in building neural nets, you can check out my easy-to-use neural net library on my GitHub. I recommend this library to beginners who are trying to experiment and grasp the different concepts of neural networks rather then professionals trying to build production grade models, as better libraries are available for that.