Week 3 - 2,2,1 Neural Network Questions

Course Q&A Math for Machine Learning Calculus for Machine Learning and Data Science
1

Hi all, would be great if someone can help me with my understanding on the 2,2,1 Neural Network on slide 503 of the PDF:

Currently I do not understand why a1 and a2 should come to different results as - from my understanding :wink:

  • both have the same inputs x1, x2 and b
  • both have the same number of iterations for the optimization and use the same algorithm
  • both use the same starting points for the optimizations
  • So my assumption would be that w11 = w12, w21 = w22 and b1=b2 after the optimization thus I would expect a1 = a2

If this would be correct, I currently do not understand why we should not go with a 2,1 Network.

Would be great if someone could help me what I am missing here. Thx for your help!

Screenshot 2024-11-21 at 22.01.22
Screenshot 2024-11-21 at 22.01.221920×1040 64.2 KB

2

Please post a screen capture image that shows the slide with your question.

3

Thank you - added screenshot to the initial post. Thx

4

That is the assumption that is incorrect. In order to get different results from training, we have to initialize the weight values randomly. That is called “symmetry breaking” and it is required for exactly this reason: if you start them the same, then they stay the same and that is not useful.

1 Like
5

Thank you so much - got it :slight_smile:
Thank you for your help!