Please help to clarify the below doubts. We are struck and please help.
Doubt 1: How we can arrive values for W1,b1,W2,b2 to match the target ? We are not able to understand how the values actually derived for it ?
Doubt 2: Can someone provide explanation for below ?
The new function is added at transition point but does not contribute to the output prior to that point. The non-linear activation function is responsible for disabling the input prior to and sometimes after the transition points.
Now, only focus on W1 and b1. Make sure we change one thing at a time ;). If we increase W1, the slope of the line will increase. If we increase b1, the position of the transition point will be shifted to the left, however, when b1 is too large, the transition point is disappeared because it is beyond the plot’s frame.
By now, we should already know adjusting W1 and b1 has what effect. Then we can look at the main plot. The goal here is that we want to adjust W1 and b1 so that orange line overlaps on the blue line. There are 3 sections in the blue lines. The first section is done for us, so we only need to adjust 2 sections out. W1 and b1 will take care of the second section, and W2 and b2 will take care of the third section.
Try to adjust W1 first, then you should see the slope of some section of the orange line to change accordingly. Adjust W1 until it is parallel to the second section of the blue line. Then adjust b1 until it levels with the second section. Do the same to W2 and b2 to match the third section.
I have shown you the meaning of a transition point. Instead of answering your question directly, I hope to see your understanding first. Here is my suggestion:
Try out my steps in (1) to fit the target
After getting the perfect fit, further adjust the parameters to understand how they affect each other
read your 2nd question again and try to come up with some explanation based on your understanding.
Thanks Sir I gone through worked out. I understand from the right graph plot Unit 0,1 and 2 that, for Unit 0 in the x-axis, from 0 till 1, there is slope so we should adjust W0 and adjusted bias for cut-off. Then for Unit 1, from 1 till 2 in the x-axis there is slope so we should adjust W1 and adjust bias for cut off. Again for Unit 2, in the x-axis from 2 till 3, there is slope so increase W2 and adjust bias negative for cutoff.
But what does it mean the following statement says sir ? The slope of the unit, 𝑤2, must be set so that the sum of unit 1 and 2 have the desired slope
I added a table on top to show in which region do each of the units “activated” (i.e. the pink line is > 0, or a > 0).
In the table, the third is the region ( z > 2 ) where both unit 1 and 2 are activated (both with a > 0). Since they are both activated, and they are added to give the final result, the final result is affected by both unit 1 and 2. To get the desired slope in the final result, we therefore need to set w2 correctly.
Thanks for the reply sir. I’m able to understand the diagram u drawn but im unable to relate it and cannot understand to the statement ( slope of the unit W2 must be set so that sum of the unit 1 and unit 2 have the desired slope). can you put it in different context pls to understand ?
When you open the lab, w2 is initialized to 0. If you do not set w2 to the right value, the result will not match with the desired target. Therefore, w2 must be set (to the right value).
You can understand it as
“I need to set the value of w2 to the correct value, so that unit 1 and unit 2 will give the slope in the circled region as desired.”
Thanks Sir i got it now. Below is the picture captured for Unit 1 parameters adjusted only . Here in this pic, Unit 1 somewhat bit of close to the desired slope (third segment in blue colour) To achieve the slope for third segment, W2 must be set correctly which w2 =2 so that Unit 1 & 2 together achieves the desired slope (third segment blue colour). Am I right sir ?
It is close, but it does not achieve the desired slope when x > 2. (The region I have circled in my previous post)
No matter how much time you spend on w1 and b1, you will never get the desired slope in the region when x > 2 (The region I have circled in my previous post). You can only get it if you set something else to w2 and b2. I will not share what w2 and b2 should be.
confused: about the third segment and the statement: ."The slope of the unit, 𝑤[1]2, must be set so that the sum of unit 1 and 2 have the desired slope.
"the second segment starts at (1,0) and ends at (2,1): from these two points we get eqn: y=x-1 which does match with the 2nd line segment with w=1 and b=-1
the third segment starts at (2,1) (i guess) and ends at (3,4): if we find an equation it turns out to be y=3x-5 but the slope(w) must actually be 2 and b must be -2 as per the lab
thank you in advance
thank you so much ! think i get it now: please do correct me if i’m wrong
that function is something like a2_0=max(0,-2x+2)+max(0,x-1)+max(0,2x-4)
the reason second segment is not affected by first ReLU function is because it becomes negative after the interval [0,1] but i still don’t understand why does the a1_0(unit 0) affect a1_w(unit 2) [ as unit 0 become 0 in interval (1,infinity)
i agree:
reasoning: the third line segment is affected by the sum of unit 1 and unit 2 because unit 1 has positive value in the interval [2,infinity)
thank you for your help
In the image above, I was wondering why the a2_0 neuron is not of the form wx + b (or more specifically a1_0*w2_0 + a1_1*w2_1 + a1_2*w2_2 + b) because the neuron is using the linear activation function?
Is there a reason it says ‘fixed weights’ for the a2_0 neuron?
