Hello fellow learners,
During the process of multiple linear regression, we update every component of w, since w is a vector. So, once we update the first component of w (w1), do we use that updated value of w1 while updating the value of w2, w3 and so on?
Or is it like we first compute the new values of w1, w2, w3, …, wn, and then update all of them at once?
Intuitively, I feel it should be the second one, but asking for confirmation.
Thank you and Happy Learning!
Yash
Hello @Yash_Barve
Welcome to the community.
Your intuition is right. We update the new values of w1,w2,…,wn all at once.
Hey @Yash_Barve,
Welcome to the community. Just to add to @shanup’s explanation, I would like to mention that Prof. Andrew in the lecture videos explicitly mentioned about using simultaneous updates. The primary reason as to why simultaneous updates are preferred over the non-simultaneous ones is computational efficiency. You can check out this great discussion which goes one step ahead. It keeps the computational efficiency aside just for fun, and then discusses which of the 2 approaches might be more suitable. I hope this helps.
Regards,
Elemento