In Linear regression the gradient of the loss with respect to w, have same shape as w so does it mean that the differentiation will take the dimension of the variable with which I'm differentiating?


You filed this under DLS C4, which is about Convolutional Networks and Linear Regression is not a topic in DLS C4. I’m guessing maybe you are talking about one of the MLS courses.

It’s also a good idea to put the long sentences in the body of the post and keep the title something relatively short, but expressive.

Yes, you’re right that the gradients are of J (the cost) with respect to one of the parameter variables we need to learn, such as w or b. So it is a partial derivative of J, but it will have the shape of the variable that it is “with respect to”, meaning w and b. That makes sense because the purpose of the gradient is to update w or b, right? So it needs to be the same shape and type as the “target” variable.