The dot product in NN

i was wondering how can we do a dot product of w[1] with a[2] in the case where the vector size of the layers are not the same…meaning if the size of layer 1 is 4 and layer 2 is 5 like in the picture

I think i misunderstood the representation of vector a and w in this part
thank you

  1. l ↩︎

  2. l-1 ↩︎

Hello @Chaya_Ben_Yehuda

Welcome to the community.

When you want to take the dot product of 2 matrices, the inner dimensions have to match.

Say you have a matrix P with shape (p1,p2) and another Q with shape (q1,q2). To take the dot product \vec P. \vec Q, the condition p2 = q1 has to be satisfied. And the output of \vec P.\vec Q will be of shape (p1,q2).

In the case of Layer 1 of size 4 and Layer 2 of Size 5:
Shape of \vec {a_1} = (4,1)
Shape of \vec {a_2} = (5,1)

The shape of \vec {w}.\vec {a_1} has to match with shape of \vec {a_2} . With the shape of \vec a_1 and \vec a_2 being fixed, lets determine the shape of \vec w

Referring to the shapes:
(w1,w2).(4,1) = (5,1)

As explained above, w2 = 4 has to be satisfied to be able to take dot product, and w1 = 5, thereby giving the shape of \vec w = (5,4), such that

(5,4).(4,1) = (5,1)

Hope its clear now.


Thank you! :slightly_smiling_face: it makes much more sense now

Good question! Moved this to course 2 week 1.