The dot product in NN

Hello @Chaya_Ben_Yehuda

Welcome to the community.

When you want to take the dot product of 2 matrices, the inner dimensions have to match.

Say you have a matrix P with shape (p1,p2) and another Q with shape (q1,q2). To take the dot product \vec P. \vec Q, the condition p2 = q1 has to be satisfied. And the output of \vec P.\vec Q will be of shape (p1,q2).

In the case of Layer 1 of size 4 and Layer 2 of Size 5:
Shape of \vec {a_1} = (4,1)
Shape of \vec {a_2} = (5,1)

The shape of \vec {w}.\vec {a_1} has to match with shape of \vec {a_2} . With the shape of \vec a_1 and \vec a_2 being fixed, lets determine the shape of \vec w

Referring to the shapes:
(w1,w2).(4,1) = (5,1)

As explained above, w2 = 4 has to be satisfied to be able to take dot product, and w1 = 5, thereby giving the shape of \vec w = (5,4), such that

(5,4).(4,1) = (5,1)

Hope its clear now.

2 Likes