Yes, it is a dot product followed by a sum. It is important to recognize the notational conventions that Prof Ng uses. When he means “elementwise” multiply, he always and only uses “*” as the operator. If he means dot product, then he just writes the operands adjacent to each other without any explicit operator.
So when he writes it this way:
Z^{[l]} = W^{[l]}A^{[l-1]} + b^{[l]}
The first operation there is dot product. I think it would be clearer to write it this way:
Z^{[l]} = W^{[l]} \cdot A^{[l-1]} + b^{[l]}
But Prof Ng is the boss and he didn’t ask my opinion, so we just have to understand his notation.
Here’s another thread from a while back that covers this in a bit more detail.