The way to debug this type of problem is to work out the “dimensional analysis” for the case at hand. That means you start with the dimensions of all the input objects and the weight matrices and then you work through the layers in forward propagation and see what the shapes of the Z and A values will be at each level. Here is an earlier thread which shows that for the particular test case here.
Please have a look at that thread and it will make clear that you are at the output layer and the input should be A2, which should be of shape 3 x 4. That would fit with W3 being 1 x 3, right? So how did it happen that your A2 ends up being 4 x 4 instead? One clue is that the shape of A1 is legitimately 4 x 4. Hmmmm.