I just checked the assignment, I can’t find this statement you mentioned in the assignment:
“dW = dW_11 + dW_12 + dW_21 + dW22”

In your example, dW will be a 2x2 matrix , and but the statement above assumes it is a scalar value (by adding 4 scalar values together). dW should be a matrix the same size as W, and we will eventually update, element-wise, each value of W based on dW.

Assuming dW is a 2x2 matrix, then dW_11 would be a single value from that 2x2 matrix. A single value without any dimensions is a scalar value.

For example, if you have a 2x2 matrix

A =
[[ 5 2 ],
[ 3 4 ]]

then A_11 would be 5. The 5 here is the scalar value. If you A_11 + A_12 + A_21 + A_22 = 5 + 2 + 3 +4 = 14, you will end up with another scalar value 14.