C5W4A1 Ex 3 - InvalidArgumentError


I’m getting this error:

InvalidArgumentError: Value for attr ‘T’ of int32 is not in the list of allowed values: bfloat16, half, float, double, complex64, complex128
; NodeDef: {{node Sqrt}}; Op<name=Sqrt; signature=x:T → y:T; attr=T:type,allowed=[DT_BFLOAT16, DT_HALF, DT_FLOAT, DT_DOUBLE, DT_COMPLEX64, DT_COMPLEX128]> [Op:Sqrt]

and the traceback signals the line

scaled_attention_logits = matmul_qk / tf.math.sqrt(dk)

I have calculated dk with the first element of k’s shape and matmul with tf.matmul, transposing the b element (k in this case), so I don’t really know what’s wrong with this line.

May someone please throw some light on this issue? Thanks in advance!

Apparently you can’t use the transpose operator .T on whatever matrix you’re trying to transpose.

Please post a screen capture image that shows the whole assert stack - it will provide more context.

Hi Tom, here’s the image. I’m not using the .T operator, but the transpose_b=True parameter on matmul_qk.

Solved by using tf.cast(dk, tf.float32) instead of just dk on the line for scaling matmul_qk (the one where one implements the formula on which dk is on a sqrt operator).

Don’t understand what the traceback had to do with that, tho :sweat_smile:

1 Like