C5W4A1 Ex 3 - InvalidArgumentError

[ludelaire]

HI!

I’m getting this error:

InvalidArgumentError: Value for attr ‘T’ of int32 is not in the list of allowed values: bfloat16, half, float, double, complex64, complex128
; NodeDef: {{node Sqrt}}; Op<name=Sqrt; signature=x:T → y:T; attr=T:type,allowed=[DT_BFLOAT16, DT_HALF, DT_FLOAT, DT_DOUBLE, DT_COMPLEX64, DT_COMPLEX128]> [Op:Sqrt]

and the traceback signals the line

scaled_attention_logits = matmul_qk / tf.math.sqrt(dk)

I have calculated dk with the first element of k’s shape and matmul with tf.matmul, transposing the b element (k in this case), so I don’t really know what’s wrong with this line.

May someone please throw some light on this issue? Thanks in advance!

[TMosh]

Apparently you can’t use the transpose operator .T on whatever matrix you’re trying to transpose.

Please post a screen capture image that shows the whole assert stack - it will provide more context.

[ludelaire]

Hi Tom, here’s the image. I’m not using the .T operator, but the transpose_b=True parameter on matmul_qk.

[ludelaire]

Solved by using tf.cast(dk, tf.float32) instead of just dk on the line for scaling matmul_qk (the one where one implements the formula on which dk is on a sqrt operator).

Don’t understand what the traceback had to do with that, tho