C5_W4_A1_Transformer Decoder

I get the following error due to the shape of x. Although I printed the shape of it along the code, it seems that from the very beginning of block1, x is already a tensor with 6 and not 5 values. Could you suggest me what I could check in the code?


tf.Tensor(
[[3 2 1]
 [2 1 0]], shape=(2, 3), dtype=int64)

InvalidArgumentError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 Decoder_test(Decoder, create_look_ahead_mask, create_padding_mask)

~/work/W4A1/public_tests.py in Decoder_test(target, create_look_ahead_mask, create_padding_mask)
234 assert np.allclose(outd[1, 1], [-0.30814743, -0.6213016, -0.77767026, 1.7071193]), “Wrong values in outd when training=True”
235
→ 236 outd, att_weights = decoderk(x, encoderq_output, True, look_ahead_mask, create_padding_mask(x))
237 assert np.allclose(outd[1, 1], [-0.0250004, 0.50791883, -1.5877104, 1.1047921]), “Wrong values in outd when training=True and use padding mask”
238

in create_padding_mask(decoder_token_ids)
10 “”"
11 seq = 1 - tf.cast(tf.math.equal(decoder_token_ids, 0), tf.float32)
—> 12 seq = tf.reshape(seq,(-1,5))
13
14 # add extra dimensions to add the padding

/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 “”“Call target, and fall back on dispatchers if there is a TypeError.”“”
200 try:
→ 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a

/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py in reshape(tensor, shape, name)
193 A Tensor. Has the same type as tensor.
194 “”"
→ 195 result = gen_array_ops.reshape(tensor, shape, name)
196 tensor_util.maybe_set_static_shape(result, shape)
197 return result

/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py in reshape(tensor, shape, name)
8371 try:
8372 return reshape_eager_fallback(
→ 8373 tensor, shape, name=name, ctx=_ctx)
8374 except _core._SymbolicException:
8375 pass # Add nodes to the TensorFlow graph.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py in reshape_eager_fallback(tensor, shape, name, ctx)
8396 _attrs = (“T”, _attr_T, “Tshape”, _attr_Tshape)
8397 _result = _execute.execute(b"Reshape", 1, inputs=_inputs_flat, attrs=_attrs,
→ 8398 ctx=ctx, name=name)
8399 if _execute.must_record_gradient():
8400 _execute.record_gradient(

/opt/conda/lib/python3.7/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
—> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:

InvalidArgumentError: Input to reshape is a tensor with 6 values, but the requested shape requires a multiple of 5 [Op:Reshape]

I am curious about this line of code in your create_padding_mask() function:
image

That line isn’t in my copy of the notebook. And it isn’t in the official assignment repo. That function should not be edited by students.

Did you add that code yourself?

I don’t know what I was thinking when I added that.

Thank you for noticing this so fast! :slight_smile:

I would guess you were trying to force a fix to some other error.

I wish that was the reason, but it’s totally worse than that. :sweat_smile: