Has anyone successfully completed this assignment recently? My code works but is coming up with numbers slightly different than the validation numbers. Just want assurance this is solvable. Thanks.
Hello Brian,
can you please be more clear about what assignment you are referring with how slight different than expected output is? So mentors can guide you better.
Probably you can share screenshot of you training model result and the expected validation numbers has to achieve!!!
Regards
DP
There’s only one programming assignment in DLS C5 W4. It’s the Transformer lab.
@Brian_Conte, I’ll re-run a fresh copy of the lab and report back. It was last updated in May 2023, but I haven’t run the new version from scratch yet.
What validation numbers are you referring to specifically? This notebook doesn’t really have a lot of “expected output” cells.
Please post screen capture images.
For this reason only I had asked him to mention assignment name!!!
REGARDS
DP
After May 2023 (I tried it in July 2023) this was working just fine. Was there any recent update?
The assignment is “Transformers Architecture with Tensorflow”, Exercise 4, in the implementation of EmbeddingLayer.
The numbers it expects are
assert np.allclose(encoded.numpy(),
[[-0.5214877 , -1.001476 , -0.12321664, 1.6461804 ],
[-1.3114998 , 1.2167752 , -0.5830886 , 0.6778133 ],
[ 0.25485858, 0.3776546 , -1.6564771 , 1.023964 ]],), "Wrong values"
However the numbers my functions returns are:
[[[ 0.23017097 -0.9810039 -0.78707564 1.5379086 ]
[-1.2280797 0.76477575 -0.7169284 1.1802323 ]
[ 0.14880148 -0.4831803 -1.1908401 1.5252188 ]]]
(I guess they are not so similar)
The exact error is:
AssertionError Traceback (most recent call last)
in
20
21 print(“\033[92mAll tests passed”)
—> 22 EncoderLayer_test(EncoderLayer)
in EncoderLayer_test(target)
13 [[-0.5214877 , -1.001476 , -0.12321664, 1.6461804 ],
14 [-1.3114998 , 1.2167752 , -0.5830886 , 0.6778133 ],
—> 15 [ 0.25485858, 0.3776546 , -1.6564771 , 1.023964 ]],), “Wrong values”
16 # assert np.allclose(encoded.numpy(),
17 # [[-0.4218296, -0.86448276, -0.41689685, 1.7032092 ],
AssertionError: Wrong values
Just now, while debugging, I get a new error:
InvalidArgumentError Traceback (most recent call last)
in
20
21 print(“\033[92mAll tests passed”)
—> 22 EncoderLayer_test(EncoderLayer)
in EncoderLayer_test(target)
4 encoder_layer1 = EncoderLayer(4, 2, 8)
5 tf.random.set_seed(10)
----> 6 encoded = encoder_layer1(q, True, np.array([[1, 0, 1]]))
7
8 assert tf.is_tensor(encoded), “Wrong type. Output must be a tensor”
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
1010 with autocast_variable.enable_auto_cast_variables(
1011 self._compute_dtype_object):
→ 1012 outputs = call_fn(inputs, *args, **kwargs)
1013
1014 if self._activity_regularizer:
in call(self, x, training, mask)
66 ffn_output = self.ffn(mult_attn_out)
67 ffn_output = self.dropout2(ffn_output, training=training)
—> 68 encoder_layer_out = self.layernorm2(mult_attn_out + ffn_output)
69 # self_mha_output = self.mha(x, x, x, mask)
70
/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/math_ops.py in binary_op_wrapper(x, y)
1162 with ops.name_scope(None, op_name, [x, y]) as name:
1163 try:
→ 1164 return func(x, y, name=name)
1165 except (TypeError, ValueError) as e:
1166 # Even if dispatching the op failed, the RHS may be a tensor aware
/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 “”“Call target, and fall back on dispatchers if there is a TypeError.”“”
200 try:
→ 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a
/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/math_ops.py in _add_dispatch(x, y, name)
1484 return gen_math_ops.add(x, y, name=name)
1485 else:
→ 1486 return gen_math_ops.add_v2(x, y, name=name)
1487
1488
/opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/gen_math_ops.py in add_v2(x, y, name)
470 return _result
471 except _core._NotOkStatusException as e:
→ 472 _ops.raise_from_not_ok_status(e, name)
473 except _core._FallbackException:
474 pass
/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
6860 message = e.message + (" name: " + name if name is not None else “”)
6861 # pylint: disable=protected-access
→ 6862 six.raise_from(core._status_to_exception(e.code, message), None)
6863 # pylint: enable=protected-access
6864
/opt/conda/lib/python3.7/site-packages/six.py in raise_from(value, from_value)
InvalidArgumentError: Incompatible shapes: [1,3,4] vs. [1,3,8] [Op:AddV2]
Any insight will be appreciated.
In the notebook, Exercise 4 is the Encoder Layer (not EmbeddingLayer).
I don’t see the values you cite in the EncoderLayer_test() function. Maybe you have an obsolete copy of public_tests.py.
Here is what the test looks like and the values tested by the assert:
Your return values seem to be consistent with the expected values that are in the current version of public_tests.py.
Hello Brian,
YOU HAVE INCLUDED MULTI_HEAD_ATTENTION OUTPUT BUT YOU ARE SUPPOSE TO USE SKIP CONNECTION WITH FFN OUTPUT
Regards
DP
Thank you!
issue resolved?
Unfortunately not. Any other ideas?
P.S. This is how I updated python_tests.whl
C:\Users\brian\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages>pip download --only-binary :all: --dest . --no-cache python_tests.whl
Why?
I was asked to above.
OK. I looked quickly, but didn’t see it.
If possible send this grader cell codes via personal DM. Click on my name and then message.
Hello Brian,
Your grader cell looks totally mixed.
Kindly re do this grader cell
your grader cell looks edited outside of ###START CODE HERE and #END CODE HERE
the grader cell for me looks like this
Please avoid copy-pasting any of the grader cells.
You are only suppose to write code between ###START CODE HERE and ###END CODE HERE###
See this is what you have sent with the codes you have written 
# # START CODE HERE
# # Pass input through the Embedding layer
# x = self.embedding(x) # (batch_size, input_seq_len, embedding_dim)
# # Scale embedding by multiplying it by the square root of the embedding dimension
# x *= tf.math.sqrt(tf.cast(self.embedding_dim,tf.float32))
# # Add the position encoding to embedding
# x += self.pos_encoding[:, :seq_len, :]
# x = self.dropout(x, training=training)
# # Pass the encoded embedding through a dropout layer
# # use `training=training`
# x = self.dropout(x, training=training)
# # Pass the output through the stack of encoding layers
# for i in range(self.num_layers):
# print(" x = ", x)
# print(" training = ", training)
# print(" mask = ", mask)
# x = self.enc_layers[i](x,training, mask)
# # END CODE HERE
# return x # (batch_size, input_seq_len, embedding_dim)
Regards
DP
hi, I have updated it so that only the code between start code and end code is modified. I still get the wrong values error. here is the updated grader cell. Thanks again for your help!
{moderator edit: code removed - not allowed by the Code of Conduct}
Removing this post. don’t post codes here. The post comment I put doesn’t have codes inside the grader cell. It was only mentioned where you have added it inside ##START CODE HERE and ###END CODE HERE##
So remove it.
Your encoder layer out code is still incorrect.
Why have you left skip_x_attention=None
# # skip_x_attention = None # (batch_size, input_seq_len, fully_connected_dim)
you need to write code for that line.
Do the correction and send me the codes via personal DM. Don’t post codes here.
Regards
DP