C5 W3 A1 Neural machine translation, got 100 points but model performing weirdly

Course Q&A Deep Learning Specialization Sequence Models
1

In this notebook, I passed all test cases, and the grader gave 100 points. However, in the section where I load the pre-trained model and test with some test cases, i.e., where you do model.load_weights('models/model.h5') and EXAMPLES =
I get very wrong results:

source: 3 May 1979
output: 1111111111 

source: 5 April 09
output: 2222222222 

source: 21th of August 2016
output: 2222222222 

source: Tue 10 Jul 2007
output: 2222222222 

source: Saturday May 9 2018
output: 2222222222 

source: March 3 2001
output: 2222222222 

source: March 3rd 2001
output: 2222222222 

source: 1 March 2001
output: 2222222222

I wasn’t able to find any other posts with the same issue. So I’m wondering what could be wrong?

Lab ID is mnvpyquslsce in case that’s helpful

2

I am not familiar with this but what are the examples you are feeding?

3

Here are the correct answers for that section:

source: 3 May 1979
output: 1979-05-33 

source: 5 April 09
output: 2009-04-05 

source: 21th of August 2016
output: 2016-08-20 

source: Tue 10 Jul 2007
output: 2007-07-10 

source: Saturday May 9 2018
output: 2018-05-09 

source: March 3 2001
output: 2001-03-03 

source: March 3rd 2001
output: 2001-03-03 

source: 1 March 2001
output: 2001-03-01

So it looks like your logic has simply propagated the first digit of the year for the entire string. How could that happen? It is puzzling that the grader still gives a full score with that output. It might be worth looking at your code to understand what happened and if there is a way to improve the grader to catch your bug. We can’t directly see your notebook by using the ID. Please check your DMs for a message from me about how to proceed.

1 Like
4

Interestingly, here’s another thread from a couple of weeks ago that has the exact same issue, but I don’t think that one had any resolution.

5

Yeah I did not have a solution to this yet. I can Share the Id of assignment if needed.

6

We can’t directly look at your notebook, but please check your DMs for a message from me about how to do this.

7

It’s the default input examples in the ipynb.

8

Figured this one out, thanks to @paulinpaloalto ! It was a small code error in my notebook that wasn’t caught by the grader

1 Like
9

Since we’ve seen this exact syndrome in the outputs shown above in other threads, it’s likely that other people have made the same mistake. Of course we can’t write out the solution code here, but maybe we can describe the bug in a way that will be enough of a hint.

Be careful when you use the post attention LSTM cell in the model. The initial_state on each iteration is the “cell state” values from the previous iteration, not the actual initial values from iteration 0. This is always the way RNN cells (plain RNN, GRU or LSTM) work: the state evolves on every iteration.

1 Like
10

Is there any way to improve the test cases so this defect is identified?

11

They would need to run the model in the grader and check the outputs. I know they have done that in other instances, but apparently they don’t here.

Or at the least, they could make the example predictions here in the notebook into a test, instead of just showing the generated output.

I was planning to file a bug about this, but might not get to it tonight …