Exercise 2. The expected model is different than mine

Course Q&A Deep Learning Specialization Sequence Models
1

the expected summary is

expected_summary = [[‘InputLayer’, [(None, 30, 37)], 0],
[‘InputLayer’, [(None, 64)], 0],
[‘Bidirectional’, (None, 30, 64), 17920],
[‘RepeatVector’, (None, 30, 64), 0, 30],
[‘Concatenate’, (None, 30, 128), 0],
[‘Dense’, (None, 30, 10), 1290, ‘tanh’],
[‘Dense’, (None, 30, 1), 11, ‘relu’],
[‘Activation’, (None, 30, 1), 0],
[‘Dot’, (None, 1, 64), 0],
[‘InputLayer’, [(None, 64)], 0],
[‘LSTM’,[(None, 64), (None, 64), (None, 64)], 33024,[(None, 1, 64), (None, 64), (None, 64)],‘tanh’],
[‘Dense’, (None, 11), 715, ‘softmax’]]

I get summary

[[‘InputLayer’, [(None, 30, 37)], 0], [‘Bidirectional’, (None, 30, 64), 17920], [‘Dense’, (None, 30, 1), 65, ‘relu’], [‘Activation’, (None, 30, 1), 0], [‘Dot’, (None, 1, 64), 0], [‘InputLayer’, [(None, 64)], 0], [‘InputLayer’, [(None, 64)], 0], [‘LSTM’, [(None, 64), (None, 64), (None, 64)], 33024, [(None, 1, 64), (None, 64), (None, 64)], ‘tanh’], [‘Dense’, (None, 11), 715, ‘softmax’]]

Where could be my mistake?

2

Hello @Dmitry_Regent,

I find Chrome’s search feature a nice way to highlight some keywords for comparison. You may do it for each of the elements to find what are missing. For example, “RepeatedVector” is not among your summary.

image
image516×710 121 KB

If you are not sure what “RepeatedVector” is, you can search for it in the lab and it was initated and used under the section for Exercise 1.

“RepeatedVector” is used in a function which is needed to be put into Exercise 2 and be assigned to a variable. Then, make sure that variable is used by another layer, otherwise, the associated “RepeatedVector” won’t show up in the summary. Tensorflow only puts things that are used into the summary.

Cheers,
Raymond

3

@Dmitry_Regent, your summary is also missing one of the Input layers.

Also, note that where it says this is the instructions…
image

… it is referring to the helper function you’ve been given earlier:

image
image1012×189 10.6 KB

Similarly, your code should also be using that post_activation_LSTM_cell() helper function.

Reading the instructions carefully is very important. Everything mentioned in the instructions is there for a reason.

4

Hi, Of cause I finished exercise 1 there and and “All tests passed!”. and I used one_step_attention(a, s_prev) function in exercise 2 as a first action in a loop

5

sorry I don’t understand, this was declared in cell 10

output_layer = Dense(len(machine_vocab), activation=softmax)

so I just used this layer in

Step 2.C: Apply Dense layer to the hidden state output of the post-attention LSTM (≈ 1 line)

    out = output_layer(s)

I thought I totally followed all the instructions… this exercises make me think some times and read work with ML is easier than passing this labs :slight_smile:

6

Code comments are really straightforward there. I just can’t understand where there should be an issue…

{mentor edit: code removed}

8

Update for those who find this thread later:

An undetected defect in one_step_attention() can cause the model to have the
wrong number of layers.

1 Like