Hi,
So my problem is that the layers are not being added to the model summary in the order that I defined them. I have no idea of what can be going on.
this is how my labels look like [[‘InputLayer’, [(None, 64)], 0], [‘InputLayer’, [(None, 30, 37)], 0], [‘RepeatVector’, (None, 30, 64), 0, 30], [‘Bidirectional’, (None, 30, 64), 17920], [‘Concatenate’, (None, 30, 128), 0], [‘Dense’, (None, 30, 10), 1290, ‘tanh’], [‘Dense’, (None, 30, 1), 11, ‘relu’], [‘Activation’, (None, 30, 1), 0], [‘Dot’, (None, 1, 64), 0], [‘InputLayer’, [(None, 64)], 0], [‘LSTM’, [(None, 64), (None, 64), (None, 64)], 33024, [(None, 1, 64), (None, 64), (None, 64)], ‘tanh’], [‘Dense’, (None, 11), 715, ‘softmax’]]
It would be easier to read and interpret the output of the later cell
model.summary()
Here’s what it looks like in my notebook:
Model: "functional_5"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_3 (InputLayer) [(None, 30, 37)] 0
__________________________________________________________________________________________________
s0 (InputLayer) [(None, 64)] 0
__________________________________________________________________________________________________
bidirectional_2 (Bidirectional) (None, 30, 64) 17920 input_3[0][0]
__________________________________________________________________________________________________
repeat_vector (RepeatVector) (None, 30, 64) 0 s0[0][0]
lstm_2[10][1]
lstm_2[11][1]
lstm_2[12][1]
lstm_2[13][1]
lstm_2[14][1]
lstm_2[15][1]
lstm_2[16][1]
lstm_2[17][1]
lstm_2[18][1]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 30, 128) 0 bidirectional_2[0][0]
repeat_vector[20][0]
bidirectional_2[0][0]
repeat_vector[21][0]
bidirectional_2[0][0]
repeat_vector[22][0]
bidirectional_2[0][0]
repeat_vector[23][0]
bidirectional_2[0][0]
repeat_vector[24][0]
bidirectional_2[0][0]
repeat_vector[25][0]
bidirectional_2[0][0]
repeat_vector[26][0]
bidirectional_2[0][0]
repeat_vector[27][0]
bidirectional_2[0][0]
repeat_vector[28][0]
bidirectional_2[0][0]
repeat_vector[29][0]
__________________________________________________________________________________________________
dense (Dense) (None, 30, 10) 1290 concatenate[20][0]
concatenate[21][0]
concatenate[22][0]
concatenate[23][0]
concatenate[24][0]
concatenate[25][0]
concatenate[26][0]
concatenate[27][0]
concatenate[28][0]
concatenate[29][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 30, 1) 11 dense[20][0]
dense[21][0]
dense[22][0]
dense[23][0]
dense[24][0]
dense[25][0]
dense[26][0]
dense[27][0]
dense[28][0]
dense[29][0]
__________________________________________________________________________________________________
attention_weights (Activation) (None, 30, 1) 0 dense_1[20][0]
dense_1[21][0]
dense_1[22][0]
dense_1[23][0]
dense_1[24][0]
dense_1[25][0]
dense_1[26][0]
dense_1[27][0]
dense_1[28][0]
dense_1[29][0]
__________________________________________________________________________________________________
dot (Dot) (None, 1, 64) 0 attention_weights[20][0]
bidirectional_2[0][0]
attention_weights[21][0]
bidirectional_2[0][0]
attention_weights[22][0]
bidirectional_2[0][0]
attention_weights[23][0]
bidirectional_2[0][0]
attention_weights[24][0]
bidirectional_2[0][0]
attention_weights[25][0]
bidirectional_2[0][0]
attention_weights[26][0]
bidirectional_2[0][0]
attention_weights[27][0]
bidirectional_2[0][0]
attention_weights[28][0]
bidirectional_2[0][0]
attention_weights[29][0]
bidirectional_2[0][0]
__________________________________________________________________________________________________
c0 (InputLayer) [(None, 64)] 0
__________________________________________________________________________________________________
lstm_2 (LSTM) [(None, 64), (None, 33024 dot[20][0]
s0[0][0]
c0[0][0]
dot[21][0]
lstm_2[10][1]
lstm_2[10][2]
dot[22][0]
lstm_2[11][1]
lstm_2[11][2]
dot[23][0]
lstm_2[12][1]
lstm_2[12][2]
dot[24][0]
lstm_2[13][1]
lstm_2[13][2]
dot[25][0]
lstm_2[14][1]
lstm_2[14][2]
dot[26][0]
lstm_2[15][1]
lstm_2[15][2]
dot[27][0]
lstm_2[16][1]
lstm_2[16][2]
dot[28][0]
lstm_2[17][1]
lstm_2[17][2]
dot[29][0]
lstm_2[18][1]
lstm_2[18][2]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 11) 715 lstm_2[10][1]
lstm_2[11][1]
lstm_2[12][1]
lstm_2[13][1]
lstm_2[14][1]
lstm_2[15][1]
lstm_2[16][1]
lstm_2[17][1]
lstm_2[18][1]
lstm_2[19][1]
==================================================================================================
Total params: 52,960
Trainable params: 52,960
Non-trainable params: 0
I am sorry, how can I paste the text in a certain “box” as you did? Also, my model summary is Model:
Thanks, yes, that is easier to read. But I guess I don’t have an explanation for why your layers all seem to be in a different order. Maybe I need to actually look at your notebook. We shouldn’t do that on a public thread, but I’ll send you a DM about how to do that.
please!
To close the loop on the public thread, here are a couple of the comments from the template code for one_step_attention:
# Use concatenator to concatenate a and s_prev on the last axis (≈ 1 line)
# For grading purposes, please list 'a' first and 's_prev' second, in this order.
If you ignore that hint and use the reverse order of the arguments, your code is still logically correct, but the layers will come out in a different order which will cause the comparison tests to fail in the notebook.
2 Likes