Attention modelf

Course Q&A Deep Learning Specialization Sequence Models
1

Kindly help my code in week 3 assignment for attention Modelf is returning the following error;

ValueError Traceback (most recent call last)
in
34
35
—> 36 modelf_test(modelf)

in modelf_test(target)
11
12
—> 13 model = target(Tx, Ty, n_a, n_s, len_human_vocab, len_machine_vocab)
14
15 print(summary(model))

in modelf(Tx, Ty, n_a, n_s, human_vocab_size, machine_vocab_size)
37
38 # Step 2.A: Perform one step of the attention mechanism to get back the context vector at step t (≈ 1 line)
—> 39 context = one_step_attention(a[t],s[t-1])
40
41 # Step 2.B: Apply the post-attention LSTM cell to the “context” vector.

in one_step_attention(a, s_prev)
17 ### START CODE HERE ###
18 # Use repeator to repeat s_prev to be of shape (m, Tx, n_s) so that you can concatenate it with all hidden states “a” (≈ 1 line)
—> 19 s_prev = repeator(s_prev)
20 # Use concatenator to concatenate a and s_prev on the last axis (≈ 1 line)
21 # For grading purposes, please list ‘a’ first and ‘s_prev’ second, in this order.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
924 if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
925 return self._functional_construction_call(inputs, args, kwargs,
→ 926 input_list)
927
928 # Maintains info about the Layer.call stack.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in _functional_construction_call(self, inputs, args, kwargs, input_list)
1090 # TODO(reedwm): We should assert input compatibility after the inputs
1091 # are casted, not before.
→ 1092 input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
1093 graph = backend.get_graph()
1094 # Use self._name_scope() to avoid auto-incrementing the name.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
178 ‘expected ndim=’ + str(spec.ndim) + ‘, found ndim=’ +
179 str(ndim) + '. Full shape received: ’ +
→ 180 str(x.shape.as_list()))
181 if spec.max_ndim is not None:
182 ndim = x.shape.ndims

ValueError: Input 0 of layer repeat_vector is incompatible with the layer: expected ndim=2, found ndim=1. Full shape received: [64]

2

Hi @Igba

The parameters passed to one_step_attention() are incorrect. It should be just a, output of the bi-directional LSTM, and s, previous state.
a is sequence of hidden states, and s is initialised to s0, and t is the timestep. At each time step, t, until Ty, the same sequence of a is used, but s is updated after calling post_activation_LSTM_cell().

1 Like
3

Thanks sir, but i am still getting same error. I sent you a private message of my code to vet please.

4

Hi @Igba

Two problems with your call to Bidirectional LSTM:

  1. wrong input units, it should be : units=n_a , not 1,
    because n_a – hidden state size of the Bi-LSTM (see comments)
  2. input shape is missing, it should be : input_shape=(m,Tx,n_a*2)
5

Thanks sir, I did this : a = Bidirectional(LSTM(units= n_a, return_sequences= True),input_shape=(m,Tx,n_a*2))

but i still got this : AttributeError: ‘Bidirectional’ object has no attribute ‘shape’

1 Like
6

you missed the (X) at the end.

7

Many thanks sir, it worked.

1 Like