Hi all, I’m getting the following error for the Encoder model
I understand that the enc_layer function has multiple layers which must be iterated over.
Clearly the issue is that I’m passing the parameters in the wrong order.
The issue I’m having is that I can’t seem to tell if the problem is with line 62 to UNQ_C5 or a relic from an earlier function call (i.e the ‘mha’ function call in line 44)
in call(self, x, training, mask)
60 # Pass the output through the stack of encoding layers
61 for i in range(self.num_layers):
—> 62 x = self.enc_layers[i](x, training = True, mask = True)
63 # END CODE HERE
64
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
1010 with autocast_variable.enable_auto_cast_variables(
1011 self._compute_dtype_object):
→ 1012 outputs = call_fn(inputs, *args, **kwargs)
1013
1014 if self._activity_regularizer:
in call(self, x, training, mask)
39 # START CODE HERE
40 # calculate self-attention using mha(~1 line). Dropout will be applied during training
—> 41 attn_output = self.mha(x, x, x, attention_mask = mask) # Self attention (batch_size, input_seq_len, fully_connected_dim)
42
43 # apply layer normalization on sum of the input and the attention output to get the
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
1010 with autocast_variable.enable_auto_cast_variables(
1011 self._compute_dtype_object):
→ 1012 outputs = call_fn(inputs, *args, **kwargs)
1013
1014 if self._activity_regularizer:
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/multi_head_attention.py in call(self, query, value, key, attention_mask, return_attention_scores, training)
472
473 attention_output, attention_scores = self._compute_attention(
→ 474 query, key, value, attention_mask, training)
475 attention_output = self._output_dense(attention_output)
476
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/multi_head_attention.py in _compute_attention(self, query, key, value, attention_mask, training)
436 query)
437
→ 438 attention_scores = self._masked_softmax(attention_scores, attention_mask)
439
440 # This is actually dropping out entire tokens to attend to, which might
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/multi_head_attention.py in _masked_softmax(self, attention_scores, attention_mask)
396 # (<batch_dims>, num_heads, <query_attention_dims, key_attention_dims>)
397 mask_expansion_axes = [-len(self._attention_axes) * 2 - 1]
→ 398 for _ in range(len(attention_scores.shape) - len(attention_mask.shape)):
399 attention_mask = array_ops.expand_dims(
400 attention_mask, axis=mask_expansion_axes)
AttributeError: ‘bool’ object has no attribute ‘shape’