Week 4 - Assignment 1 - Exercise 4

Course Q&A Deep Learning Specialization Sequence Models
1

Hi, I’m having a bit of a problem with the Encoding Layer exercise. I don’t really know how to call the functions. I think I’ve read through the text and don’t quite get it. For the mha I first thought that I should pass x.shape[0], x.shape[1],x.shape[2] and then after that (x) or some sort of version of that. The shape/value inside mha and then pass x. But that doesn’t seem to be working. I quess I would run in to the same problem with the other calls where there is a comment at the end of the line like

(batch_size, input_seq_len, embedding_dim)

Anybody have any idea of what is the right way to go here? For the dropout I’m just using training and the passing the output from prev. Should I also have add x and attn_output together before passing them as arguments and do the same when in the second layer normalization but just with different arguments? I just don’t know what we are supposed to pass as arguments. Any help or input would be appreciated.

2

I have the same issue

1 Like
3

Ok so I was googling about and found the solution pretty much. I was trying to google how to call the mha and didn’t find anything for some time until I did. I was always using the shape of x but you should just use x. Don’t know why and wasn’t looking for the solution. I was googling somehting like:

how to use multiheadattention custom layer examples

And eventually I got to a tutorial on the Keras page which has this exact function. Again wasn’t looking for that exact thing but rather how to call mha in general. But I found it and couldn’t unsee it. You can find it the same way, kvamvake, but hopefully someone can help explain why this is the case.