C5 W4 exercise 1.2

Course Q&A Deep Learning Specialization Sequence Models
1

Hi, I know there are a lot of posts on the programming exercise of week 4, course 5 being sort of uphill. I already read this blog on transformers for a better understanding of positional embeddings, and also checked the optional notebooks of that same week on these topics as suggested on several other posts, and although their implementation of the ‘positional_encoding’ and ‘get_angles’ functions do the same as mine, I went as far as literally copying their code and pasting it, and I’m still getting errors or not passing the test for ‘positional_encoding’.

After reading several posts and almost all their comments, it seems my implementation of ‘positional_encoding’ is correct, as it is exactly similar to the one pointed out several times by many people, which is just applying np.sin and np.cos to slices of the angles array. So it seems the problem is with ‘get_angles’. I already tried explicitly casting the type of d to float32, printing the shapes of all the different parameters to make sure everything is alright, and again, copying the implementation from the optional notebooks, and everything seems to fail. I literally have no idea what else to do, and I have being stuck in this function for 3 days know, I’m going crazy, pls halp.

The error prompt includes a small slice of my code, sorry for that, but I literally am desperate here, and this very simple implementation is pointed out in several places by many people, so I really hope this doesn’t count as a violation to the honor code. PS: Although I’m 99% confident right know that the problem is not with ‘positional_encoding’ but with ‘get_angles’, I know pasting this error might not be too relevant for the later, but ‘get_angles’ passes the test and I cannot paste my implantation, so I don’t know how else can someone possibly help me.

IndexError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 positional_encoding_test(positional_encoding, get_angles)

~/work/W4A1/public_tests.py in positional_encoding_test(target, get_angles)
26 d_model = 16
27
—> 28 pos_encoding = target(position, d_model)
29 sin_part = pos_encoding[:, :, 0::2]
30 cos_part = pos_encoding[:, :, 1::2]

in positional_encoding(positions, d)
19
20 # apply sin to even indices in the array; 2i
—> 21 angle_rads[:, 0::2] = np.sin(angle_rads[:, 0::2])
22
23 # apply cos to odd indices in the array; 2i+1

IndexError: invalid index to scalar variable.

1 Like
2

Hope this shape information helps with positional_encoding:

angle_rads.shape = (8, 16)
pos_encoding.shape = (1, 8, 16)
1 Like
3

I agree, the issue is probably in get_angles().
The only trick in get_angles() is this line of code:

    # Get i from dimension span k
    i = k // 2
1 Like
4

you mean I should, or should not be using this operation? Because right now I am using it (i = k //2) and it is still giving me errors. I also tried to check the shape of the angle_rads array inside the positional_encoding function as suggested in the other answer provided, but it gives me just “()”, I guess a scalar, I don’t understand why. Is there a way in which you can possibly see my assignment as to receive a more specific feedback on what my error might be? I really don’t see where it might be coming from, as again, the tests are being passed inside get_angles(). Thanks again for the help.

5

Please check your private messages.