In the positional_encoding function, it is used the get_angles function to initialize a matrix angle_rads. One of the arguments of get_angles is k, but I don’t understand how to define this variable.
Please move this topic to the right subcategory. Here’s the community user guide to get started.
As far as defining k is concerned, the embedding dimension, d is supplied as a parameter. See get_angles to notice k -- Row vector containing the dimension span [[0, 1, 2, ..., d-1]]. Have you seen numpy.arange ?
I’m not sure what I’m doing wrong here. I’m getting my final shape as (1,1,16) instead of (1,8,16), so there appears to be a problem with how I’m getting the angles, which means I’m probably doing something wrong when defining k.
I used the same formula that the unit test for get_angles uses (using np.arange and np.newaxis), but I guess that’s not working here?
Here’s some information about the shapes you should get when running positional_encoding_test:
angle_rads.shape = (8, 16)
pos_encoding.shape = (1, 8, 16)
Does this help?
No, I already knew that. My angle_rads.shape is (1,16), so clearly I’m not passing something correctly to the get_angles function, but as I said, I copied the code from the get_angles unit test, which uses d as a parameter, so I’m not sure what I’m doing wrong.
hello cheberling,
can you share the screenshot of the error, that gives lot of information about how to correct the code.
Regards
DP
Have you applied the correct angles code
Calculate the angles using pos, i and d
angles = ?
there is a note before the assignment cell for
Note: In the lectures Andrew uses vertical vectors, but in this assignment all vectors are horizontal. All matrix multiplications should be adjusted accordingly.
and then in next section apply all angles to angle_rads for initialising the matrix
See if this helps.
Otherwise, share the screenshot of your output, to know where you might be going wrong.
Regards
DP
My ex1 for get_angles passed all tests, so I assume it’s correct (?). Unless I’m supposed to transpose the output or similar to switch to horizontal vectors?
In the following screenshot, before the assertion error I’m printing the following:
pos, d, angle_rads, angle_rads.shape, pos_encoding.shape
there seems to be error for the previous cell to this positional_encoding test. Can you personal DM the code for that particular cell, to see where you might have gone wrong. Most probably, your angle_rads code has issue.
You can share the get angles code also.
Please don’t share your code here. Click on my name and click message to share the code.
Regards
DP
Hello Cheberling,
I was guessing right your angles and angel_rads for initialising matrix code is incorrect dear. Kindly do the needful.
-
Error in your get angles code: remove the extra two parentheses. the syntax does show the desired expression in the further expression, although it might sometimes show for that cell the desired output, it will cause error in the next forward cell.
-
The first code where you initialise the matrix for angle rads its mentioned for all angles, here the code need to be arrange for positional as well as dim. So once you call get_angles, first you need call function for np.arrange and then tuple positional and dim code .
If still unable to pass this cell, let me know.
Keep Learning!!!
Cheers
DP
You need to reply here, so that others in future can learn through such post.
Keep Learning!!
Regards
DP
I was able to pass the exercise after changing the call to get_angles. In addition to transforming ‘d’ with np.arange and np.newaxis in order to get the k parameter for the get_angles function, I also had to modify ‘positions’ with np.arange and np.newaxis. The unit test that the lab provides for exericse 1, get_angles, shows how to prepare the function parameters.
1 Like