Transformer NB get_angles equations broken?

Hi I don’t understand how on earth we are supposed to program the get_angles() function in the Transformer Notebook because it literally says:
"Hint […] If k = [0, 1, 2, 3, 4, 5] , then, i must be i = [0, 0, 1, 1, 2, 2]", which means that ‘i’ must be 0 twice. Then plug that into the equation “theta(pos, i, d) = pos/(10000*(2i/d))” (also quoted verbatim from instructions). Then you get infinities (or NaNs) at the 0s which causes it to fail the asserts in the unit tests.

Am I missing something here?

The instructions are very confusing.

The “hint” is intended to tell you that the ‘i’ values come from (edited) floor division of the k values.

So that converts in code into “i = k // 2”.

Then to compute “angles”, you use pos, np.power(), a constant of 10000, 2*i, and np.float32(d).

You’re only implementing this equation:
… where theta is the “angles” variable.

1 Like

Actually your implementation is wrong, (2i/d) is an exponent of 10000, not a product.


Oh I see that appears to be the root of the problem, I can tell now the (2i/d) is ever so slightly elevated but not obviously enough for me to realize it was a power.


P.S. ‘//’ is just truncated division not modulo (modulo is ‘%’)

Anyhow thanks for your answer!! No idea how long that would’ve taken me to figure out lol

Thanks for pointing out my incorrect term, “modulo” was the first word that popped into my head. I’ll update my reply.

1 Like