# C5W4: Transformer Architectures with TensorFlow **URL:** \ **Category:** Sequence Models\ **Tags:** week-module-4, coursera-platform\ **Created:** [June 11, 2021, 11:18pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545 "2021-06-11T23:18:35Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![falonso](https://avatars.discourse-cdn.com/v4/letter/f/d2c977/32.png) [@falonso](https://community.deeplearning.ai/u/falonso)\ **Post date:** [June 11, 2021, 11:18pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/1 "2021-06-11T23:18:35Z")
The last assignment of course 5 week 4 was really a disaster. Nothing is understood, it is not well explained. It is much more difficult than the previous tasks. I ended up copying from the trasnformers tutorial from the tensor flow page (without being capable of understanding ). PLEASE IMPROVE IT! ---
**Author:** ![arosacastillo](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/arosacastillo/32/159244_2.png) [@arosacastillo](https://community.deeplearning.ai/u/arosacastillo)\ **Post date:** [June 12, 2021, 10:58am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/2 "2021-06-12T10:58:43Z")
Hi Falonso, Sorry to read your frustration. The Transformers topic is indeed complex but at the same time very interesting and a hot field right now at the NN community. Congrats on having finished the assignment even with the difficulties you are mentioning. I believe the Coursera team is trying to improve every version of the notebook thanks to the feedback they receive, so I encourage you to pass a specific improvement note to them, saying for instance the number of the exercises that you consider not well explained and which are the useful tips they should include. Whichever worked for you, can work for other students as well, so your opinion is valuable. Try to send the feedback from the help center by email. Happy learning, Rosa ---
**Author:** ![MuskaanManocha](https://avatars.discourse-cdn.com/v4/letter/m/3da27b/32.png) [@MuskaanManocha](https://community.deeplearning.ai/u/MuskaanManocha)\ **Post date:** [June 12, 2021, 11:26am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/3 "2021-06-12T11:26:18Z")
I feel better after reading this. That it is indeed a bit complex. But rightly added in the end of the full specialization šŸ™‚ ---
**Author:** ![GordonRobinson](https://avatars.discourse-cdn.com/v4/letter/g/f19dbf/32.png) [@GordonRobinson](https://community.deeplearning.ai/u/GordonRobinson)\ **Post date:** [June 13, 2021, 7:58pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/4 "2021-06-13T19:58:53Z")
To add a little to Arosaā€™s comment. You are not the first to criticize this assignment! A few weeks ago there was a Zoom meeting between the course staff and mentors that was largely initiated by that criticism. Some of us provided detailed feedback to the course staff about our personal views of the assignment (and week 4 lectures). The staff are listening and and are discussing the issues with the instructors. In my personal comments I also used phrases similar to your ā€œnothing understoodā€, so I know how you feel. If there are specifics about what you donā€™t understand, please share them so that the staff and instructors can get even more views about how it could be improved. ---
**Author:** ![steven.j36](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@steven.j36](https://community.deeplearning.ai/u/steven.j36)\ **Post date:** [August 20, 2021, 2:20am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/5 "2021-08-20T02:20:00Z")
I pretty much agree with this sentiment. Why did it switch to object-oriented programming? I was still learning to use tensorflow layers. ---
**Author:** ![Subho](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/subho/32/5046_2.png) [@Subho](https://community.deeplearning.ai/u/Subho)\ **Post date:** [August 28, 2021, 10:39pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/6 "2021-08-28T22:39:49Z")
Yes the last assignment is really a disaster. ---
**Author:** ![Mrtranducdung](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@Mrtranducdung](https://community.deeplearning.ai/u/Mrtranducdung)\ **Post date:** [September 21, 2021, 4:43am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/7 "2021-09-21T04:43:59Z")
Dear @arosacastillo , I have passed all the test, but when I summit, I can not pass the assignment. The message is like this: Cell #18. Canā€™t compile the studentā€™s code. Error: AssertionError(ā€˜Wrong values case 1ā€™) Could you please help me to solve the problem? ---
**Author:** ![arosacastillo](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/arosacastillo/32/159244_2.png) [@arosacastillo](https://community.deeplearning.ai/u/arosacastillo)\ **Post date:** [September 21, 2021, 8:11am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/8 "2021-09-21T08:11:04Z")
Hi Mrtranducdung, Please have a read to the solutions proposed here: > [@C5 W4 A1: Can't compile student's code (old thread, please don't add new replies)](https://community.deeplearning.ai/t/cant-compile-students-code/37161): > > Hello Iā€™m doing the final assignment of course 5 week 4 and getting recurring errors, despite the fact that it says ā€œAll assignments passedā€ when I submit. I know that this is possible in certain cases. But for me, it seems to be happening in ALL cases (I complete a function, get ā€œAll assignments passedā€, submit and then get a different reason for ā€œcanā€™t compile studentā€™s codeā€) In the picture, you can see a history of some of the errors. The most recent one happened after my attempt to get ridā€¦ > [@Compile student's code](https://community.deeplearning.ai/t/compile-students-code/25778): > > Hello there, I have common ā€œcompile studentā€ problem and I did reset and clear output and reset my note booked and rewrote thatā€¦ but I still have the problemā€¦ any help please. course 4, week 2, HW 2 > [@C5W2A2: Cell #14. Can't compile the student's code. Error: ValueError('shapes (5,2) and (5,) not aligned: 2 (dim 1) != 5 (dim 0)',)](https://community.deeplearning.ai/t/c5w2a2-cell-14-cant-compile-the-students-code-error-valueerror-shapes-5-2-and-5-not-aligned-2-dim-1-5-dim-0/22798/3): > > are you solve this proplem? It is always a good practice to do a search in the forum if people have posted previously similar errors. I found many solutions on my own as a student this way šŸ™‚ Happy learning Rosa ---
**Author:** ![dheeraj5](https://avatars.discourse-cdn.com/v4/letter/d/ecccb3/32.png) [@dheeraj5](https://community.deeplearning.ai/u/dheeraj5)\ **Post date:** [September 24, 2021, 11:38am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/9 "2021-09-24T11:38:02Z")
Hi Dheeraj here. Really feeling frustated. Completed the whole course in full flow. But in Transformer part i am unable to submit the Answers. And it grader show me the 0/100 so please help me to clear the Assessment. ---
**Author:** ![canonv](https://avatars.discourse-cdn.com/v4/letter/c/94ad74/32.png) [@canonv](https://community.deeplearning.ai/u/canonv)\ **Post date:** [October 5, 2021, 5:33am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/10 "2021-10-05T05:33:50Z")
Same frustration here. I finished all other homework by myself, except this one. ---
**Author:** ![mcsmrx](https://avatars.discourse-cdn.com/v4/letter/m/35a633/32.png) [@mcsmrx](https://community.deeplearning.ai/u/mcsmrx)\ **Post date:** [October 13, 2021, 4:43pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/11 "2021-10-13T16:43:40Z")
I am stuck in scaled\_dot\_product\_attention\_test with an exception: AssertionError: Wrong masked weights No idea how to proceed. ---
**Author:** ![Russel\_Crowe](https://avatars.discourse-cdn.com/v4/letter/r/439d5e/32.png) [@Russel\_Crowe](https://community.deeplearning.ai/u/Russel_Crowe)\ **Post date:** [October 14, 2021, 11:33am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/12 "2021-10-14T11:33:10Z")
I got the same error message. I found out that I forgot to subtract mask from 1 before multiplying it. When I had it like this: scaled\_attention\_logits += (1-mask) \* -1.0e9 ā€¦ things worked out. I hope this was helpful! ---
**Author:** ![mcsmrx](https://avatars.discourse-cdn.com/v4/letter/m/35a633/32.png) [@mcsmrx](https://community.deeplearning.ai/u/mcsmrx)\ **Post date:** [October 15, 2021, 3:41am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/13 "2021-10-15T03:41:29Z")
Yes it works thanks!. ---
**Author:** ![mcsmrx](https://avatars.discourse-cdn.com/v4/letter/m/35a633/32.png) [@mcsmrx](https://community.deeplearning.ai/u/mcsmrx)\ **Post date:** [October 16, 2021, 6:06pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/14 "2021-10-16T18:06:13Z")
Thanks, that was helpful and I finished the whole assignment after that. ---
**Author:** ![fabio.borges](https://avatars.discourse-cdn.com/v4/letter/f/9de053/32.png) [@fabio.borges](https://community.deeplearning.ai/u/fabio.borges)\ **Post date:** [February 6, 2022, 11:58pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/15 "2022-02-06T23:58:14Z")
Hi, I understand conceptually what the Q,K and V matrices are, however at this point in the encoder code: # calculate self-attention using mha(~1 line). Dropout will be applied during training The call requires the matrices q, k and v as arguments, but itā€™s not clear where those matrices come from. Since the comment says ā€œ~1 lineā€, I would expect them to be readily available, but where are they? Honestly, this is not a course on TensorFlow/Keras, there are other courses for those topics. Although I have consulted TFā€™s docs several times in this specialization, I donā€™t think I should be required to dig deep into TF to finish the assignments. I should just use it as an aid. So I would appreciate if someone could please tell me what the heck do those three matrices come from, so I can pass them to the mha() call. Again, this is not a conceptual question, Itā€™s more about the ā€œtechnicalitiesā€ of the implementation, which I should not be required to know. Thanks a lot! ---
**Author:** ![Mhmemeth](https://avatars.discourse-cdn.com/v4/letter/m/e79b87/32.png) [@Mhmemeth](https://community.deeplearning.ai/u/Mhmemeth)\ **Post date:** [March 30, 2022, 6:10pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/16 "2022-03-30T18:10:25Z")
Likewise. Thanks for the help! ---
**Author:** ![Thomas\_Vermaelen](https://avatars.discourse-cdn.com/v4/letter/t/8e8cbc/32.png) [@Thomas\_Vermaelen](https://community.deeplearning.ai/u/Thomas_Vermaelen)\ **Post date:** [April 6, 2022, 12:47pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/17 "2022-04-06T12:47:46Z")
> [@Russel\_Crowe](#): > > (1-mask) \* -1.0e9 a mha object has already been instantiated within the EncoderLayer class, in ā€œdef **init** (ā€¦)ā€, which already passes the q, k and v matrices. Within the the call() method, you must call mha using self.mha(), and pass in the input ā€œxā€ I believe, as such : self.mha(x) ---
**Author:** ![anshbhatnagar007](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@anshbhatnagar007](https://community.deeplearning.ai/u/anshbhatnagar007)\ **Post date:** [April 19, 2022, 2:28am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/18 "2022-04-19T02:28:03Z")
yes, real this assignment is way out of the scope just start to learn TensorFlow and then add the concept of OOP in that and the video also did not help that much in the assignment ---
**Author:** ![Yuchen\_Zheng](https://avatars.discourse-cdn.com/v4/letter/y/c5a1d2/32.png) [@Yuchen\_Zheng](https://community.deeplearning.ai/u/Yuchen_Zheng)\ **Post date:** [June 30, 2022, 11:42am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/19 "2022-06-30T11:42:11Z")
Hi everyone, does anyone meet this problem as below: Cell #16. Canā€™t compile the studentā€™s code. Error: AssertionError(ā€˜Wrong type. Output must be a tensorā€™) ![image](https://global.discourse-cdn.com/dlai/original/3X/b/8/b82826af1b61e3c8b601296c0cbc7455e3543757.png) This happens at the last cell in Exercise 3 - scaled\_dot\_product\_attention. But I aslo find that the NameError: name ā€˜scaled\_dot\_product\_attention\_testā€™ is not defined ![image](https://global.discourse-cdn.com/dlai/original/3X/e/d/ed34659bb2eacde3b7880f1353c6701fe2a337c8.png) Could anyone help to solve this problem? Thanks a lot! ---
**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\ **Post date:** [June 30, 2022, 2:38pm UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545/20 "2022-06-30T14:38:37Z")
`scaled_dot_product_attention_test` is defined in the file `public_tests.py`. Itā€™s possible that you didnā€™t run a cell / accidentally deleted the import. Hereā€™s the cell to run: ```auto from public_tests import * get_angles_test(get_angles) # Example position = 4 d_model = 8 pos_m = np.arange(position)[:, np.newaxis] dims = np.arange(d_model)[np.newaxis, :] get_angles(pos_m, dims, d_model) ``` If the import was missing from the start, please refresh your workspace and try again. See `Refresh your Lab Workspace` section [here](https://www.coursera.support/s/article/360044758731-Solving-common-issues-with-Coursera-Labs?language=en_US) [Next page](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545.md?page=2)