# Sequence Models **URL:** https://community.deeplearning.ai/c/course-q-a/deep-learning-specialization/dls-course-5/16.md [Latest](https://community.deeplearning.ai/latest.md) · [Categories](https://community.deeplearning.ai/categories.md) · [Tags](https://community.deeplearning.ai/tags.md) --- ## [Help! My code is correct, but the grader gives me 0 points! - Course 5 Version](https://community.deeplearning.ai/t/help-my-code-is-correct-but-the-grader-gives-me-0-points-course-5-version/11041)
**Author:** [@Mubsi](https://community.deeplearning.ai/u/Mubsi)\ **Replies:** 1\ **Last updated:** [June 1, 2021, 9:14pm UTC](https://community.deeplearning.ai/t/help-my-code-is-correct-but-the-grader-gives-me-0-points-course-5-version/11041 "2021-06-01T21:14:09Z")
For general tips on why your submission might be failing, visit this post. For tips specific to this Course, look below. --- ## [Meet your mentors - DLS Course 5!](https://community.deeplearning.ai/t/meet-your-mentors-dls-course-5/601)
**Author:** [@agparth](https://community.deeplearning.ai/u/agparth)\ **Replies:** 0\ **Last updated:** [April 15, 2021, 4:32pm UTC](https://community.deeplearning.ai/t/meet-your-mentors-dls-course-5/601 "2021-04-15T16:32:51Z")
Hello, learners/fellow magicians. Congratulations on reaching the final part of this quest aka specialization! Your sequential learning has led you to Course 5 of this specialization! To help you out, we have some supe… --- ## [About the Sequence Models category](https://community.deeplearning.ai/t/about-the-sequence-models-category/55)
**Author:** [@deeplearning](https://community.deeplearning.ai/u/deeplearning)\ **Replies:** 2\ **Last updated:** [January 1, 2024, 9:58am UTC](https://community.deeplearning.ai/t/about-the-sequence-models-category/55 "2024-01-01T09:58:06Z")
Welcome the DLS Course 5: Sequence Models Discourse page! In the fifth course of the Deep Learning Specialization, you will become familiar with NLP models and their exciting applications such as speech recognition, mus… --- ## [Pedagogy of C5 W4 A1 Transformer (Ex4: Encoder Layer)](https://community.deeplearning.ai/t/pedagogy-of-c5-w4-a1-transformer-ex4-encoder-layer/3892)
**Author:** [@marcus-waldman](https://community.deeplearning.ai/u/marcus-waldman)\ **Replies:** 43\ **Last updated:** [October 2, 2024, 3:04am UTC](https://community.deeplearning.ai/t/pedagogy-of-c5-w4-a1-transformer-ex4-encoder-layer/3892 "2024-10-02T03:04:46Z")
Hi, I just want to start this by saying how impressed I am with the pedagogy overall throughout all four classes. Concepts really build one on top of the other and all challenges–and make no mistake challenges are necess… --- ## [C5W4: Transformer Architectures with TensorFlow](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545)
**Author:** [@falonso](https://community.deeplearning.ai/u/falonso)\ **Replies:** 42\ **Last updated:** [January 6, 2026, 12:03am UTC](https://community.deeplearning.ai/t/c5w4-transformer-architectures-with-tensorflow/13545 "2026-01-06T00:03:56Z")
The last assignment of course 5 week 4 was really a disaster. Nothing is understood, it is not well explained. It is much more difficult than the previous tasks. I ended up copying from the trasnformers tutorial from the… --- ## [Week 1 Assignment 1 Backpropagation](https://community.deeplearning.ai/t/week-1-assignment-1-backpropagation/15940)
**Author:** [@reddyroh](https://community.deeplearning.ai/u/reddyroh)\ **Replies:** 19\ **Last updated:** [July 20, 2024, 3:11pm UTC](https://community.deeplearning.ai/t/week-1-assignment-1-backpropagation/15940 "2024-07-20T15:11:11Z")
Why do we need to add da\_prevt to the upstream gradient, i.e. da\_next = da\[:,:,t\] + da\_prevt not just da\[:,:,t\]? I don’t understand the math here. This is for rnn\_backward function. Why is da\_prevt being added to the ups… --- ## [W1 A2 | Ex-2 Sample Function](https://community.deeplearning.ai/t/w1-a2-ex-2-sample-function/39506)
**Author:** [@aquilaxc](https://community.deeplearning.ai/u/aquilaxc)\ **Replies:** 5\ **Last updated:** [July 16, 2023, 7:47am UTC](https://community.deeplearning.ai/t/w1-a2-ex-2-sample-function/39506 "2023-07-16T07:47:56Z")
Hi, I notice that in the hint of Step 3: Also notice that 𝑦̂ ⟨𝑡+1⟩y^⟨t+1⟩, which is y in the code, is a 2D array. I cannot figure out why is ‘y’ a 2D array? Shouldn’t it be a 1D probability vector over the vocabulary? … --- ## [Week 1 - Assignment 3](https://community.deeplearning.ai/t/week-1-assignment-3/2377)
**Author:** [@amirhossein\_berenji](https://community.deeplearning.ai/u/amirhossein_berenji)\ **Replies:** 40\ **Last updated:** [September 8, 2023, 5:09pm UTC](https://community.deeplearning.ai/t/week-1-assignment-3/2377 "2023-09-08T17:09:38Z")
Hi everyone I am not able to get the music\_inference\_model(LSTM\_cell, densor, Ty=100) work as it is supposed to. I had worked on it for a couple of hours, but I could not find a way around it that everything goes safe a… --- ## [W1 A3 | Ex-1 | djmodel- AttributeError: layer “lstm” has multiple inbound nodes](https://community.deeplearning.ai/t/w1-a3-ex-1-djmodel-attributeerror-layer-lstm-has-multiple-inbound-nodes/7763)
**Author:** [@garrofederico](https://community.deeplearning.ai/u/garrofederico)\ **Replies:** 39\ **Last updated:** [September 29, 2022, 6:08pm UTC](https://community.deeplearning.ai/t/w1-a3-ex-1-djmodel-attributeerror-layer-lstm-has-multiple-inbound-nodes/7763 "2022-09-29T18:08:36Z")
I’m having the following error in Ex 1, djmodel funcition: “AttributeError: The layer “lstm” has multiple inbound nodes, with different output shapes. Hence the notion of “output shape” is ill-defined for the layer. Use… --- ## [Course 5: Week 1: Music inference model (LSTM)](https://community.deeplearning.ai/t/course-5-week-1-music-inference-model-lstm/8016)
**Author:** [@Hinnerk8](https://community.deeplearning.ai/u/Hinnerk8)\ **Replies:** 27\ **Last updated:** [June 26, 2025, 4:03am UTC](https://community.deeplearning.ai/t/course-5-week-1-music-inference-model-lstm/8016 "2025-06-26T04:03:42Z")
Hey guys, i have some problems with the inference model and i am not able to fix it. When running my code it tells me: Input 0 is incompatible with layer lstm: expected shape=(None, None, 90), found shape=\[90, 1, 1\] S… --- ## [C5\_W4\_A1 DON'T PANIC! Transformer help is on the way](https://community.deeplearning.ai/t/c5-w4-a1-dont-panic-transformer-help-is-on-the-way/130469)
**Author:** [@pslusarz](https://community.deeplearning.ai/u/pslusarz)\ **Replies:** 6\ **Last updated:** [October 2, 2024, 3:23am UTC](https://community.deeplearning.ai/t/c5-w4-a1-dont-panic-transformer-help-is-on-the-way/130469 "2024-10-02T03:23:39Z")
As noted elsewhere on the forum, the Transformer lab is a bit of a train wreck, with insufficient explanation in the lectures, and insufficient number of suggestions in the assignment. While the course staff is working o… --- ## [\[Week 2\] Assignment 1: Operations on Word Vectors - Debiasing - Exercise 5.2](https://community.deeplearning.ai/t/week-2-assignment-1-operations-on-word-vectors-debiasing-exercise-5-2/2676)
**Author:** [@Andreas](https://community.deeplearning.ai/u/Andreas)\ **Replies:** 54\ **Last updated:** [February 16, 2024, 5:13pm UTC](https://community.deeplearning.ai/t/week-2-assignment-1-operations-on-word-vectors-debiasing-exercise-5-2/2676 "2024-02-16T17:13:42Z")
Are the formulas really correct and is the expected output really correct?? I tried different things (used np.linalg.norm or calculated everything without this function) but still can’t get the expected output. My resu… --- ## [Permission denied course 5](https://community.deeplearning.ai/t/permission-denied-course-5/300430)
**Author:** [@Sina\_Kian](https://community.deeplearning.ai/u/Sina_Kian)\ **Replies:** 23\ **Last updated:** [April 9, 2023, 9:10pm UTC](https://community.deeplearning.ai/t/permission-denied-course-5/300430 "2023-04-09T21:10:30Z")
Dear Tutors, I finished the excersie 5 of the last course. But I cannot nor submit niether save my work. It says autosave failed, and permission denied. What Can I do? --- ## [\[C5W1A1\] wrong results of lstm\_backward](https://community.deeplearning.ai/t/c5w1a1-wrong-results-of-lstm-backward/8480)
**Author:** [@wziz](https://community.deeplearning.ai/u/wziz)\ **Replies:** 14\ **Last updated:** [August 2, 2024, 5:39pm UTC](https://community.deeplearning.ai/t/c5w1a1-wrong-results-of-lstm-backward/8480 "2024-08-02T17:39:17Z")
I got the wrong values of results from the function lstm\_backward. But the shapes of the results and all other corresponding functions are passed. Below are my code and results. I’ve been stuck here for 2 days, please h… --- ## [W4 A1 | Is there a typo in Multi-head attention slides?](https://community.deeplearning.ai/t/w4-a1-is-there-a-typo-in-multi-head-attention-slides/135478)
**Author:** [@Hardik\_Modi2](https://community.deeplearning.ai/u/Hardik_Modi2)\ **Replies:** 9\ **Last updated:** [November 10, 2022, 3:46am UTC](https://community.deeplearning.ai/t/w4-a1-is-there-a-typo-in-multi-head-attention-slides/135478 "2022-11-10T03:46:21Z")
The notation I am going to use is as follows: In context of self-attention (first image below): Superscript in angular brackets means the 'ith word in the input sequence. for e.g. in q\, k\, v\ and A\ the … --- ## [\[week 4\] Transformer Network - get\_angles](https://community.deeplearning.ai/t/week-4-transformer-network-get-angles/2711)
**Author:** [@Georgii](https://community.deeplearning.ai/u/Georgii)\ **Replies:** 56\ **Last updated:** [January 10, 2026, 8:51pm UTC](https://community.deeplearning.ai/t/week-4-transformer-network-get-angles/2711 "2026-01-10T20:51:18Z")
Have the issue: “Submatrices of odd and even columns must be equal”. Code in “get\_angles” function: {mentor edit: code removed} Looks it computes values and keeps dimensions right (even without “.astype(np.float)”) --- ## [Course 5 Week 1 Assignment 2 Exercise 2](https://community.deeplearning.ai/t/course-5-week-1-assignment-2-exercise-2/16181)
**Author:** [@psteele](https://community.deeplearning.ai/u/psteele)\ **Replies:** 16\ **Last updated:** [December 11, 2023, 1:11pm UTC](https://community.deeplearning.ai/t/course-5-week-1-assignment-2-exercise-2/16181 "2023-12-11T13:11:17Z")
Hi, i’m hitting an error related to the numpy.random.choice function that seems to have to do with probabilities that are slightly above the tolerance for this function. I get the following error: ----------------------… --- ## [W4 A 1 | Ex- 4 | Encoder Layer](https://community.deeplearning.ai/t/w4-a-1-ex-4-encoder-layer/9609)
**Author:** [@Fangyi](https://community.deeplearning.ai/u/Fangyi)\ **Replies:** 15\ **Last updated:** [May 30, 2023, 9:14am UTC](https://community.deeplearning.ai/t/w4-a-1-ex-4-encoder-layer/9609 "2023-05-30T09:14:13Z")
Hi, I’m really frustrated with this week’s exercises and I can’t find really helpful instructions. The syntax of this exercise make so little sense to me… (Solution code removed by staff) I got assertion error for wron… --- ## [Cell #24. Can't compile the student's code. Error: FileNotFoundError(2, 'No such file or directory')](https://community.deeplearning.ai/t/cell-24-cant-compile-the-students-code-error-filenotfounderror-2-no-such-file-or-directory/60787)
**Author:** [@amirnasab](https://community.deeplearning.ai/u/amirnasab)\ **Replies:** 23\ **Last updated:** [December 16, 2024, 11:10pm UTC](https://community.deeplearning.ai/t/cell-24-cant-compile-the-students-code-error-filenotfounderror-2-no-such-file-or-directory/60787 "2024-12-16T23:10:57Z")
I have this isue on week 3 of course 5 on trigger word detection. can anyone help me please? everything is all passed but i get this log --- ## [Week 1 Assignment 2 Exercise 2 - sample](https://community.deeplearning.ai/t/week-1-assignment-2-exercise-2-sample/72500)
**Author:** [@Oleksandr\_Semenov](https://community.deeplearning.ai/u/Oleksandr_Semenov)\ **Replies:** 12\ **Last updated:** [January 27, 2024, 11:59pm UTC](https://community.deeplearning.ai/t/week-1-assignment-2-exercise-2-sample/72500 "2024-01-27T23:59:07Z")
Hello there, I have an issue with this exercise. My code gets only first 4 indexes right \[23, 16, 26, 26, 19, 25, 4, 1, 15, 6, 4, 25, 23, 23, 10, 14, 16, 12, 11, 3, 20, 16, 22, 23, 14, 15, 6, 9, 26, 2, 6, 1, 26, 11, 2,… --- ## [Deeplearning Speacialization Course 5 , week 2- Problem in Emojify assignment](https://community.deeplearning.ai/t/deeplearning-speacialization-course-5-week-2-problem-in-emojify-assignment/19196)
**Author:** [@Ash23](https://community.deeplearning.ai/u/Ash23)\ **Replies:** 30\ **Last updated:** [December 26, 2023, 3:53pm UTC](https://community.deeplearning.ai/t/deeplearning-speacialization-course-5-week-2-problem-in-emojify-assignment/19196 "2023-12-26T15:53:42Z")
Cell #14. Can’t compile the student’s code. Error: KeyError(‘never’,) The problem is in the test case of cell #14. In the sentence to average function the variable avg is assigned same shape as word in word\_to\_vec\_map() … --- ## [Week 1 assignment 3, exerrcise 1](https://community.deeplearning.ai/t/week-1-assignment-3-exerrcise-1/13000)
**Author:** [@Akshat](https://community.deeplearning.ai/u/Akshat)\ **Replies:** 3\ **Last updated:** [December 11, 2023, 3:37pm UTC](https://community.deeplearning.ai/t/week-1-assignment-3-exerrcise-1/13000 "2023-12-11T15:37:00Z")
I did everything as asked,( I suppose), and I am struck. In my model summary the layers are not in exact order my model summary:- expected summary:- can anyone plz help me… --- ## [Trigger word detection - Spectrogram is wrong](https://community.deeplearning.ai/t/trigger-word-detection-spectrogram-is-wrong/12451)
**Author:** [@Mandyzmr](https://community.deeplearning.ai/u/Mandyzmr)\ **Replies:** 12\ **Last updated:** [June 28, 2023, 3:36pm UTC](https://community.deeplearning.ai/t/trigger-word-detection-spectrogram-is-wrong/12451 "2023-06-28T15:36:33Z")
Hi team, For the assignment to define create\_training\_example, I passed my code online, while when i double checked it offline, it pops up an error as follows: What might be the issue? --- ## [Getting the premission denied error and jupyter notebook doesn't save, also I don't see the submit assignment button](https://community.deeplearning.ai/t/getting-the-premission-denied-error-and-jupyter-notebook-doesnt-save-also-i-dont-see-the-submit-assignment-button/299326)
**Author:** [@Avinash\_Vootkuri](https://community.deeplearning.ai/u/Avinash_Vootkuri)\ **Replies:** 36\ **Last updated:** [April 1, 2023, 5:05pm UTC](https://community.deeplearning.ai/t/getting-the-premission-denied-error-and-jupyter-notebook-doesnt-save-also-i-dont-see-the-submit-assignment-button/299326 "2023-04-01T17:05:42Z")
Hi, Unable to submit assignment DLS C5 W1, getting permission denied error when saving the file. --- ## [Week 4 - Encoder error - Incompatible shapes: \[2,3,4\] vs. \[1,5,4\] \[Op:AddV2\])](https://community.deeplearning.ai/t/week-4-encoder-error-incompatible-shapes-2-3-4-vs-1-5-4-op-addv2/11932)
**Author:** [@chaitanyamanem](https://community.deeplearning.ai/u/chaitanyamanem)\ **Replies:** 7\ **Last updated:** [January 30, 2023, 4:17pm UTC](https://community.deeplearning.ai/t/week-4-encoder-error-incompatible-shapes-2-3-4-vs-1-5-4-op-addv2/11932 "2023-01-30T16:17:02Z")
i am getting error : Incompatible shapes: \[2,3,4\] vs. \[1,5,4\] \[Op:AddV2\]) in #UNQ\_C5 x giving output of shape (2, 3, 4) because Encoder\_test passing (2,3) array as input and embedding\_dim=4 but self.pos\_encoding return… --- ## [C5W1A3 exercise2 error confused](https://community.deeplearning.ai/t/c5w1a3-exercise2-error-confused/90016)
**Author:** [@ladan\_yang](https://community.deeplearning.ai/u/ladan_yang)\ **Replies:** 5\ **Last updated:** [July 18, 2022, 9:52am UTC](https://community.deeplearning.ai/t/c5w1a3-exercise2-error-confused/90016 "2022-07-18T09:52:54Z")
Does anyone know what does this mean? I am confused :pleading\_face: --- ## [W4, Exercise 3 - scaled\_dot\_product\_attention: what does the dimension of q, k, v mean?](https://community.deeplearning.ai/t/w4-exercise-3-scaled-dot-product-attention-what-does-the-dimension-of-q-k-v-mean/48878)
**Author:** [@larryleguo](https://community.deeplearning.ai/u/larryleguo)\ **Replies:** 3\ **Last updated:** [June 22, 2022, 12:29am UTC](https://community.deeplearning.ai/t/w4-exercise-3-scaled-dot-product-attention-what-does-the-dimension-of-q-k-v-mean/48878 "2022-06-22T00:29:33Z")
I get the code done and passed, but I still do not understand the dimensions of q, k, v: q – query shape == (…, seq\_len\_q, depth) k – key shape == (…, seq\_len\_k, depth) v – value shape == (…, seq\_len\_v, depth\_v) Wha… --- ## [C5W2A2: Cell #14. Can't compile the student's code. Error: ValueError('shapes (5,2) and (5,) not aligned: 2 (dim 1) != 5 (dim 0)',)](https://community.deeplearning.ai/t/c5w2a2-cell-14-cant-compile-the-students-code-error-valueerror-shapes-5-2-and-5-not-aligned-2-dim-1-5-dim-0/22798)
**Author:** [@reetik07](https://community.deeplearning.ai/u/reetik07)\ **Replies:** 7\ **Last updated:** [April 21, 2022, 9:42pm UTC](https://community.deeplearning.ai/t/c5w2a2-cell-14-cant-compile-the-students-code-error-valueerror-shapes-5-2-and-5-not-aligned-2-dim-1-5-dim-0/22798 "2022-04-21T21:42:43Z")
I have completed the assignment 2 of the week 2 of Course 5 with all the test cases passed. Yet it is showing the same error each time I submit my assignment. --- ## [W4 A1 | Ex-3 | Scaled Dot Product Attention](https://community.deeplearning.ai/t/w4-a1-ex-3-scaled-dot-product-attention/5746)
**Author:** [@Zarif](https://community.deeplearning.ai/u/Zarif)\ **Replies:** 26\ **Last updated:** [March 24, 2025, 7:30pm UTC](https://community.deeplearning.ai/t/w4-a1-ex-3-scaled-dot-product-attention/5746 "2025-03-24T19:30:56Z")
I am trying to build the function scaled\_dot\_product\_attention(q, k, v, mask) -done matmul between q and k transverse -determined the size of dk using dk.size -determined scaled\_attention\_logiits dividing matmul by sq… --- ## [W1, A2, part3 optimize](https://community.deeplearning.ai/t/w1-a2-part3-optimize/242726)
**Author:** [@ROHIT\_MALAWAT](https://community.deeplearning.ai/u/ROHIT_MALAWAT)\ **Replies:** 9\ **Last updated:** [August 8, 2024, 3:16am UTC](https://community.deeplearning.ai/t/w1-a2-part3-optimize/242726 "2024-08-08T03:16:30Z")
while coding my all test cases passed but grader failed that part, what to do ?? [Next page](https://community.deeplearning.ai/c/course-q-a/deep-learning-specialization/dls-course-5/16.md?page=1)