Hi,
I’ve just completed the NMT quiz for NLP C4_W1. I found the quality of this quiz to be quite poor and it was not a helpful assessment of my learning. Frankly, it was not a good use of my time, which was disappointing as I’ve enjoyed many of the other quizzes in this specialization.
Some of the issues I found:
- Q1: The choice “They are not that useful” is a filler answer & doesn’t test knowledge.
- Q2: This question asks for benefits and 2 of the 4 answers are clearly not benefits: “The use of attention ends up giving you less accurate results.” & “It is significantly slower to use attention and therefore it is not recommended to use it.” As in Q1, these are wasteful filler answers that don’t test knowledge.
- Q3: This question asserts that queries, keys, and values are required components of attention, but it seems this is not true for all variations of attention. The first lecture video in this course that introduces attention - “Seq2seq Model with Attention” (describing the 2014 paper) - doesn’t use QKV, and the lecture video “Queries, Keys, Values, and Attention” makes clear in the first minute that QKV was only introduced in 2017 as one way of implementing attention.
- Q3: Also in Q3, the “hint” in the “Values” answer isn’t really a hint, it just tells you to select this option. The question is “What is required…” and the hint is “you need this”. Come on!
- Q7: I found the framing of this question confusing. BLEU and Rouge are introduced as precision- and recall-oriented evaluation methods, respectively, but this question is just about Rouge, and the lecture video’s description of Rouge does not provide a Rouge-specific definition of “precision”.
- Q9: This question uses the term “the golden one” as an apparent colloquialism, which was immediately confusing. I had to go back and think: Was “the golden one” a technical term that Younes mentioned in the lecture? Was it mentioned in an earlier course in this specialization? Using informal language like this is confusing!
I take the time to share this only because I’ve largely enjoyed this specialization so far & look forward to the quiz as a helpful way to test my learning. I’d rather get some questions wrong & carefully check my understanding than get an easy 100% on the quiz and not really check whether I’ve learned.
I hope this quiz is improved for future students!
Thank you,
Erik