Hey @Matt_Gerhold, first of all, I would like to highlight a software issue that is currently being faced by many learners across DLS, in which the correct answers are not getting assigned to questions. Here’s a thread about this issue.
Now, let’s see one by one, if in some case, your question has been assigned the correct answers, and still, you feel like there is an inconsistency. In your first query, you have answered your query yourself. As you said, satisficing metrics are threshold-based. In the first question, the thresholds haven’t been provided in the question. Therefore, as per the language of the question, all the 3 metrics will be considered as optimizing metrics, and in that case, it will be difficult to decide, which metric to prioritize. I guess, this resolves your query for the first and second questions.
Coming to your sixth question, you have mentioned about your experiment on MNIST dataset. In this case, I would like to ask you 2 questions. What % of increase did you get in training time upon adding 10% more images in the dataset? Can you trade-in that increase for the peace of Peacetopia? And the second question is that, what if you optimize your training process, for instance, using Mini-Batch Gradient Descent instead of Batch Gradient Descent, etc, then will you get the same increase in training time?
Now, coming to your query about 5th and 14th questions, I would like to appreciate you taking your learning from one question and applying to another. But if you carefully read the questions, you will find that there is a key difference between the 2 questions. In the 5th question, the true distribution is still the same, that is dev and test sets still represent the examples that the model will face once it is deployed. In this case, adding these different examples in the training set, will only make it more robust.
However, in the 14th question, the true distribution has been modified, since the dev and test sets no longer represent the examples that the model is facing during its deployment, and since the model is optimized using the dev/test sets, hence, it makes sense to redefine the dev/test sets and using the redefined dev/test sets, define a new evaluation metric.
As to your last query, I agree with you, since I don’t recall this example being discussed in any of the lectures too. Still, you will find that the meaning of the particular option doesn’t rely on the example mentioned alongside. And I will make sure to pass it down to the team for future revisions. Thanks a lot for your input and feedback.
Before I conclude this answer, I would just like to mention one point. These MCQs might have more than one option that might seem to be correct, but we have to weight those options against one another, to find the most suitable option. I hope this helps.
Regards,
Elemento