Question 7: Production machine learning combines which key disciplines?
‘Modern software development’ is checked as correct, but software testing not… with the following slightly cryptic correction statement:
“Close, but not exactly right. While software testing is important to generate production level machine learning systems, this discipline is the combination of two broader subdisciplines.”
What two broader subdisciplines ?
Also ‘Feature selection and engineering’ is marked as wrong with the equally confusing statement:
“Try again. As a hint, what larger discipline encompasses feature selection?”
The question does not explicitly state that only 2 of the 4 possible answers is accepted as correct. Otherwise, anything that is correct… is correct, not only those that don’t overlap with other possible answers that may be a "combination of broader subdisciplines or encompassed by “larger discipline” … (?)
Question 8: What are the typical challenges of a production grade ML system ? (Check all that apply)
‘Deploying the model to serve requests.’ is checked as incorrect with the following statement:
“This might not be the case. While this is a fundamental aspect of ML systems, it is not a challenge”
… I find this confusing. It’s become a question about how one defines the word “challenge” vs “fundamental aspect”. Even if the question had been "what are the typical challenges as opposed to the fundamental aspects of a production… ", I’d still be unclear what is being asked for.
‘Assessing model performance’ is also checked as incorrect, with another odd statement:
“Not entirely accurate, as figuring out the right performance metric is a general ML challenge, but is not restricted to production systems.”
The question wasn’t “What are exclusively typical challenges of a production grade ML system that are not typical of other general ML challenges?”
And ‘Training the model on real world data’ is checked as incorrect for the same reason as above.
It seems to me that these 2 questions (#7 and #8) are asking for something more specific but rather than specifying it in the text of the questions themselves they are described or eluded to in the text of the correction statements.