Hi! From the previous question, it seems we have a large data-mismatch problem here. Why do we think the friend is not right here?
Thanks for the help!
You’re right in that the evidence seems to indicate a large data mismatch problem. However, the friend is commenting on the ease of tasks, which can only be estimated with Bayes optimal error (or human-performance proxy). We don’t have this for classification task for the front-facing camera dataset, so we can’t comment on the relative ease of tasks.
Actually, it is mentioned that “You also know that human-level error on the road sign and traffic signals classification task is around 0.5%” in quesiton 6.
Also variance (Training subtracted from Training-Dev) is much lower than data mismatch value.
So I think the current correct answer is wrong.
The quiz answer seems correct. You should take into account that we have mixed data in the training set.
the human-level error in question 6 can be used as Bayes error for the dev/test set, but can it be used to indicate Bayes error for pics we get from Internet?