Conceptual doubt in Chain of assumptions in ML

Why is “Fit test set well on cost function” a stage in flow? I was of the understanding that dev set is more like a pseudo test set - which you tune on. But test set is never used till the last moment when the final model performance is to evaluated.

Fitting on test set will lead to data leakage - which can lead to poor performance in production.
Hoping the community helps me gain clarity on this aspect

Thank you

Yes if you put it this way you are right there is no fitting of the test set!

I think you’re just misinterpreting what he means by “fit” there. He doesn’t mean that you run training using the test set. He just means that the accuracy on the test set is good (and the cost is low), as a result of the training and tuning you did using the train and dev sets. That is a stage that you have to achieve in order to succeed. Then you hope that the performance on real world data is also good, but that is not guaranteed. This should all be clear if you listen again to the words that he actually says to accompany this slide.