How to perform Optimizing and Satisficing on N-metrics? I did not understand the concept of distributing N metrics for Optimizing and Satisficing i.e 1 for optimizing and N-1 for Satisficing.
The high level of take away is that to compare learning algorithms we need to define a single most important metric – the optimizing metric. The other evaluation metrics we call satisfying metrics.
To compare learning algorithms:
- Evaluate them on the test set and calculate the optimizing and satisfying metrics.
- Discard all the algorithms that doesn’t satisfy our satisfying metric.
- Pick the algorithm with the best value for optimizing metric.
- If there are many algorithms with the same value for optimizing metric, pick the one with the best satisfying metric.