Disproportionate Importance

Hello Moderators,

Can you please explain disproportionate importance? can you give an alternate example to web search?

Thanks

Himanshu

Himanshu,
Can you give better context. I did not get your question. May be refer to the section of the week/topic, that also helps.

Hi Himanshu
Even if with a large delay I will try to give you my feedback.
It is not enough for a ML model to have an average high accuracy. To be successful an ML model must have high accuracy on particular data slices that are of crucial importance. Andrew Ng gives the example of navigational queries that are received differently by the end user than transactional/Informational queries. The success of an ML model depends above all on the ability to do well on particular slices of data. The final user gives the judgement on the effectiveness of the ML application. If your learning algorithm does bad even if on a small slice of data but of a disproportionate importance it is not ready to go to production. The weights of data are not the same for every samples. It’s up to us to catch the different slices of data and give them the right weights.

Hello

I got what you explained , can you throw some light on disproportionately importance in your statement , quoted as : “If your learning algorithm does bad even if on a small slice of data but of a disproportionate importance it is not ready to go to production”

What I meant to say so 1 incorrect data of 100 samples is larger impact than 1 incorrect of 1000 samples.

1 Like

Hi,

Here is an alternate example:
You predict the air temperature of the next day on a 2D grid 10x10. 1 point on the grid is a city, all the other ones are forests, fields etc.
You use as a metric the mean absolute error of prediction vs measurement for all points on the grid. You can have a very low error if the prediction is totally wrong for the city point, but perfect everywhere else.
Because most users want to know the temperature in the city, this point is of disproportionately important and you should not put the model to production .

3 Likes

I like the analogy you used to explain disproportionate importance. It makes it more clear to understand.