C1W3 how is the probability of two random labelers agreeing calculated?

How is this probability calculated in the HLP video of C1W3? Which rules of probability are being used here?

Hello @chetna
The calculation of Human-Level Performance (HLP) in machine learning does not typically involve the direct use of any specific probability rule. Rather, HLP is a measure of the level of performance achieved by a human expert on a particular task, and it is used as a benchmark to evaluate the performance of machine learning models.

To calculate HLP, one typically gathers data on the performance of human experts on the task in question, and then compares that performance to the performance of a machine learning model on the same task. HLP is then calculated as the percentage reduction in error rate achieved by the machine learning model relative to the error rate of the human experts.

While probability theory and statistical methods may be used to analyze the data and calculate error rates, HLP itself is not a probability-based measure. Rather, it is a comparison of the performance of humans and machines on a specific task, and is typically expressed as a percentage or other numerical value.

1 Like