Use relative error (percentage) or absolute error (percentage points) to analyze avoidable bias and variance?

Course Q&A Deep Learning Specialization Structuring Machine Learning Projects
1

In the videos Professor Ng says of avoidable bias and variance to focus on the one with the larger percentage. However, I believe he means percentage points.

For example, suppose the Bayes Error is 1%, the Training Error is 5%, and the Dev Error is 15%. Then, according to Professor Ng, Avoidable Bias is 4% and the variance is 10%.

Using percentages to compare other percentages is ambiguous however. Do we mean relative or absolute difference? So less ambiguously, we would say that there is a 4 percentage point (%p) difference between Training and Dev Errors, and 10%p difference between Training and Dev error.

However, from a relative perspective, comparing Training Error to Bayes Error is a 5x increase, or 500% increase. While comparing Dev Error to Training Error is a 3x increase, or 300% increase.

My question is, are variance and avoidable bias measured in terms of percentage points (absolute error), or percentage (relative error)? Can you give a justification why we use one rather than the other? This is example is extreme, but if in the former, you would minimize bias, in the latter variance.

2

Hello @eoin12345abc ,
In my reply I will do my best to answer your question. Please do not hesitate to ask a followup question if you feel stuck.

Variance and avoidable bias are generally measured in terms of absolute error (percentage points) rather than relative error (percentage). The justification for using absolute error is that it allows for a more direct and interpretable comparison between the errors, which helps in identifying the main source of error in a model and deciding on the appropriate steps to improve its performance.Avoidable bias is calculated by subtracting the optimal error rate (unavoidable bias) from the training error, representing the error that can be reduced by improving the algorithm. Variance, on the other hand, is the gap between the training and testing accuracy. By measuring these errors in absolute terms (percentage points), we can directly compare them and determine whether the model is suffering more from bias or variance.

Best,
Can