Bias and Variance

[Elemento]

Hey @Alexander_Leon,
My bad. So, essentially your question revolves around “How to determine bias when there is no human-level (or baseline) performance available?”. There have been a great many discussions on this in the past, let me link a few of those:

• Bayes error, human-level performance and overfitting (structured data)
• Human Level Performance, how to set it?
• What to do when there is no human-level performance baseline?
• How to measure human level performance against human made labels?
• Approach when human-level performance not available

You will find that these posts share a great deal of knowledge regarding your query. Do check these out.

@Alexander_Leon

Using that video as reference, if I split my Kaggle dataset into three and computed that the training error was 10.8% and my cross-validation error was 14.8%, then does this demonstrate high bias or high variance?

Now, when it comes to a Kaggle competition, there is a simple hack. We just check out the top scorer in the leaderboard Let’s say that he/she receives a 1% error, so we can simply establish this as the baseline performance (since we want to beat the top scorer), and if this is the case, you can easily determine that your model has high bias and considerable variance as well, and off-you go! Let me know if this helps.

Cheers,
Elemento