Hi coms! I am unable to understand the Bias and Variance concept and i tried several time to grab this concept but in vain can someone explain to me in simple word?
@Muhammad_Shahrose_Kh
did you check this video already in which Prof. Ng provides a great explanation on this topic:
Which specific part is unclear?
In general it’s about the trade-off between underfitting ans overfitting, see also these threads:
- Overfitting or Underfitting - what is the issue? - #3 by Christian_Simonis
- Bias and variance , tradeoff - #2 by Christian_Simonis
quoting from Wikipedia:
The bias–variance dilemma or bias–variance problem is the conflict in trying to simultaneously minimize these two sources of error that prevent supervised learningalgorithms from generalizing beyond their training set:[1][2]
- The bias error is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting).
- The variance is an error from sensitivity to small fluctuations in the training set. High variance may result from an algorithm modeling the random noise in the training data (overfitting).
Best regards
Christian
Think of bias as a model’s tendency to consistently make the same kind of mistake or to predict results that are systematically off the mark. It’s like having a target that’s consistently off-center. If a model has high bias, it means it’s oversimplified and not capturing the underlying complexity of the data. This can lead to underfitting, where the model is too rigid to adapt to the intricacies of the data, resulting in consistently inaccurate predictions.
Variance, on the other hand, refers to a model’s sensitivity to small fluctuations or noise in the training data. Imagine trying to hit the same target multiple times, but each time your aim slightly varies. If a model has high variance, it means it’s capturing the noise in the training data rather than the underlying patterns. This leads to overfitting, where the model is too flexible and fits the training data extremely well but struggles to generalize to new, unseen data, causing erratic predictions.
Finding the right balance between bias and variance is essential for a model to perform well. If you reduce bias, you might increase variance, and vice versa. The goal is to have a model that’s flexible enough to capture the underlying patterns without getting bogged down by noise. Striking this balance is often referred to as the bias-variance trade-off. It’s like finding the optimal aiming point that consistently hits the target without being too affected by small variations.
In simpler terms, think of bias as how well you aim at the target (accuracy), and variance as how consistently you hit the same spot (precision). Finding the right combination of accurate aiming and consistent hitting is the key to building a model that generalizes well to new data.
Best regards
elirod
Hi @Ahmad_Khalid1,
If you like the example of writing alphabets we can continue with that,
BIAS
Suppose a teacher show a class how to write English alphabets on blackboard. Now each student (model) try to replicate these alphabets what they learned. They all may succeed to write all asked letters but you can note that may ‘A’ , ‘B’ or any other character of two students are not same. Indeed they are the same characters that teacher asked. But biased introduced due to intrinsic properties of students(model) or data or their perception i.e. (sitting posture, how they hold pen, in short their writing style).
If student write more like his teacher he would have less bias.
We can note same behaviour in spoken languages in form of pronunciation of same word across different geographical areas and accent in languages across continents.
Variance
How much same alphabets from same student variate from each other on different attempts. To understand this try writing lowercase letter (i, a, r) at least five times. you will note the variance in your written characters (predicted values). And if you want to see magic try drawing opening and closing braces many times. { } you surely note the variance in closing braces. 
If your written characters are too much similar you would have less variance.
And thanks for asking this question. 
@Ahmad_Khalid1 Since you are arabic speaker, the following analogy should close your question:
إليك شرح لتشبيه “التحيز مقابل التباين” (Bias vs. Variance) باستخدام مثال رامي السهام باللغة العربية:
تخيل أننا نطلق السهام على هدف (اللعب)، ونريد إصابة مركز الهدف (النقطة الحمراء).
1. التحيز العالي (High Bias) - “الرامي المنحاز”
تخيل رامي سهام يملك بندقية أو قوسًا ماطرًا (معيبًا)، أو تدرب بطريقة خاطئة جعلته يصوب دائمًا ناحية اليسار.
-
المشهد: كلما أطلق سهامه، تذهب جميعها وتتجمع في جهة اليسار بعيدًا عن المركز. السهام متجمعة مع بعضها (منظمة)، لكنها بعيدة عن الهدف.
-
المعنى في الذكاء الاصطناعي: هذا يمثل “Underfitting” (التعميم الزائد). النموذج بسيط جدًا ومتحيز، فهو يفشل في التقاط النمط الصحيح للبيانات (يخطئ في التدريب وفي الواقع).
2. التباين العالي (High Variance) - “الرامي المتذبذب”
تخيل رامي سهام ماهر ولكنه متوتر جدًا أو يعتمد على الحظ في كل رمية.
-
المشهد: يرمي السهم الأول فيصيب أعلى الهدف، والثاني يصيب الأسفل، والثالث يمينًا، والرابع يسارًا. السهام متناثرة في كل مكان (عشوائية).
-
المعنى في الذكاء الاصطناعي: هذا يمثل “Overfitting” (التخصيص الزائد). النموذج معقد جدًا وحساس لأي تغيير صغير، فهو “يحفظ” البيانات بدلاً من أن يتعلم منها، مما يجعله غير دقيق عند مواجهة بيانات جديدة.
3. الهدف المثالي (Low Bias, Low Variance)
-
المشهد: جميع السهام تذهب وتتجمع بدقة في وسط الهدف (مركز النقطة الحمراء).
-
المعنى: هنا يكون “التحيز” منخفضًا (التصويب صحيح) و"التباين" منخفضًا (السهام متراصة ومستقرة). هذا هو الهدف الذي نسعى إليه في أي نموذج ذكاء اصطناعي.