Overfit undefit justright linear quadratic polynomial

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

Hi there!
Linear and quadratic functions can also lead to overfitting when multiple features exist (x1, x2, and x3).

When there is a single feature in a linear equation, then underfitting occurs.
When there is a single feature in a quadratic equation, then *just right or balanced" happens. Linear
When there is a single feature in a polynomial equation, overfitting happens.
I think I am not wrong.
Waiting for our mentor’s approval.

image
image1400×788 59.1 KB

2

Do you have a question?

1 Like
3

Am I right or wrong?

1 Like
4

Your three sketches seem correct.

I don’t totally understand what else you are asking about.

1 Like
5

A single feature in a linear equation will lead to underfitting.
I asked that when overfitting in linear equations occurs, there are multiple features, right?

1 Like
6

Sorry, I still don’t understand, which “equation” are you referring to? Is it the model?

The model is always going to have the same number of weights as you have features.

Some datasets will require a more complex model (i.e additional quadratic or polynomial features) in order to get a low cost solution.

If you make the data set more complex, a linear model will probably be underfit.

A quadratic or polynomial model might overfit a complex data set. The fix for this is to use regularization, to limit the influence of the higher-order features. It does this by reducing the magnitude of the weights.

1 Like
7

We can close this discussion.
Thanks

1 Like
9

Hi @farhana_hossain,

In addition to Tom‘s great reply:

  1. Polynomial fits usually tend to oscillate, especially in the boundary / edge regions where you have labels.

  2. this statement is not generally true:

The statement is quite true with respect to linear models where you have one weight for each feature + the bias. So in 1D linear regression you have two weights (the bias and the weight for the gradient / slope).

But especially in rather complex nonlinear models with hidden layers, like deep neural networks, you can have waay more weights than input features.

Also: important aspects to consider in the underfitting / overfitting discussion is often:

  • the capacity of the model (e.g. shaped by the model weights)
  • the complexity of the relationship you want to model (e.g. a challenging nonlinear classiciation task)
  • the data type (e.g. unstructured video data)
  • the amount of data and it’s ratio to the model capacity mentioned above (the more data you have and the more complex the task you wanna solve (e.g. classifying unstructured video data) the more model capacity (= more weights or more hidden layers) you can allow resp. you are going to need so solve your business problem.

Here some more info if you wanna dive deeper into the topic:

Happy learning!

Best regards
Christian

1 Like