How do you determine what are the dimension of vector x which are new features ?

Like in the videos x1 and x2 are used but if you are assuming w, x and b all as parameters how do you determine the shape/dimensions of it ?

How do you determine what are the dimension of vector x which are new features ?

Like in the videos x1 and x2 are used but if you are assuming w, x and b all as parameters how do you determine the shape/dimensions of it ?

Experimentation. You want enough features to have good variance, but not so many that training becomes too slow.

Hello @Nikunj_Saluja,

This is also a good time to revisit the lectures in MLS Course 2 Week 3 on the topics of Bias, Variance, Model evaluation, and Learning curve. If we give too many dimensions to w and x (they have to have the same dimension), it is likely to overfit the model to the training data, causing a high variance problem and this can be seen in comparing the learning curves on the training data and evaluation data (cv set). In constrast, if we give too few dimensions, then a high bias can occur. Therefore, we want to find a balanced dimension that get us the best performance on the evaluation set, and in fact, if you have heard of a term called â€śhyperparameter tuningâ€ť, the dimension is one such hyperparameter, just like the polynomial degree in Course 1, that requires us to tune via, as Tom said, experimentation.

Cheers,

Raymond

1 Like