In week 3, Decision Boundary lecture, i have a question about using z = w.x + b for g(z) = 1/(1 + exp(-z)). why we would use linear function z instead of other function such as polynomial ?
Hello - I believe this is covered to some degree in the later section on āRegularisation to Reduce Overfitting - The Problem of Overfittingā (see slide 29+ of the notes). While higher order polynomials will tend to fit the training data better, they may not generalise well for data not seen during the training phase. Regularisation techniques help you find the āgoldilocks zoneā between high bias (under fitting) and high variance (over fitting), that is an order of polynomial thatās not too great and not too small.
1 Like
Consider that in the form z = w * x + b, itās entirely possible that w and x are vectors, where some of the values in āxā are derived via computing polynomial combinations of a smaller set of observations.
Since such a system would be impossible to plot (because if X has āNā features, then the plot of g(z) would be N+1 dimensional), so that scenario is not used for any lecture examples.