Decision boundary - how to choose the polynomial

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

Module 1 - classification decision boundary

In case of dataset examples taken in lecture they have either one feature or two features. They are easy to visualize. Based on visualization it is easy to choose what type of polynomial curve will potentially fit the data best. Decision boundary lecture takes examples of wx+b and for two features it takes example of a circle w1 * x1^2 + w2 * x2^2 + b which becomes a circle.
Question is in case of training sets where there are many features, it is not possible to visualize. In that case how to choose the right polynomial before starting to compute the weights for fine tuning the model curve?

2

Two ideas.
Try all of the combinations of polynomial terms, increasing the order until you get good enough results. Only use the minimum cost as the metric.

Or use a neural network. The nonlinear activation in the hidden layer will automatically generste a comples model, without you having to engineer any additional features.

3

Dear @letusc,

When visualization isn’t possible due to many features, follow this data-driven approach:

  1. Use Polynomial Feature Expansion
    Generate polynomial combinations automatically using tools like PolynomialFeatures in sklearn.
  2. Apply Regularization
    Use models like Logistic Regression with L2 (Ridge) or L1 (Lasso) to avoid overfitting.
  3. Use Cross-Validation
    Test different polynomial degrees (e.g., 1, 2, 3) and select the one with the best validation performance.
  4. Automate with Pipelines and GridSearch
    Combine feature expansion and model tuning into a pipeline and let GridSearchCV pick the best combination.