W3 assignment feature mapping: higher order?

In section 3.3-Feature Mapping, I am experimenting using a 2nd order (set degree = 2 in map_feature(X1, X2) function) to learn the “Exam 1/Exam 2” data used in the first part of the assignment. The code appears to work, but when it comes to the learning process, the costs are all showing “nan”, presumably cost function returns an infinite or undefined result (e.g., divided by zero).

This leads to the question of setting the initial fitting parameters: w_initial and b_initial. In the assignment, these initial parameters are pre-defined using random.seed(1) for the 1st and 6th order problems. However, for other higher order, these parameters don’t work. They don’t even converge.

For 2nd and other order, what are some guidance to choose the initial parameters? Did I overlook the lambda and alpha values too?

The most important issue when you create new features is normalizing the data set. This prevents the gradients from exploding, which can cause problems with NAN values for cost.

It’s not a problem with the initial weight values.

For the initial parameters (for linear and logistic regression), always use all-zeros. There is no method to pick anything better.

Thank you for your quick response! I will give it a shot to normalize the dataset. I assume you are referring to the normalization discussed in Week 2.

At a glance, the Exam1/Exam2 score are both in the range from 0 to 100. In this case, does normalization help?

Hello @lchow,

And seems you just reduced the degree. If you don’t mind to share your notebook with me, I can have a look and suggest how to debug the nan problem. Let me know and I will tell you how to share it with me.

Raymond

The range between the smallest and largest feature values is too large. Normalization will help with that.

Hi @TMosh, I did a quick coding on normalizing the data. It works! Thank you again for your help

Hi Raymond, thank you for offering your help. Normalizing the data as Tom suggested work. I’m still working on adding a couple features to my notebook, i.e., coding from scratch vs. using libraries. I will be happy to share it with you when I am done.

That’s good news! I love to discuss learner’s own work and I look forward to your notebook!

Cheers,
Raymond