I would greatly appreciate a mentor’s help. I am stuck on Ex8.
I’ve passed every unittest prior to Ex 8.
But I ran into an error that I don’t know what went wrong, specifically:
I know there is a mismatch in array size. Especially a mismatch between w and X. My w matrix came out as (4x4) and if I reshape it in to a column vector (16x1), obviously, I can’t multiply to my X_train matrix of size (4x7).
I am not sure what I did wrong. I would greatly appreciate some help.
Thanks!
w should be a column vector with the same number of elements as the number of features, which is 4 in this particular test case. Put print statements to show the shape of w at various points in your model code like this:
print(f"w.shape {w.shape}")
Put that right after the call to the initialize routine. Then put it right before the call to the predict function that throws the error. What do you see?
You could also put that down in the optimize code to see if anything changes there, although that code involves loops so the output could be pretty voluminous. BTW we’re assuming here that all your lower level functions pass their unit tests.
Actually the thing to do would be to put that print statement in optimize right before the final return statement, so you’ll only get one line of output from it.
But the assumption is that your bug is in the model logic, not in the lower level routines. So the question is how does the value of w get transmitted from the results of optimize to the point in the model code that calls predict, which is where the failure actually occurs? Please examine that logic carefully.
Thanks for the quick response.
I have previously put the w.shape statement for debug in model and got (4x4) which is what puzzled me because I don’t know how it ended up with a 16X1 shape.
I have checked the logic with all my previous exercises, and they came out consistent with the unittest expectations. (All unittests passed up to Exercise 8).
From what I have seen thus far, the logic seems to be correct. That’s why I am stuck because I think the bug might be subtle.
Well, what is the shape right at the return statement in optimize? Where does it show up as 4 x 4 for the first time? That will never work in propagate or optimize.
Show us the shape printouts that you actually get. It must be 4 x 1 after the initialize and before the call to optimize, right?
I added those prints and a few others and here’s what I see:
type(X_train) <class 'numpy.ndarray'>
X_train.shape (4, 7)
X_test.shape (4, 3)
Y_test.shape (1, 3)
before optimize w.shape (4, 1)
num_iterations 50 learning_rate 0.01
at return in optimize w.shape (4, 1)
in model before predict call w.shape (4, 1)
All tests passed!
This is what I have from the various debug print statements I have put in:
w_optimze = [[ 6.84710229 6.84710229 6.84710229 6.84710229]
[-3.92595481 -3.92595481 -3.92595481 -3.92595481]
[-3.54686417 -3.54686417 -3.54686417 -3.54686417]
[ 2.23643535 2.23643535 2.23643535 2.23643535]]
w_optimize_debug = [[ 6.84710229 6.84710229 6.84710229 6.84710229]
[-3.92595481 -3.92595481 -3.92595481 -3.92595481]
[-3.54686417 -3.54686417 -3.54686417 -3.54686417]
[ 2.23643535 2.23643535 2.23643535 2.23643535]]
w params:
[[ 6.84710229 6.84710229 6.84710229 6.84710229]
[-3.92595481 -3.92595481 -3.92595481 -3.92595481]
[-3.54686417 -3.54686417 -3.54686417 -3.54686417]
[ 2.23643535 2.23643535 2.23643535 2.23643535]]
W_optimize_debug is a print statement right before the return statement in optimize
Would it be easier if I send you my notebook?
In your model function, how you are initializing parameters with zeros? Are you using initialize_with_zeros function? What is the argument for that function?
I don’t understand. w should never be anything but a column vector, right? So you’re saying that right after the call to initialize_with_zeros, w is already 4 x 4? How could that happen given that your initialize_with_zeros function passes its test cases?
If that’s not what you’re saying, then you need to speak more clearly. Success at debugging requires clear thinking.
I appreciate the help. I just learned Python a few weeks ago. So I am very new to the syntax.
Also, thanks for reminder on w and b initialization. I didn’t use initialize_with_zeros rather I did it via np.zeros. But I have modified my code to use the initialize_with_zeros function now.
This is what I have done:
w, b = initialize_with_zeros(X_train.shape[0])
And here are the dimensions:
X_train_model shape: (4, 7)
X_test_model shape: (4, 3)
w_model_prior_optimize shape: (4, 1)
The size of w has changed to the correct shape with the data:
w_optimize_debug = [[ 6.00427503]
[-5.02104055]
[-3.50221117]
[ 1.03927252]]
I have the following error instead now:
Ok, we’re making progress. The new error means that you hard-coded the number of iterations when you called optimize from model. If you are new to python, you need to be careful about how “named” or “keyword” parameters work. If there are any equal signs in the parameter list of the call to optimize, that is a mistake: it means you are over-riding the values that are passed in to model at the top level. Or if you omit the learning rate and number of iterations altogether, then that is a different form of “hard-coding”: you are saying that you want to use the default values declared in the definition of optimize.
If any of what I said above does not make sense, try googling “python keyword parameters” and read some tutorials about them.
Also note that it is late my time: I am signing off for the evening and will be offline for the next 8 or 10 hours.
Will check the function call to optimize as suggested.
Thanks for the help thus far, both!
Big help on the reminder to use the initialize_with_zeros and not hard code the parameters on the optimize function call.
I have modified both:
{Moderator Edit: Solution Code Removed}
The result is:
Cost after iteration 0: 0.693147
All tests passed!
If you pass all the tests, run the following cell to train your model.
Cost Result:
Cost after iteration 0: 0.693147
Cost after iteration 100: 0.584508
Cost after iteration 200: 0.466949
Cost after iteration 300: 0.376007
Cost after iteration 400: 0.331463
Cost after iteration 500: 0.303273
Cost after iteration 600: 0.279880
Cost after iteration 700: 0.260042
Cost after iteration 800: 0.242941
Cost after iteration 900: 0.228004
Cost after iteration 1000: 0.214820
Cost after iteration 1100: 0.203078
Cost after iteration 1200: 0.192544
Cost after iteration 1300: 0.183033
Cost after iteration 1400: 0.174399
Cost after iteration 1500: 0.166521
Cost after iteration 1600: 0.159305
Cost after iteration 1700: 0.152667
Cost after iteration 1800: 0.146542
Cost after iteration 1900: 0.140872
train accuracy: 99.04306220095694 test accuracy: 70.0
Thank you again both.
Congrats on passing all the tests. Before submitting your assignment, delete all the extra lines of code which you added for debugging purposes. Also, sharing your code is not allowed, so, do not post your code in a public thread.
Best,
Saif.
Will do. It was a bit of struggle. Couldn’t have done it with your help!
It’s great news that you have everything working now. With some lessons learned in the process, no doubt.
Just one minor point: you’re still hard-coding the value of print_cost on the call to optimize. That doesn’t really do any harm and the grader doesn’t care, but there is really no need to do that because the real training cells that invoke model provide the value as True.
Thank you for the feedback I will update the hardcoding you pointed out.
