Course 1 week 2: bias 'b' value type in model vs initialize_with_zeros

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hi DeepLearning.ai team,
Hope you are doing good. Thank you for the updated assignments with strict validation of types.

I got the following error in the model_test while initializing bias b using initialize_with_zeros:

Error: Data type mismatch in variable b. Got <class ‘float’> but expected type <class ‘numpy.float64’>
Error: Wrong shape for variable b.

The initialization of weights and bias inside initialize_with_zeros was:

b = 0 or(incorrectly typed and should have been b = 0.) or b = np.float(0)

This was due to the assertion in initialize_with_zeros:

assert type(b) == float

then, inside the model, i overwrote the value of b from initialize_with_zeros with

b = np.float64(0)

This helped with test case.

Can you please guide on any of my incorrect assignment/usage?

Update history:

  1. strike b = 0 (should have typed as b = 0.
2 Likes
2

Hi thiyanesh.
You want to make sure b is a float, not an integer when you specify in the ‘initialize_with_zeros’, that is, you want ‘0.’ but not ‘0’.
Does that help you?

3

Hi thiyanesh,

Suki is right. You have to make sure that b is a float in this case.
If you write b = 0, you get an integer.
You can try to print the type of your object with:

print(type(b))

What type do you get when you initialize b as np.float(0)?

Best,

Anamarija

1 Like
4

@suki @afofonjka ,
Thank you for the reply.
I have updated the initial post to delete incorrect b = 0 usage (b = 0 or, instead of 0., i typed 0 in the post)

In actual code, b is initialized as b = np.float(0)

print(type(b)) returned <class 'float'>

The primary issue was

  1. initialize_with_zeros has assert type(b) == float (so b = np.float(0) works fine)
  2. model_test threw the following error when model was initialized by invoking initialize_with_zeros.

Error: Data type mismatch in variable b. Got <class ‘float’> but expected type <class ‘numpy.float64’>

  1. so, had to explicitly assign b = np.float64(0) inside model
5

In the interest of better expressiveness of your code, and to avoid tying yourself to types defined in one specific Python package, I would go with @suki, and initialize as b = 0. .

6

@yanivh,
Thank you for the suggestion.(i had tried 0. too and got the same error)

Please find more details:

  1. Assigning b = 0. or b = np.float(0) behaves predictably in initialize_with_zeros(). The method execution succeeds with message All tests passed.
  2. When initialize_with_zeros() was invoked in model() to initialize w and b, as

w, b = initialize_with_zeros(X_train.shape[0])

then model_test complains as follows

Error: Data type mismatch in variable b. Got <class ‘float’> but expected type <class ‘numpy.float64’>

Currently, i have added the following line in model to initialize b to fix the test.

b = np.float64(0)

Looking at the source code of model_test:

expected_output = {'costs': [np.array(0.69314718)],
 'Y_prediction_test': np.array([[1., 1., 1.]]),
 'Y_prediction_train': np.array([[1., 1., 1.]]),
 'w': np.array([[ 0.00194946],
        [-0.0005046 ],
        [ 0.00083111],
        [ 0.00143207]]),
 'b': np.float64(0.0008311888169172717),
 'learning_rate': 0.0001,
 'num_iterations': 50}

I assume, this expectation in model_test might be the cause 'b': np.float64(0.0008311888169172717), when initializing to b = 0. or b = np.float(0)

More specifically, the source code of multiple_test contains:

            if test_case['name'] == "datatype_check":
                success += datatype_check(test_case['expected'], target_answer, test_case['error'])

Can you please correct if my understanding is incorrect?

7

@thiyanesh
Now I understand what you are having trouble with.
Hint:
The 'b': np.float64(0.0008311888169172717) you are seeing in the expected value is the ‘updated’ b after the backpropagation.
Hope that helps?
Let me know if you have any questions.

8

@suki
Thank you for the reply.

Yes, i was able to get the model_test by initializing b = np.float64(0)
It’s not important, still wondering, are there anyone else facing the same issue or it’s a specific issue?

Thanks again.

9

Dear @thiyanesh I was seeing something similar to what you were experincing yesterday. Then this morning my notebook environment somehow resetted and lost his checkpoints. After redoing from scratch, I am not seeing any kind of issue.

10

@crisrise,
Thank you for taking your time to check and reply on this topic.

It might be helpful to check the code of model_test to confirm the behavior. can you please add a new cell above or below model_test and add the following code

import inspect
print(inspect.getsource(model_test))

If the current code contains 'b': np.float(, then it might explain why it works now. (it used to 'b': np.float64( for me).

11

Dear @thiyanesh in my case expected_output now contains ‘b’: np.float64(…, not sure what was before.

12

Hi all, I’ve just been hit by this issue as well. There is (or was) a discrepancy between the expected types for b in the tests.

13

@Colin ,
We can explicitly assign the value with expected type to temporarily submit the assignment.

14

Thanks, that is what I had done. But I think that the tutorial should be fixed : people without a python / programming background won’t be able to figure this out.

15

Hi everyone!

I will try my best to clarify what should be expected of data types :slight_smile:

Just as @suki hinted earlier, the data type of b is changed in backprop, i.e., inside the optimize method:

To summarize, in model(...):

  1. The data type of b should be float after the call to initialize_with_zeros(...).
  2. The data type of b should be numpy.float64 after it has been reassigned with a value from the params dictionary, which in turn is an output of the `optimize(…), i.e., after backprop has finished.

However, you should not really need to think about the data types if you implemented both initialize_with_zeros(...) and optimize(...) the way we thought you would :relaxed:

If you think you are all set, you can verify by checking the data types with print statements (in model(...)):

# (≈ 1 line of code)   
# initialize parameters with zeros 
# w, b = ...
print(type(b))

#(≈ 1 line of code)
# Gradient descent 
# parameters, grads, costs = ...

# Retrieve parameters w and b from dictionary "parameters"
# w = ...
# b = ...
print(type(b))

# Predict test/train set examples (≈ 2 lines of code)
# Y_prediction_test = ...
# Y_prediction_train = ...

After model_test(model) has executed, you should see:

<class 'float'>
<class 'numpy.float64'>
<class 'float'>
<class 'numpy.float64'>
<class 'float'>
<class 'numpy.float64'>
 All tests passed.

If not, I recommend you to check your implementation of propagate(...):

# BACKWARD PROPAGATION (TO FIND GRAD)
#(≈ 2 lines of code)
# dw = ...
# db = ...
print(type(db))

The data type of db should be numpy.float64.

For those of you who have this problem, can you find the code where it starts to go wrong?

We will do our best to help find the error :slight_smile:

16

Hello, I am having issues with the initialize parameter exercise 4.2. I first had a problem with the parameter b type, and followed recommendations in this post. However, the following error shows up (below). I spent time trying to figure it out, but I have no clue how to solve it.

<class ‘float’>
w = [0. 0.]
b = 0.0
<class ‘float’>
<class ‘float’>
<class ‘float’>
Error: Wrong shape for variable 0.
<class ‘float’>
2 Tests passed
1 Tests failed
AssertionError Traceback (most recent call last)
in
6 print ("b = " + str(b))
7
----> 8 initialize_with_zeros_test(initialize_with_zeros)

~/work/release/W2A2/public_tests.py in initialize_with_zeros_test(target)
59 ]
60
—> 61 multiple_test(test_cases, target)
62
63 def propagate_test(target):

~/work/release/W2A2/test_utils.py in multiple_test(test_cases, target)
140 print(’\033[92m’, success," Tests passed")
141 print(’\033[91m’, len(test_cases) - success, " Tests failed")
→ 142 raise AssertionError(“Not all tests were passed for {}. Check your equations and avoid using global variables inside the function.”.format(target.name))
143

AssertionError: Not all tests were passed for initialize_with_zeros. Check your equations and avoid using global variables inside the function.

17

@jonaslalin ,

Assuming we are discussing about the same version of the notebook (not sure how to verify),

  1. def initialize_with_zeros should initialize b = np.float(0) or b = 0. to honor the initialize_with_zeros_test test (expected as float).
  2. print(type(b)) is <class 'float'>
  3. propagate_test still has type of b as <class 'float'>
  4. propagate has type of db as <class 'numpy.float64'>
  5. as per db type, optimize makes a deep copy of b as b = copy.deepcopy(b)
  6. this updated b will become float64 due to b = b - learning_rate * db (since db is float64)
  7. this updated b is contained within params and returned to caller

Now looking at the actual cause of the error

  1. inside def model , b is initialized using w, b = initialize_with_zeros(X_train.shape[0]) method
  2. so, type(b) will be float just after initialization
  3. subsequent optimize call will return params that contains b as float64
  4. but the actual primitive b inside model is still a float
  5. the following response of model does not use the b from params[“b”] and directly returns the initialized b (same with w)
d = {"costs": costs,

     "Y_prediction_test": Y_prediction_test, 
     "Y_prediction_train" : Y_prediction_train, 
     "w" : w, 
     "b" : b,
     "learning_rate" : learning_rate,
     "num_iterations": num_iterations}
  1. the actual fix will be to update the returned d that contains b as "b" : params["b"] (also “w”)
d = {"costs": costs,

     "Y_prediction_test": Y_prediction_test, 
     "Y_prediction_train" : Y_prediction_train, 
     "w" : params["w"], 
     "b" : params["b"],
     "learning_rate" : learning_rate,
     "num_iterations": num_iterations}

As mentioned in my original post, the expected output will match the returned value

expected_output = {‘costs’: [np.array(0.69314718)],
‘Y_prediction_test’: np.array([[1., 1., 1.]]),
‘Y_prediction_train’: np.array([[1., 1., 1.]]),
‘w’: np.array([[ 0.00194946],
[-0.0005046 ],
[ 0.00083111],
[ 0.00143207]]),
‘b’: np.float64(0.0008311888169172717),
‘learning_rate’: 0.0001,
‘num_iterations’: 50}

Hopeful this clarifies the perspective.

18

@Enrique ,

4.2 works fine for me.

Please refer my above post for section 8. If your version of notebook is same as mine, then updating def model’s value of b inside d should fix the test case.

Existing:

d = {"costs": costs,

     "Y_prediction_test": Y_prediction_test, 
     "Y_prediction_train" : Y_prediction_train, 
     "w" : w, 
     "b" : params["b"],
     "learning_rate" : learning_rate,
     "num_iterations": num_iterations}

to

d = {"costs": costs,

     "Y_prediction_test": Y_prediction_test, 
     "Y_prediction_train" : Y_prediction_train, 
     "w" : params["w"], 
     "b" : params["b"],
     "learning_rate" : learning_rate,
     "num_iterations": num_iterations}

A single line change of "b" : b, to "b" : params["b"], might mostly fix the issue.(similarly for “w” too)

1 Like
19

Jonas, thank you for your quick reply. Unfortunately, I haven’t reached that part of the assignment yet. I am literally in Section 4.3, trying to initialize the parameters (perhaps I should initiate a new post?)
I have initialized w and b correctly. Both variables w, b are a floating array and a floating bias, respectively. but when I run the last part of section 4.3 (In[14] in my notebook), the error shows up when computing the
‘initialize_with_zeros_test(initialize_with_zeros)’
I haven’t found anything like it in the forums, and I really do not understand what the problem is. The thing is that I cannot continue the assignment due to this very small -and apparently unimportant- issue.
Thank you for your help.

20

@Enrique ,
Can you please share your complete code of section 4.2 (initialize_with_zeros) and also the initialize_with_zeros_test part

Further, you can actually check the code of initialize_with_zeros_test by adding a new cell and executing the following

import inspect
print(inspect.getsource(initialize_with_zeros_test))

My notebook has expected_output_b = 0. inside initialize_with_zeros_test