# W2\_A2\_Ex6 optimizing log(0) error **URL:** \ **Category:** Neural Networks and Deep Learning\ **Tags:** coursera-platform\ **Created:** [January 1, 2024, 10:54pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476 "2024-01-01T22:54:41Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Paul\_Durcek](https://avatars.discourse-cdn.com/v4/letter/p/ee59a6/32.png) [@Paul\_Durcek](https://community.deeplearning.ai/u/Paul_Durcek)\ **Post date:** [January 1, 2024, 10:54pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476/1 "2024-01-01T22:54:41Z")
I’m getting a NaN error on iteration 42 of the 2nd optimization loop in exercise 6. I’m trying to be careful to not reveal my answer publicly per forum guidelines. Debugging through it, it turns out that the `A` ends up being `1.00000000+e00` which causes cost function `np.log(1-A)` bit to call `np.log(0)` which is undefined. Is there a zero safe way to execute the `np.dot(1-Y, np.log(1-A).T)` bit with zero safety? This is the A value where things break: ```auto A:[[1.43030959e-29 6.31160658e-42 1.00000000e+00]] ``` ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 1, 2024, 11:38pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476/2 "2024-01-01T23:38:18Z")
Yes, you get NaN from the cost function if any of your \hat{y} values “saturate” and round to exactly 1. Of course if we were doing pure math here, the output of sigmoid could never be one, but it can happen in floating point. But the bigger point here is that this should not be happening. There must be something else wrong with your code. Are you sure you pass all the tests for the various functions? Please show the output of the `model` test and then what you get when you run the real training which is I assume where you’re seeing that NaN value. **Update:** Oh, sorry, exercise 6 is `optimize`, not `model`. Then the story is more straightforward: this most likely means your “update parameters” logic is incorrect. Things to check are that you subtracted rather than adding and that you are using the actual learning rate that was passed in, not hard-coding it to the default or the like. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 1, 2024, 11:41pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476/3 "2024-01-01T23:41:42Z")
But it is an interesting point that in a more general case, it might be necessary to defend yourself against this “saturation” problem. In 64 bit floats, it only takes z \> 36 to saturate sigmoid. In 32 floats it’s even less. [Here’s a thread](https://community.deeplearning.ai/t/cost-function-problem/108146/2) which discusses a general way to do that. Eventually we will convert to using packages like TensorFlow and they have sophisticated implementations for defending against this. ---
**Author:** ![Paul\_Durcek](https://avatars.discourse-cdn.com/v4/letter/p/ee59a6/32.png) [@Paul\_Durcek](https://community.deeplearning.ai/u/Paul_Durcek)\ **Post date:** [January 2, 2024, 6:37pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476/4 "2024-01-02T18:37:07Z")
Thanks for your response. Here are the outputs from the `propage` and `optimize` functions. All tests pass until the `optimize`. Potentially issue with `dw`|`db` calculation causing `w` and `b` to be too large? Propagate method output: ![propagate_output](https://global.discourse-cdn.com/dlai/original/3X/b/2/b222288cb5c8f9479eb18971796d189dffc79c6d.png) Optimize method outputs (standard and with additional details): ![optimize_output_error](https://global.discourse-cdn.com/dlai/original/3X/c/3/c3c51fafb6c7e2ba66b495d90eb550f9048f8540.png) ![optimize_output_cost_nan](https://global.discourse-cdn.com/dlai/original/3X/c/4/c40f1cdb08d35054c53c4078ddcb8760f51306c2.png) ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 2, 2024, 7:26pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476/5 "2024-01-02T19:26:56Z")
Yes, your values are clearly wrong in the `optimize` case. Did you compare your code to the math formulas for the “update parameters” step? I’ll bet you used + instead of - there. We subtract because the gradient points in the direction of most rapid _increase_ of the function (the cost in our case). What we want is to _decrease_ the cost, so we go in the opposite direction. ---
**Author:** ![Paul\_Durcek](https://avatars.discourse-cdn.com/v4/letter/p/ee59a6/32.png) [@Paul\_Durcek](https://community.deeplearning.ai/u/Paul_Durcek)\ **Post date:** [January 2, 2024, 8:24pm UTC](https://community.deeplearning.ai/t/w2-a2-ex6-optimizing-log-0-error/532476/6 "2024-01-02T20:24:10Z")
Thank you very much; I got it working. I had - in the `w` and `b` adjustment methods, but I was calling `sigmoid` on the `dw|db` values in the adjustment function instead of alpha hence ignoring the learning rate. This case can be closed.