# Cost function problem

**URL:** <https://community.deeplearning.ai/t/cost-function-problem/108146>\
**Category:** Neural Networks and Deep Learning\
**Tags:** coursera-platform\
**Created:** [March 13, 2022, 1:08pm UTC](https://community.deeplearning.ai/t/cost-function-problem/108146 "2022-03-13T13:08:28Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\
**Post date:** [March 13, 2022, 5:59pm UTC](https://community.deeplearning.ai/t/cost-function-problem/108146/2 "2022-03-13T17:59:30Z")

</div>

This is an interesting question! Of course the \hat{y} values are the output of sigmoid, so they can never be exactly 0 or 1 mathematically. But here we are dealing with the finite limitations of floating point representations, not the abstract beauty of \mathbb{R}, so it can actually happen that the values “saturate” and end up rounding to be exactly 0 or 1.

There are several ways to handle that:

You can test your \hat{y} values for exact equality to 0 or 1 and then slightly perturb the values before the cost computation:

```auto
A[A == 0.] = 1e-10
A[A == 1.] = 1. - 1e-10

```

You can also use _isinf()_ and _isnan()_ to replace any saturated values that happen after the fact, although that’s a bit more code since you need to catch the bad values while the cost is still the “loss” in vector form.

`loss[np.isnan(loss) | np.isneginf(loss)] = 42.`

You could also replace the non-numeric values with 0., depending on whether the saturated value is correct or not (saturates to 1 when the actual label is 1). If the label is 0 and the prediction saturates to 1, then those cases represent a big error that should be punished pretty severely by the loss function. Of course the actual J value doesn’t really affect the gradients in any case: the derivatives are calculated separately.

You can look up the documentation for numpy _isnan()_ and _isneginf()_. There are two cases to worry about:

If the \hat{y} is 1 and the y is 0, then you get 1 \* -\infty for the (1 - Y) term, which is -\infty. But if \hat{y} is 1 and the y is 1, then you get 0 \* -\infty for the (1 - Y) term and that is NaN (not a number). Of course you have the same cases in the opposite order for the Y term, when you hit \hat{y} = 0.

---

_[View the full topic](https://community.deeplearning.ai/t/cost-function-problem/108146)._