Loss, math domain error

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

The sigmoid function defined below

def sigmoid(z):
    return 1.0 / (1.0 + np.exp(-z))

returns 1.0 when z is greater than 36. In this case log(1-sigmoid(z)) raises “Math domain” error.
This is due to python’s precision for floating point numbers.
When training, and printing out the cost, it crashes due to the reason I mentioned.
Anyone knows how to solve the problem?

2

Hello @Alireza_Kamyab

Welcome to the community.

Are you doing log(1-z) or log(1 - sigmoid(z))?

3

srry, I am doing 1-sigmoid(z).

4

Can you share your notebook through DM (click on my name → Message → Attach the notebook)

5

When using the sigmoid() activation, typically the system will learn weight values that avoid generating large magnitude z values that cause this issue.

So in practice it isn’t really an issue.