It’s an interesting question with several levels to the answer:
From a purely mathematical standpoint, sigmoid(z) can never exactly equal 0 or 1. So you can’t end up with log(0) in the cross entropy loss function if you’re doing “pure math”.
But we don’t have the luxury of using “pure” math and the abstract beauty of \mathbb{R} here: we’re stuck with the limitations of finite floating point representations. In 32 bit or 64 bit floating point, the value of sigmoid can round to (“saturate” is the technical term) either 0 or 1 exactly. In that case, your loss function value can end up as either Inf or NaN.
But that doesn’t really wreck the convergence of back propagation, because we don’t actually use the final J or L value in that computation. We use the derivative of the loss function and you can see from the back prop formulas that the derivatives are fine if \hat{y} == 0 or 1.
If you want to make sure that your J values remain interpretable, you can add a defense mechanism against the saturation, as described on this thread. But as pointed out in the previous paragraph, that is not necessary in order for back propagation to continue working even in the saturation case. You can use accuracy instead of J as the proxy for how your convergence is working.