It is an interesting question! Of course in pure math terms, the output of sigmoid can never exactly equal 0 or 1. But we are dealing with the pathetic limitations of finite floating point representations here, not the abstract beauty of \mathbb{R}, so this can actually happen. Here’s a thread from DLS which discusses this point.
I’m not familiar with the assignments in MLS, so it’s also possible that I’m missing something here and you should not be hitting this case. But the above link applies in the general case …