# Question about week 3 assignment

**URL:** <https://community.deeplearning.ai/t/question-about-week-3-assignment/181053>\
**Category:** Improving Deep Neural Networks: Hyperparameter tun\
**Tags:** coursera-platform\
**Created:** [August 8, 2022, 9:52pm UTC](https://community.deeplearning.ai/t/question-about-week-3-assignment/181053 "2022-08-08T21:52:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mehdi\_sepehri](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mehdi\_sepehri](https://community.deeplearning.ai/u/mehdi_sepehri)\
**Post date:** [August 8, 2022, 9:52pm UTC](https://community.deeplearning.ai/t/question-about-week-3-assignment/181053/1 "2022-08-08T21:52:53Z")

</div>

Hi! I have a question about the forward\_propagation function in assignment week 3. Why do we return Z4 just without using any activation function for that layer?

---

<div class="post-metadata">

**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\
**Post date:** [August 8, 2022, 11:28pm UTC](https://community.deeplearning.ai/t/question-about-week-3-assignment/181053/2 "2022-08-08T23:28:08Z")

</div>

I forget whether Prof Ng discusses this anywhere in the lectures, but it turns out that the TF/Keras loss functions all support a selection of whether the inputs are “logits” (meaning the linear activation output) or actual “post activation” values. The argument that controls this is _from\_logits_ and it takes a Boolean value and defaults to _False_. Have a look at the documentation for [TF categorical cross entropy loss.](https://www.tensorflow.org/api_docs/python/tf/keras/losses/CategoricalCrossentropy) The reason they offer the _from\_logits = True_ mode is that it is more efficient and more “numerically stable” to compute the activation and the loss at the same time. For example, it becomes easier to deal with the “saturation” case in which some of the outputs turn out to be exactly 0 or exactly 1. That never happens from a “pure math” point of view, but we are dealing with finite floating point representations here, so it can actually happen. In those cases, the loss would be undefined if you don’t handle that case (NaN or Inf).

So Prof Ng always uses _from\_logits = True_ mode from this point forward. The activation function is still being applied, but it happens “inside” the loss function. The same option exists for binary cross entropy loss and the sparse version of categorical cross entropy.