# (Tensorflow assignment )Why not compute A3 in forward\_prop function? **URL:** \ **Category:** Improving Deep Neural Networks: Hyperparameter tun\ **Tags:** coursera-platform\ **Created:** [April 25, 2021, 11:09am UTC](https://community.deeplearning.ai/t/tensorflow-assignment-why-not-compute-a3-in-forward-prop-function/1658 "2021-04-25T11:09:06Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![nikolafuse](https://avatars.discourse-cdn.com/v4/letter/n/4491bb/32.png) [@nikolafuse](https://community.deeplearning.ai/u/nikolafuse)\ **Post date:** [April 25, 2021, 11:09am UTC](https://community.deeplearning.ai/t/tensorflow-assignment-why-not-compute-a3-in-forward-prop-function/1658/1 "2021-04-25T11:09:06Z")
Why we just compute A1 and A2 with relu function and return fuction with Z3 and compute its cost. Why not A3 with _softmax classifier_ since we are predict multi-class labels. Thanks in advance. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [April 25, 2021, 5:32pm UTC](https://community.deeplearning.ai/t/tensorflow-assignment-why-not-compute-a3-in-forward-prop-function/1658/2 "2021-04-25T17:32:25Z")
That’s because they want you to use the _Binary\_crossentropy_ loss function with the _from\_logits = True_ argument, which causes the _sigmoid_ calculation to be incorporated in the loss computation. That is preferred because they can manage numerical stability better when they do the two together. E.g. dealing with problems with “saturated” _sigmoid_ output values. [Here’s the doc page for Binary\_crossentropy](https://www.tensorflow.org/api_docs/python/tf/keras/losses/binary_crossentropy). They don’t really explain what the _from\_logits = True_ argument does in the assignment, but they do literally write out the code for you in the instructions. Maybe it would help if they explained it a bit more. I’ll suggest that. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [April 26, 2021, 10:02pm UTC](https://community.deeplearning.ai/t/tensorflow-assignment-why-not-compute-a3-in-forward-prop-function/1658/3 "2021-04-26T22:02:01Z")
Although now that you mention it, this is a multi-class problem, so maybe the loss function should actually be _categorical\_crossentropy_ in which case it would be the _softmax_ calculation instead of _sigmoid_ that is being done internally. I can’t tell from the documentation if perhaps the binary version is smart enough to handle either case. Of course you can think of _softmax_ as the multi-class generalization of _sigmoid_ and the math is very similar. I will investigate further. ---
**Author:** ![nramon](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/nramon/32/129097_2.png) [@nramon](https://community.deeplearning.ai/u/nramon)\ **Post date:** [April 29, 2021, 5:56pm UTC](https://community.deeplearning.ai/t/tensorflow-assignment-why-not-compute-a3-in-forward-prop-function/1658/4 "2021-04-29T17:56:45Z")
I was wondering about this too, @paulinpaloalto. _categorical\_crossentropy_ seemed like the obvious choice. _binary\_crossentropy_ works, but it would’ve made more sense had this been a multi-label classification problem. I guess what’s important is that @nikolafuse’s intuition is correct and the reason why the last activation (be it _softmax_ or _sigmoid_) doesn’t have to be explicitly computed was clearly explained. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [May 11, 2021, 4:32pm UTC](https://community.deeplearning.ai/t/tensorflow-assignment-why-not-compute-a3-in-forward-prop-function/1658/5 "2021-05-11T16:32:25Z")
Update on this issue: in the notebook, they don’t give you any logic to assess the results of the training, but I went ahead and cooked up my own version of that. It turns out that the training works fine and you get good prediction accuracy on both the train and test sets, but you really need to use Adam optimization (as the say in the instructions) as opposed to the SGD optimizer that they actually gave us in the code as written. I conclude from this that the _binary\_crossentropy_ logic in TF is smart enough to handle the fact that this is really not a binary classification and everything just works as expected.