# Activation function definition **URL:** \ **Category:** Neural Networks and Deep Learning\ **Tags:** week-module-1, week-module-2, ai-discussions, dl-ai-learning-platform\ **Created:** [April 16, 2026, 10:07pm UTC](https://community.deeplearning.ai/t/activation-function-definition/891557 "2026-04-16T22:07:53Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 16, 2026, 10:07pm UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/1 "2026-04-16T22:07:53Z")
Hello, for the following activation function (excuse my poor notation)… g(w dot x + b), where g is the sigmoid function … is the activation function really _ **just** _ the sigmoid function (as I’ve heard it referred to)? Or does the activation function refer to the full composite function (i.e. it also includes the sigmoid function’s input function of w dot x + b)? ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [April 16, 2026, 10:17pm UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/2 "2026-04-16T22:17:25Z")
The sigmoid function itself is 1/(1+exp(-z)) In the case of logistic regression, we use z = w\*x + b. ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 17, 2026, 3:20am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/3 "2026-04-17T03:20:57Z")
Thank you, but my question is what the exact definition of “activation function” is. Using the example you gave, is the activation function strictly the sigmoid part of it: 1/(1+exp(-z)), where z can be anything, OR is the activation function the sigmoid function with its inputs specified: 1/(1+exp(-(w\*x + b))) ---
**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\ **Post date:** [April 17, 2026, 3:26am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/4 "2026-04-17T03:26:42Z")
> [@Bill\_Klinkow](#): > > is the activation function strictly the sigmoid part of it: 1/(1+exp(-z)), where z can be anything Yes! The activation function is just the sigmoid part of it, and z can be anything. z = w dot x + b is the output of a layer, and we apply an activation function to that output. Cheers! ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 18, 2026, 11:35pm UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/5 "2026-04-18T23:35:38Z")
I’m not sure I follow: > z = w dot x + b is the output of a layer My understanding (so far) for, say, a dense layer with 2 units, is that the “output” of the layer (also called an “activation vector”?) would be calculated: ```auto [ 1/(1+exp(-(w1*x + b1))) 1/(1+exp(-(w2*x + b2))) ] ``` which includes the dot product you referenced AND a sigmoid function. But, in any case, > The activation function is just the sigmoid part of it makes sense to me, so thank you! ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [April 18, 2026, 11:53pm UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/6 "2026-04-18T23:53:50Z")
Each neural unit includes the activation function. ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 19, 2026, 1:11am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/7 "2026-04-19T01:11:18Z")
> Each neural unit includes the activation function. Indeed. My latest comment was asking for an explanation of why [rmwkwok](https://community.deeplearning.ai/u/rmwkwok) described > z = w dot x + b as the **output** of a layer. My understanding is that the **output** of a layer is computed using that equation AND the activation function. Alternatively, if looking at just **z = w dot x + b** for a layer output, it would simply be the **x** (that’s the output of a previous layer). Either way, **z = w dot x + b** doesn’t describe the output of a layer as far as I can see. ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [April 19, 2026, 1:24am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/8 "2026-04-19T01:24:16Z")
(w dot x + b) would be the output of a linear layer, with no activation function. ---
**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\ **Post date:** [April 19, 2026, 1:39am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/9 "2026-04-19T01:39:14Z")
Hello @Bill_Klinkow, If we look at this definition of tensorflow’s dense layer ([link](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense)): ![image](https://global.discourse-cdn.com/dlai/original/3X/e/6/e6b925059e4ebd7fd9075790c5973319f4e38e72.png) we can see that a Dense layer can come without an activation (which is equivalent to having “`linear`” as its activation). Furthermore, we can also find this standalone [activation layer](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Activation) in the package. Sometimes, people consider activation as part of a dense layer, and sometimes, people consider them two separate layers. That explains my choice of words, but both choices are understandable to me 😉 Cheers, Raymond ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [April 19, 2026, 1:44am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/10 "2026-04-19T01:44:05Z")
It’s really a minor point whether the activation is considered part of the unit output. A nonlinear activation is necessary either way. ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 19, 2026, 3:17am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/11 "2026-04-19T03:17:37Z")
Thanks Raymond! The tf Dense class was part of my confusion - in the course I’m taking there is an example that uses the Dense class and sets activation to “linear”, but described the layer as having no activation function. Thanks for clarifying using that exact example! It’s too bad that having no activation function is sometimes referred to as linear activation, both in speech and in the TensorFlow library, but now I know! (And I _guess_ it kind-of makes sense.) Also the rest of your comment really helped add context. Cheers ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 19, 2026, 3:39am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/13 "2026-04-19T03:39:58Z")
> (w dot x + b) would be the output of a linear layer, with no activation function. True, I was confused because the example had an activation function. ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 23, 2026, 5:43am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/14 "2026-04-23T05:43:43Z")
Follow up, I finally learned why a linear activation function is technically different from, but equivalent to, no activation function. Linear activation is f(x) = x, i.e. no change. Sweet, it all makes sense. Thank again for your help. ---
**Author:** ![Bill\_Klinkow](https://avatars.discourse-cdn.com/v4/letter/b/ecc23a/32.png) [@Bill\_Klinkow](https://community.deeplearning.ai/u/Bill_Klinkow)\ **Post date:** [April 23, 2026, 5:49am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/15 "2026-04-23T05:49:55Z")
Follow up: I learned the linear activation function is f(x) = x. So performing linear activation doesn’t change anything. That’s why “no activation function” and “linear activation function” are used interchangeably - they produce equivalent results. ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [April 23, 2026, 6:01am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/16 "2026-04-23T06:01:03Z")
Your function f(x) = x is the degenerate case of f(x) = w\*x + b, where w = 1 and b=0. Both are considered linear functions. ---
**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\ **Post date:** [April 28, 2026, 1:10am UTC](https://community.deeplearning.ai/t/activation-function-definition/891557/17 "2026-04-28T01:10:25Z")
You are welcome, @Bill_Klinkow! It’s good to know you made progress! Keep learning, and cheers, Raymond