In the course, we use a linear regression as input of the activation function. I understant that we need the output to be 0 or 1 (hence, the sigmoid) but why do we use linear regression ? Can’t we use the input directly with a parametrs to optimize in the exponential arguments and a constant added to the sigmoid ? Moreover, does this mean we could use a quadratic function instead of a linear regression with 3 parameters to optimize ?
Which course is this referring to? The reason is because its called linear (uses a linear function and than the regression part using the sigmoid). You could use a quadratic function as input (it is still linear though) and then the sigmoid! It just makes the whole computations more complex.
Do not confuse “linear regression” with “a linear activation”. They are different concepts.
TensorFlow has a nice feature where it will automatically generate classifications if the model uses a linear activation in the output and you specify the “from_logits = True” parameter.
Their implementation of this is much more computationally efficient than if you use a sigmoid activation followed by a comparison threshold.