How can i find that witch function is Non-linear

Deep Learning Specialization
Someone told me that nonlinear is something that can’t fit in this equation f(a) + f(b) = f(a + b), but that is incorrect because tanh is nonlinear and it also works with this function.
Can you tell me how to find out if an equation is nonlinear or linear?

An equation is considered linear if it satisfies 2 conditions:

  • Additivity: f(x + y) = f(x) + f(y)
  • Homogeneity: f(αx) = α f(x) for all α

tanh does not satisfy these properties, so it is not considered a linear function! In deep learning, we use non-linear activation functions to learn complex non-linear relationships within the data.


Thanks a lot.

1 Like

No, it doesn’t. Run the following code:

a = np.tanh(1.)
b = np.tanh(2.)
c = np.tanh(3.)
print(f"a + b = {a + b}“)
print(f"c = {c}”)

The output of that code is the following:

a + b = 1.7256217360315818
c = 0.9950547536867305

As Alireza said above, tanh is a non-linear function.

Note that one other way that it is common to see the definition of linearity stated is that the function f is linear if the following is true for all scalar values a and b and all input values x and y from the domain of f:

f(ax + by) = af(x) + bf(y)

That covers both of the conditions in Alireza’s formulation in one shot.