Video : A geometric notion of singularity

Course Q&A Math for Machine Learning Linear Algebra for Machine Learning and Data Sc...
1

The instructor points out that the coefficients to the right side of the system of equations do not matter in determining the singularity of the system. And then he shows how the lines are translated if we make them 0.
In this case the contradictory case turned into redundant ,but how can we be sure if that will always happen?

2

Hi @vaibhavoutat!

I don’t know exactly what you mean by being sure that this will always happen, so I will assume that you want a formal argument to the fact that we can always turn a contradictory case into redundant. There are several ways to do so, but here there is one that only uses what you have seen so far in the course:

Generally, any linear combination of variables spans a line (resp. a plane in three dimensions) in the plane (resp. space). We know that given two lines, there are three mutually exclusive possibilities:

  1. They intersect at a unique point
  2. They do not intersect
  3. They intersect in infinite points (they are the “same” line)

Note that we say that two lines are parallel if condition 2 or 3 are satisfied. In the case of a line, the constant term will dictate how far the line is to the origin and we can always shift the line to pass through 0.

Formally, you can write a system of equations as

a_1x + b_1y = c_1 \\ a_2x + b_2 y = c_2

If you draw a line, you can geometrically find that the slope of a line with given equation ax + by = x is just -\frac{b}{a}.

Two lines are parallel if and only if they have the same slope. In that case, if and only if

-\frac{b_1}{a_1} = -\frac{b_2}{a_2}

Which is equivalent to

a_2b_1 = a_1b_2

This tells us that a_2 = K a_1 and b_2 = K b_1 (in that case K = \frac{b_2}{b_1} = \frac{a_2}{a_1}). Thus, we can re-write the system to be:

a_1 x + b_1 x = c_1 \\ Ka_1 x + Kb_1 y = K(a_1 x + b_1y) = c_2

Therefore,

a_1 x + b_1 x = c_1 \\ a_1 x + b_1y= \frac{c_2}{K}

So what tells if this is contradictory or not is whether
c_1 = \frac{c_2}{K} or c_1 \neq \frac{c_2}{K}. In particular, setting c_1 = c_2 = 0 will always make such systems have infinite solutions, whereas if the above quantity is different, then it is a clear contradiction.

I hope that helps, let me know if you have any further questions.

Thanks,
Lucas