Linear dependency and singularity

Course Q&A Math for Machine Learning Linear Algebra for Machine Learning and Data Sc...
1

My question is about the following lesson:
https://www.coursera.org/learn/machine-learning-linear-algebra/lecture/e5jZi/linear-dependence-and-independence specifically at 3:45/7:20 , where he says “this matrix and the system are singular”

He gave the example of three equations:
a=1
b=2
a+b=3

Even though rows are linearly dependent, but the system is NOT singular.
It is Non Singular - as there is unique solution to this i.e. a=1 and b=2.
So, why does the instructor say that it is singular(singular meaning not having unqiue solution)?

3

Because there are three variables in that system of equations. c is not included in any of the 3 equations, so it can have any value, meaning that there are an infinite number of solutions to that system of equations.

Here’s a slide from just a bit earlier in that same video which makes that point pretty clearly:

Screenshot 2025-01-31 at 2.11.57 PM
Screenshot 2025-01-31 at 2.11.57 PM2868×1928 472 KB

2 Likes
4

Thanks for your response. But it is still not clear to me.

  1. I didn’t understand the point of taking a c variable which doesn’t even exist in the system, meaning i can take d,e,f,g variables all having 0 value which doesn’t make any point. Am i missing a very obvious point here ?
  2. even after considering the third variable , ‘c’ in this case, we still can’t have infinite solutions, as the system still will have a unique solution where a=1, b=2 and c=0. Plugging in any other values for these variables won’t hold true for all the equations considered together.
5

You’re only given two equations. The third one is just the sum of the first two, so that doesn’t add new information.

You have three unknowns.

You can’t solve for three unknowns if you have only two equations.

6

It is just a question of definitions. In this lecture Prof Serrano is talking about the relationship between systems of linear equations and matrices. So he has to define the number of variables, which in turn determines the size of the matrices. He has chosen to use the case of systems of linear equations in 3 variables, so we have 3 x 3 matrices or 3 x 4 if you use the “augmented” matrices including the RHS terms of the equations. It’s been a while since I actually watched this lecture from the beginning, but I would bet you that he explains this in the earlier part of the lecture. If you are still feeling uncertain, it never hurts to watch the lectures more than once.

For question 2, yes, a = 1, b = 2 and c = 0 is one solution, but there are infinitely many because literally any value of c works. c = 1 works, so does c = 2, so does c = -42 * \pi * e^{2.505679} and so forth. That’s what he means by saying that considered as a system of equations in 3 variables, it has an infinite number of solutions and the associated 3 x 3 matrix is singular.