In the 3rd lab titled " Visualizing tweets and Logistic Regression models", we plot a line to show the cutoff between the positive and negative regions. The gray line is the where the dot product of theta and X equal to 0, meaning they are perpendicular. What is the intuition behind this?

You are correct that the dot product of two orthogonal vectors is zero.

But thatâ€™s not whatâ€™s happening here.

The dot product is an efficient way of computing the linear combination of the weights and features of an example. If that value is positive (or zero), you have a â€śTrueâ€ť result. If itâ€™s negative, you have a â€śFalseâ€ť result.

Thanks for your reply! Could you please elaborate on why when theta*x is positive, itâ€™s a true result, and false otherwise? Is this specific to logistic regression?

Yes, for logistic regression, that is by definition.

Remember that we take sigmoid(\theta \cdot x) and then that is interpreted as the probability of a â€śyesâ€ť answer. Note that sigmoid(0) = 0.5 and sigmoid is monotonic. So a positive input gives you a probability > 0.5 and a negative input to sigmoid gives you a probability < 0.5.

As Tom says, thatâ€™s the definition of Logistic Regression. The other important point is that this doesnâ€™t â€śjust happenâ€ť: we train the function so that it learns the coefficients (the elements of \theta) which give the best possible match to the training data we are using. Of course there is no guarantee that your data is â€ślinearly separableâ€ť, so Logistic Regression may not work well in all classification cases. In many cases, we will need a more complex decision boundary, so we will need more expressive functions. One approach to that is to graduate to Neural Networks, which will be covered in a later NLP course.

Thank you for the detailed explanation.

Thank you!