If we consider more than two features then what kind of shape we are going to have for mean squared error cost function.

If we consider more than two features mean squared error cost function will also have a bow shape with one local minima.

Yes, the linear regression cost function is always convex, regardless of the number of features.

We have seen the cost function plotted w.rt (w,b). When we have n features, the cost function will be plotted over (w1,w2,…wn,b) dimensions. But the Shape will still be convex with a single minima (For Linear Regression with Squared Error loss function).