Correlation and scatter matrices help a lot as you mentioned e.g. linear regression. For example you could apply feature engineering techniques to describe the non-linearity already in your features and check and adapt your feature design based on your visualization. Here also a relevant thread for you: