what should I try to solve first, high bias or high variance?
A first-order procedure (assuming you have enough training data):
- High bias means your model is too simple. Add some more complex features, so you get higher variance.
- Too-high variance means the model is over-fit. Add regularization.