Tweaking Hyper parameters vs getting more training data

Hi there,

I often struggle to answer a question. I almost always have very little data at my exposure and most often I end up building an underfit model. If I try to increase the complexity of the network, I see overfitting even after adding regularization in the network. So my question would be is there an approach that could tell me the results that I see are due to low data or due to non-optimal hyper parameters or some issues with my network.

Thanks