How can I improve my model’s accuracy without adding more data

Course Q&A Machine Learning in Production
,
1

Hi everyone,

I’m currently working on a deep learning project as part of Week 3 of the course. I have a model that performs reasonably well on the training data, but its accuracy on validation/test data is lower than I’d like. My dataset is limited, so I cannot collect more data to improve performance.

I’d like advice on practical ways to increase model accuracy without adding new data. I’ve read about several potential strategies, but I’m not sure how to apply them effectively. Some approaches I’m considering include:

  • Data augmentation: Creating variations of existing data, but I’m unsure which techniques work best for my type of data.

  • Regularization: Using dropout, L1/L2 penalties, or batch normalization to prevent overfitting.

  • Hyperparameter tuning: Adjusting learning rate, batch size, number of layers, or activation functions.

  • Transfer learning: Leveraging pre-trained models to improve performance on small datasets.

  • Ensemble methods: Combining predictions from multiple models.

I’m looking for guidance on which of these methods tend to have the most impact in practice, and any tips on implementing them effectively. Examples, references, or personal experiences would be really helpful!

Thank you in advance for your advice.

2

Did you mean to post this question under Machine learning specialization ?

Do these links help?

  1. Link 1
  2. Link 2

Deep learning specialization does get into the details of addressing this high variance problem you’re facing. Please describe the project in detail.

1 Like
3

The main idea is for the model to learn from a variety of occurring scenarios (or better say learn from most of them), i.e., be trained on most occurring examples of your problem, and then use some form of “approximation” for unseen ones. It’s like filling in the missing gaps of a puzzle. Whether you do it in one or another method, it doesn’t matter as long as it’s effective in filling these missing gaps.