Hi, I have a question about the Baseline Performance mentioned in Course 2 Week 3.
One of the areas we can establish a baseline performance is to leverage the previous model implementation error. But I’m confused about what error we should use as the baseline performance. Should we use the J_train, J_cv, or J_test from the previous implementation as the baseline?
For example, I have an ML model - call it model v1, and I’m trying to improve it using model v2. Which one (J_train, J_cv, or J_test from model v1) should I use as the baseline?
A further question, if v1 has already been built for 1 year, should I get the new data to recalibrate before getting the baseline OR just use the old data to measure the baseline error?
In general, what’s the best practice to retrieve baseline performance from the previously built model?
J_test is the best metric for comparison, since it gives you a metric for how well your model makes predictions on new data.
When to add new data and create a new model is totally up to you. It might be based on whether you’re still getting sufficiently good performance from the original model. It also depends on how much effort or energy is required to train the system. Computer resources might be expensive, it depends on how large the model is.
Hi
in addition to what TMosh said
As we use validation (J_CV) to compute and set hyper parameters of overfitting and underfitting for example like lambda , degree of input(polynomial degree) so when we reach to the hyper parameters that have a small (J_CV) ,so we have a some confidence that when we use parameters that had learnt from model and hyper parameter that we set by calculate (J_CV) so we give the test set to check the global parameters the model if we deploy it …so for summary
train set use to set parameters of model
validation set use to set hyper parameters
test set use to check the result of all parameters that we have set from (train & validation ) set to doing deployment
please feel free to ask any questions,
Thanks,
Abdelrahman