Train/Dev/Test explanation

[kampamocha]

Hi @Noam_Mizrachi,

As @gautamaltman said, the parameters are what the model learns during training (e.g. weights), while the hyperparameters are those you set up before the training (e.g. number of neurons).

The process can be done in this way:

1. Set up specific hyperparameters and train with the training set.
2. Evaluate the trained model with the dev set. If the performance is not good enough, you can tweak your hyperparameters and train again (step 1), until you are satisfied with the performance on the dev set.
3. With your final model, then you evaluate the test set to have an estimation of the performance with unseen data.

Here you have some previous discussions about this topic that may help you:

Early stopping - why dev set error starts to increase?

So I train on a train set, check on a dev set, and train on a train set after changing something? I still don’t understand the chart with iterations. If I’m only evaluating my model on the dev set, I would feed my NN with inputs, run forward propagation, and check error (Y hat vs. true Y). There will be no multiple iterations - just a single iteration. My understanding is that a single iteration is forward propagation + back propagation. Multiple iterations are run in the training phase (each …

Differences in Dev and Test set

Hi, I am slightly confused by their differences in definition and use. My understanding is that dev set is a set of data used to tune hyperparameters, compare the difference in algorithms/models we trained (to see which performs better)? If we are comparing 2 algorithms does that mean we would split the train set further into 2 parts to train 2 algorithms or should we train both on the same training data. Prof Ng mentioned that dev sets are used to hold out cross validation. What does that mean…