I’m sure I’m missing something obvious here, but I’ve been beating my head against the wall (metaphorically) for a couple of weeks on this.
I understand transfer learning; I’ve not only used it in the past, but have taught it (using Matlab, on the Alexnet model).
I found another post on this topic, but the discussion there did not enlighten me.
The previous use of the augmenter required iteration over a set of images. Clearly, we want to apply it only to the training images, yet there does not appear to be any partitioning of the inputs into training, validation, and test sets in the function parameters.
Does application of the augmenter require iteration over each image? Is the augmentation somehow supressed automatically during inference and only applied during training? Should the right hand side of the "x = " be a list comprehension?
As I said earlier, I’m convinced I’m missing something obvious.
See section 1.1 where train_dataset and validation_dataset are defined. Since we provide the same parameters except subset, it’s the same as creating non-overlapping training and validation sets of split sizes 80% and 20%. image_dataset_from_directory is responsible for iterating over images and providing input to the caller.
model.fit will internally call augmentation layers. These augmentation layers will perform their task only while training (see RandomFlip):
A preprocessing layer which randomly flips images during training.
When model training is inactive, these augmentation layers will let the input pass through without any modifications.
One way to control the training state of the model is to use tf.keras.backend.set_learning_phase
1 Like
Thank you, balaji_ambresh, for responding to my question regarding whether the augmentation is applied during inference.
However, I’m still stuck! Do I iterate over all elements of input and apply the data augmenter? Would I use a list comprehension for this, or an explicit for loop? Do I need to call expand_dims before applying the data augmenter (I assume so, because that’s what was done in the example)?
The comment:
# apply data augmentation to the inputs
is puzzling, as no input is provided to the function; I realize we’re specifying a model here; how do I indicate what the input is?
A batch of input is of shape [batch_dim, height, width, channels]. If you’re manually feeding a single image, use expand_dims to add a dummy dimension before feeding to a layer. Vectorization and batching play an important role in deep learning libraries. So, you don’t have to feed a single image at a time to a NN.
Shifting attention to alpaca_model, observe the input_shape parameter of the base_model. Though this reflects the shape of a single example, tensorflow internally performs mini batching based on the call to model.fit based on batch_size parameter. Providing a Dataset or arrays of Xs and Ys is sufficient.
See this link on transfer learing. Notice how a single image is fed to the model using tf.expand_dims as shown here. Look at this section on add a classification head where the invocation of data_augmentation Sequential model is done without worrying about the batch dimension.
1 Like