I am trying to calculate L2 loss between features maps of the deepest classifier and each shallow classifier in a CNN network. I am having trouble understanding how this loss should be fed back to shallow feature maps
Hi @Samhita_V
This is not a question related to a course’s exercise and so, I am assuming a general question.
Speaking generically, you can calculate the L2 loss between feature maps and propagate this loss back to the shallow feature maps. Here is a snippet code representation of what I am trying to say:
input = ...
target = ...
# create a criterion for the L2 loss
criterion = tf.keras.losses.MeanSquaredError()
# compute the loss between the input and target
loss = criterion(input, target)
# get the model optimizer
optimizer = ...
# compute the gradients
grads = tf.GradientTape().gradient(loss, model.trainable_variables)
# apply the gradients to update the weights
optimizer.apply_gradients(zip(grads, model.trainable_variables))
Keep learning!
Can you explain what is happeneing here? Can we not apply model.compile directly and use custom loss?
grads = tf.GradientTape().gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
Also since input and target will have different shapes those will have to be adjusted too?
1 Like
The question was to try to understand how the loss works. So, in the snippet code you are manually computing and applying gradients. Essentially, the snippet code is doing what model.compile() and model.fit() do together, but with more control over the individual steps, which makes more easy to understand the process. You can get more details in the link below.
Keep learning!
alright thank you sm!
Hello @Samhita_V
Is your question more related to the TensorFlow Advanced Techniques Specialisation?
As I can see based on the mention of gradient Tape
yes it was!