Programming Assignment: Dinosaur Island-Character-Level Language Modeling gradient clipping in practice

Course Q&A Deep Learning Specialization Sequence Models
1

One of the goals of this assignement is to apply gradient clipping in a model built almost from scracthc.

how would one apply gradient clipping in practices e.g. plain vanilla fully connected neural network?

tf.clip_by_value | TensorFlow Core v2.7.0

tf.clip_by_norm | TensorFlow Core v2.7.0

do you call this clipping on the optimizer before you compile a model?

2

tf.clip_by_value makes sense based on the assignment. You could try both for your problem at hand.

Clipping can be implemented as a custom layer tf.keras.layers.Layer  |  TensorFlow Core v2.7.0 so that it becomes part of the model.