Exercise 3 - optimize
Implement the optimization process (one step of stochastic gradient descent).
The following functions are provided:
def rnn_forward(X, Y, a_prev, parameters):
""" Performs the forward propagation through the RNN and computes the cross-entropy loss.
It returns the loss' value as well as a "cache" storing values to be used in backpropagation."""
....
return loss, cache
def rnn_backward(X, Y, parameters, cache):
""" Performs the backward propagation through time to compute the gradients of the loss with respect
to the parameters. It returns also all the hidden states."""
...
return gradients, a
def update_parameters(parameters, gradients, learning_rate):
""" Updates parameters using the Gradient Descent Update Rule."""
...
return parameters
In the optimize function, the instruction says
def optimize(X, Y, a_prev, parameters, learning_rate = 0.01):
“”"
Execute one step of the optimization to train the model.
Arguments:
X -- list of integers, where each integer is a number that maps to a character in the vocabulary.
Y -- list of integers, exactly the same as X but shifted one index to the left.
and in the test part, X and Y is given as:
X = [12, 3, 5, 11, 22, 3]
Y = [4, 14, 11, 22, 25, 26]
according to my understanding, X and Y should be like:
X = [12, 3, 5, 11, 22, 3]
Y = [3, 5, 11, 22, 3, …]
would you please explain the error in my comprehension?
By the way, can I obtain the exact codes of these funds?
rnn_forward, rnn_backward and update parameters?
thanks