Dinosaurus_Island_Character_level_language_model optimize

Exercise 3 - optimize

Implement the optimization process (one step of stochastic gradient descent).

The following functions are provided:

def rnn_forward(X, Y, a_prev, parameters):
    """ Performs the forward propagation through the RNN and computes the cross-entropy loss.
    It returns the loss' value as well as a "cache" storing values to be used in backpropagation."""
    return loss, cache

def rnn_backward(X, Y, parameters, cache):
    """ Performs the backward propagation through time to compute the gradients of the loss with respect
    to the parameters. It returns also all the hidden states."""
    return gradients, a

def update_parameters(parameters, gradients, learning_rate):
    """ Updates parameters using the Gradient Descent Update Rule."""
    return parameters

In the optimize function, the instruction says
def optimize(X, Y, a_prev, parameters, learning_rate = 0.01):
Execute one step of the optimization to train the model.

X -- list of integers, where each integer is a number that maps to a character in the vocabulary.
Y -- list of integers, exactly the same as X but shifted one index to the left.

and in the test part, X and Y is given as:
X = [12, 3, 5, 11, 22, 3]
Y = [4, 14, 11, 22, 25, 26]

according to my understanding, X and Y should be like:
X = [12, 3, 5, 11, 22, 3]
Y = [3, 5, 11, 22, 3, …]

would you please explain the error in my comprehension?

By the way, can I obtain the exact codes of these funds?
rnn_forward, rnn_backward and update parameters?


1 Like

Hi @maxma
What is the version of your notebook? I could not find the pointed functions.

1 Like

Which assignment are you working on?

In the notebook in your thread title, Exercise 3 is the “lstm_cell_forward()” function, and there is no “optimize” in this notebook.

As Carlos and Tom have said, we aren’t sure which notebook you are actually talking about here, but in general any functions used that are not in the notebook will be imported. It could be from a python library or it could be from a local file accompanying the notebook. There is a topic about this on the DLS FAQ Thread.

sorry, it is this assignment: Dinosaurus_Island_Character_level_language_model

You are right that Y should be the same as X but shifted one index to the left. However, our test cases are just random numbers to test your code. It’s not real data. You can ignore this.