Can someone please explain what the model() function does? How is it able to produce words? I am not understanding many things in this function.
Firstly, what does it mean to sample 7 dino names after each iteration? In one iteration we are computing loss–>computing gradients–>clip gradients–>update weights. From where are we getting 7 names?
How to get ‘idx’?
What I understand is we are using stochastic gradient descent, so after 1 iteration, we use 2nd example and feed it to the network, and so on. Then the number of iterations should be less than or equal to the number of words in the example dataset right? But here length of examples = 1536 which is less than the number of iterations.
We are building a character level model. This means the model predicts the next chracter for a given character as input.
Assuming you are familiar with RNN, we input one-hot encoded character as input along with previous activation state and get a prediction of what the next character might likely be using softmax activation. Please see
rnn_forward in utils.py to understand the details.
7 names for every
2000 iterations to show how the model is starting to make progress closer to real dino names and also to make the grader happy (see grader cell below the model cell which checks for the last generated name).
Please read the markdown to understand what
idx means. There’s a good level of explanation about it and what to do when the iteration count goes beyond the number of examples.