In the Dinosaurus Island we’ve learned to implement a character level generator.
In the next assigments, we’ve progressively used Keras for LSTM cells and such.
There is one thing I cannot understand with Keras cells, and it’s wether or not the output y is fed as the next x<i+1> in the next cells.
I looked at the documentation, tried to find architecture drawings or Keras tutorial, but overall I still find this extremely confusing.
Does someone have an answer ?
This is discussed in the lectures, and in the diagrams that are embedded in the notebook.
What I am interested is the red arrow (and sampling) implemented during week 2.
I looked in the next assignments over the next weeks, and we don’t reproduce the same behaviour.
Do you have a pointer I could look up ?
Edit: Maybe this red arrow is a standard behaviour for a RNN (not GRU nor LSTM cell) and my question is pointless, but this is very unclear in Keras (at least to me).
The reason those red arrows appear in this assignment is that this is the “sampling” process, which only happens after the network is trained.
This is making new predictions, so we don’t have the sequence of known ‘x’ values.
Instead, to get the samples we use the output of each cell as the input to the next cell.
This technique doesn’t have anything specific to do with Keras. I think it has more to do with the other assignments in the course don’t demonstrate very much sampling after the training is completed.
@TMosh I agree, but in this assignment, we implemented it by hand.
I would like to know how to reproduce the same architecture using Keras.
Perhaps you can’t. Keras doesn’t do very much automation for recurrent networks. It is intended to help define a direct pipeline that has no recurrent cycles.
Oh I see. It’s true I didn’t consider the possibility that you simply can’t do that right away.
Hello @TMosh !
I have found something you might be interested in knowing: GitHub - sekharvth/dinosaur-name-generation-keras: Name your own dinosaurs
This person reimplemented the assignment using Keras. I’m currently looking into it, as it’s a few years old so it might need some rewriting.
All the best !
Thanks for the reference.
@TMosh I should have paid more attention because actually the Jazz Generation is exactly what I needed.