In this assignment of language model generation, we use same inputs and outputs.
That is if input is “abcdef” , we expect model to generate “abcdef”. So what is preventing the model from learning identity and to completely ignore the sequence?
Ideally the input should be “abcdef” and output should be “bcedfg” right? I am also sensing the problem in the modelling itself. I think we should predict probabilities of just one character given the sequence of previous n characters instead. Basically make the model many to one instead of many to many.
I mean abc->d , bcd->e etc…
Hi sachin_B_S,
Note that GRULM contains a ShiftRight layer. As you can read in the docs, the default value for the number of shifts to the right is equal to 1. So in default mode, the model will attempt to predict the next character.