Eshaan_Gupta:

LSTM video

Perhaps the most common one, is that instead of just having the gate values

be dependent only on a t-1, xt.

Sometimes people also sneak in there the value c t -1 as well.

This is called a peephole connection.

Play video starting at :7:9 and follow transcript7:09

Not a great name maybe, but if you see peephole connection,

what that means is that the gate values may depend not just on a t-1 but

and on x t but also on the previous memory cell value.

And the peephole connection can go into all three of these gates computations.

So that’s one common variation you see of LSTMs one technical

detail is that these are say 100 dimensional vectors.

If you have 100 dimensional hidden memory cell union.

So is this and so say fifth element of

c t-1 affects only the fifth element of the correspondent gates.

So that relationship is 1 to 1 where not every element

of the 100 dimensional c t-1 can affect all elements of the gates, but

instead the first element of c t-1 affects the first element of the gates.

Second element affects second elements and so on.

But if you ever read the paper and see someone

talk about the peephole connection, that’s what they mean,

that c t -1 is used to affect the gate value as well.

Here are the exact subtitles from the video.