Attention sequence model week3(make post attention steps output depend on prior step)

Course Q&A Deep Learning Specialization Deep Learning Resources
1

hi, i finished the assignment of attention sequence model (week3), where we built attention model which translates dates written by human into mm-dd-yyyy format. in this model the post attention lstm step did not have input from prior step as it makes sense for this model. i have two questions:

  1. code changes required to add additional input to post attention lstm cells other than the hidden state and context. i tried but was struggling a little so would love to get some pointers and will try myself later too.
  2. we could train this new architecture and i am assuming the model will just make the dependency on prior output very small and still work. thoughts?

thanks much,
Manish

1 Like