Can anyone explain about what is the meaning of vector in encoder of transformer architecture of generative AI
In order to get a meaningful answer here, you’ll need to be a bit more specific or give reference to a particular lecture in a particular course. There are a number of courses here that explain Transformer Architectures, in particular DLS Course 5 (Sequence Models) Weeks 3 and 4 and NLP Course 4 (Attention Models). There are probably courses that address this in the various TF specializations as well, but DLS Course 5 is the best foundational course about this general topic.
Just generally speaking, there are a number of vectors involved in the encoder phase of a sequence model that does some form of translation. Typically at least the cell memory state and then the LSTM or GRU state depending on the type of RNN that is used for the encoder. Those are vectors, but the values change per timestep and per training iteration, so it depends on how you view them whether they are represented as vectors or tensors.
The best approach to understand how these models are built is to take one of the courses that cover that, as mentioned above.
1 Like