Ques. In attention is all you need lesson, the use of feed forward network in the encoder and decoder module is not quiet clear. It will be helpful if someone can explain it clearly. I did not understand the explaination given in the reading material too.
1 Like
Dear @bshree,
May I confirm if your concern has been resolved, or if you still require assistance?
–
Keep Learning AI with DeepLearning.AI - Girijesh
Feedforward Neural Networks (FNNs) are the most commonly used neural network type where information flows in a single direction (input layer receives the data, hidden layers process and understands complex representation and output layer produces the final output).
In transformer architecture, both encoder and decoder are trying to learn the complex representations using their FNNs, the encoder builds the contextual understanding from the user’s prompt while the decoder further takes generated tokens from encoder as an input and train its layers to generate the next set of tokens.
To understand more about these neural networks and their functioning in detail, I would advise going through the neural-networks-deep-learning course that talks about these networks in more detail.
Happy learning!
1 Like