The literal explanation of Q,K,V is (Please correct me if I’m incorrect am wrong)
Q - What information I am looking for.
I think this means given a sentence what information about this token should I search for.
K - given the sentence what information do this token contains.
V - what information do I actually provide.
is the working of K, V not same. like information contained by the token , and information provided by the token will be the same right ?
this is the confusion that I am having.
I will be grateful if anyone could clarify as to what is the exact requirement of the both.
by fundamental technicality of attention mechanism, k and v are distinct entity playing different role in the attention mechanism.
although by context k and v might be same, mathematically Q and K are used to calculate the attention weights (the relevance scores), while V determines what information gets blended into the output.
Hi Deepti, thank you for your clarification, it cleared some cloud.
I am trying to understand if they are analytically same then why do we require both, what happens when we use the same matrix in both the places during computation. ?
It’ll be really helpful if you have an example.
This is the point that is sticking in my way to learn the backend of self attention
Hi sayak21,
K and V are not the same as they have different learned weight matrices Wk and Wv. The dot product between queries and keys serves to match aspects of meaning of tokens in light of the desired output in order to determine which values should be passed through to the rest of the network. So the queries and keys serve to determine the extent to which there is a meaningful match between tokens, whereas values provide a general indication of the meaning of tokens.
I hope this clarifies.
By the way, if you want to attain an optimal understanding, the Transformers in practice course may be interesting to you (Transformers in Practice - DeepLearning.AI).
@sayak21
To understand all three Q, K and V, imagine a search for an article about “cars”:
-
Query (Q) -you searched for text (say “fast cars”). It represents what you are looking for at that moment.
-
Key (K) - the title or tag on an article (in this case “the history of cars”). It represents what a piece of information offers to match a query. it could be a single article or a list of articles.
-
Value (V) the text inside the actual article. It represents the actual content passed forward once a match is found.
to understand more precisely query, key and value; one needs to have an understanding of attention mechanism where each of 3 holds different weight matrices when a question is asked (Q), then the one holds the information(K) and then finally the actual information related to the question is asked (V)
as @reinoudbosch mentioned you can explore that transformer course, but I also recommend RAG course in dlai as the instructor has explained concepts very much in detail(but take this course after you have done Natural language processing specialisation)
Another way to know more in detail about attention mechanism is to read the article “Attention is All You Need”
if you want I can share that article with you.
regards
Dr. Deepti
Thankyou so much @Deepti_Prasad and @reinoudbosch for your explanations, its so clear now . I did a dry run too and now the process is so clear. Thankyou…
just to let you know a bit more.
before transformer, models only used “Key” and “Value” ( commonly known as an encoder-decoder pair), which treated words as static. Adding the Query allows a word to change its focus depending on the surrounding sentence. “Bank” becomes a financial institution or a riverbed depending on the word dynamically key and value were used causing sometimes incorrect output, hence addition of query allowed more distinctive output of information.
Good luck.
REGARDS
Dr. Deepti