Multiple sets with same probabilities in the output of softmax?

Course Q&A Generative AI with Large Language Models
1

I’m a bit confused with top-K or top-p functions.
Let’s say if we have selected top-p as 0.3 and say words have the following prob.

cat-0.2
race-0.1
hmm-0.01
tom-0.11
chase-0.19

Now, there can be two such sets where p=0.3
{cat-0.2, race-0.1} and {tom-0.11, chase-0.19}

Out of these sets how is ambiguity resolved/ which set is chosen ?

2

Hi,

Top-p definition: Select an output using the random-weighted strategy with the top-ranked consecutive results by probability and with a cumulative probability <=p.

So, sorting the list again:

cat-0.2
chase-0.19
tom-0.11
race-0.1
hmm-0.01

If top-p=0.3, the result is just {cat}. Because {cat, tom} would have a total probability greater than 0.3.

1 Like
3

thanks