Hello,
I have created an application which takes a context and generates question and answer from the context. For this I have chosen T5 model ( as it is a sequence to sequence task). Initially, I was improving in my task (the dataset size which I tried initially was of 1000 examples later I increased the dataset size to 25000 ) but after few epochs (around 2nd epoch) I am unable to improve my rouge score at all. I tried to train for 50 epochs but the result is the same. My rouge score was not at all improving( I got around 0.3 in rouge-1, 0.18 in rouge-2). The result was my model generating repeated choices for multiple choice questions.
Further implementation details are given here GitHub - sujith2303/Quizbot.ai.
-
What can be the reason for not improving my rouge scores and the model’s quality of generating questions?
-
Tried using few shot at inference but there is not use. Why don’t few shot learning apply here?
-
If I use different temperatures and change the number of questions to return parameters, there is not change. It is generating same questions. For instance , if I use return sequences as 10, it is generating 10 same questions. Is there any other way to generate different output?