I tried to put all the text in a paper into the “context” of the system message, but it ended up with an error, because there is max limit of GPT3.5 token number.
Need some help.
Depending why you need the full paper as a context. You may like to create some essence of the paper, or use abstract section. You may like to try fine tuning of creating training Q&A set corresponding to the paper…
1 Like
Yes there is a maximum limit to the input size based on token count, and it’s likely that many papers would exceed that limit. This limit may be expanded in the future, but in the meantime, the best solution might be processing the paper as smaller sections or providing an abstract/summary. Hope that helps!
1 Like