I would like to know, how large could the input be in the system message when creating a chatbot to provide commands to the tool on how behave. Is there a max size? I saw in another message that maybe I can store all messages in the database and then pass it to the tool to provide context. But what if I have a big history?
As of writing, the gpt-3.5-turbo model has a token limit of 4,096 , while the gpt-4 model has a limit of 8,192 tokens.Apr 21, 2023
Yeah you could use somekind of loop retrieving a bigger number of messages.
Rate limits discussed at that URL. might be helpful to take a read through
This one too
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to handle rate limits\n",
"\n",
"When you call the OpenAI API repeatedly, you may encounter error messages that say `429: 'Too Many Requests'` or `RateLimitError`. These error messages come from exceeding the API's rate limits.\n",
"\n",
"This guide shares tips for avoiding and handling rate limit errors.\n",
"\n",
"To see an example script for throttling parallel requests to avoid rate limit errors, see [api_request_parallel_processor.py](api_request_parallel_processor.py).\n",
"\n",
"## Why rate limits exist\n",
"\n",
"Rate limits are a common practice for APIs, and they're put in place for a few different reasons.\n",
"\n",
"- First, they help protect against abuse or misuse of the API. For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.\n",
show original