Stateful chatbot with OpenAI API (or other AI API)?

Short Course Q&A ChatGPT Prompt Engineering for Developers
1

Hi,

I am wondering if there is any AI API on the market that lets us develop stateful chatbots, that allow feeding information in and storing it for future reference?

I saw in the ChatGPT Prompt Engineering for Developers that every time there is a new prompt from a user, the system has to provide context, which to me seems like a bad practice, feeding the same information to the bot for each user prompt. What is the size limit of that context and how does it affect the API usage pricing?

2

@mtkes
4096 tokens can be a maximum of one message exchange. This includes the amount of prompt tokens, the amount of message tokens, and even the amount of response tokens. So if you use 4090 tokens for the prompt and messages, the response will stop after 6 tokens, even if it would have been longer, because you exceed the token limit.