in one of the early labs, I tried my own experiment, entering unstructured text. one of the tasks I gave in the prompt was to return json that included a list of urls from the text. on the first result, chatgpt only return one url (of 3). I tried to run the same prompt again and the 2nd (and 3rd time), chatgpt returned all 3 urls in the resulting json list. was this result only by chance? is there something in the programming of the llm that tries to change/improve results on every iteration (even if the input/prompt has not changed)?
The way how LLM works cause that exactly the same prompt will give different results (different - does not means better).
This is due to at least few things:
- Stochastic Nature of Language Models
- Temperature Setting
- Token Sampling Strategy
And model do not have memory of previous interactions so it shouldn’t be a reason why new interactions don’t have to be better.