In the ungraded lab which generates code-as-plan for a given customer query, we have two roles in messages:
“System” and “User”
While “System” contains a simple prompt where we only tell the LLM its role as an code generator, “User” contains a detailed description of what the code should do, how it should handle the corner cases as well as the customer query.
My question is:
Why don’t we define what the code should do, and how it should handle the corner cases in “System” prompt and only have the Customer query as part of the “User” prompt?
In this case, as the lab only presents a short interchange, it does not make a big difference whether or not to include what the code should do and how to handle corner cases in the system prompt. The course is based on the idea that it is best practice to divide tasks this way between the system and the user prompt.
If an interchange is longer or more complex dividing tasks this way matters for various reasons, including maintaining focus and security. Here is an interesting article that also discusses how system prompts may adversely impact LLM outputs:
That is indeed a good observation. To answer this, I would say the main reason we separate them is flexibility.
If you put all task details and corner cases in the System prompt, the system becomes too rigid. Every request would carry the same instructions, even when they are not relevant.
