Bug in the Customer Service Agent ungraded lab

So, after executing all of the steps with the following user prompt:

“I want to buy 3 pairs of classic sunglasses and 1 pair of aviator sunglasses.”

The LLM produced a plan as a Python code, but this code had bugs. The final response from the code was this:

“Your purchase of 3 Aviator and 3 Classic sunglasses is complete; your total is $420.”

So it sold me 3 pairs of Aviator sunglasses while I requested 1. It might be a serious issue in a real world system.

How can we deal with this in real world?

You are absolutely right to point this out. This could be a serious issue in a real-world system, especially when an agent is taking actions that involve purchases or other consequential decisions.

With AI systems in general, and agentic systems in particular, there is always a possibility of producing an incorrect result. These systems work probabilistically, and they can sometimes hallucinate or make mistakes even when the instructions seem clear.

To reduce the risk and have more control over the outcome, we can introduce guardrails and clearly defined boundaries around what the agent is allowed to do. For example, we could validate the quantities before allowing the purchase to be completed.

Another important approach is to have a human in the loop, particularly for high-impact actions. The agent can prepare the purchase or recommendation, but a human can verify the details before the transaction is actually executed.

So, in a real-world implementation, we should not simply assume that the agent’s output is always correct. We need appropriate validation, guardrails, and, when necessary, human oversight to make the system safer and more reliable.