C1M3 end-to-end pipeline fails but 100% score achivable

Course Q&A Agentic AI
,
1

The reflection_and_rewrite code I wrote and passed its unit test doesn’t seem to be used in the end-to-end pipeline test cell since I get the following error. I did restart the kernel and re-ran all the cells as well. However when submitting the lab for grading, I was able to get 100%.

JSONDecodeError                           Traceback (most recent call last)
Cell In[8], line 63, in reflection_and_rewrite(report, model, temperature)
     62 try:
---> 63     data = json.loads(llm_output)
     64 except json.JSONDecodeError:

File /usr/local/lib/python3.11/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    343 if (cls is None and object_hook is None and
    344         parse_int is None and parse_float is None and
    345         parse_constant is None and object_pairs_hook is None and not kw):
--> 346     return _default_decoder.decode(s)
    347 if cls is None:

File /usr/local/lib/python3.11/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
    333 """Return the Python representation of ``s`` (a ``str`` instance
    334 containing a JSON document).
    335 
    336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338 end = _w(s, end).end()

File /usr/local/lib/python3.11/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
    354 except StopIteration as err:
--> 355     raise JSONDecodeError("Expecting value", s, err.value) from None
    356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
Cell In[12], line 8
      5 print(preliminary_report)
      7 # 2) Reflection on the report (use the final TEXT to avoid ambiguity)
----> 8 reflection_text = reflection_and_rewrite(preliminary_report)   # <-- pass text, not messages
      9 print("=== Reflection on Report ===\n")
     10 print(reflection_text['reflection'], "\n")

Cell In[8], line 65, in reflection_and_rewrite(report, model, temperature)
     63     data = json.loads(llm_output)
     64 except json.JSONDecodeError:
---> 65     raise Exception("The output of the LLM was not valid JSON. Adjust your prompt.")
     67 return {
     68     "reflection": str(data.get("reflection", "")).strip(),
     69     "revised_report": str(data.get("revised_report", "")).strip(),
     70 }

Exception: The output of the LLM was not valid JSON. Adjust your prompt.
2

Hi Kris777,

It looks like although your prompt may succeed in making reflection_and_rewrite produce valid JSON, it may also sometimes fail to do so. This could be due to some variation in the output of the LLM (which may be related to the temperature variable).

Whichever may be the case (not seeing your actual prompt), this is a useful reminder that LLMs can produce varying outputs that need to be checked - in this case for valid JSON.

Thanks for sharing your experience!