I’m working through the Agentic AI course and fairly regularly run into failing unit tests on the graded labs; the tests themselves are faulty. Several times I merely re-run my graded code cell without making changes, re-run the unit test cell, and what was previously a failing test now passes, and vice-versa. Example: module 3 graded lab, “reflect and rewrite”, the unit test is looking for 4 specific words in my prompt: “strengths”, “limitations”, “suggestions”, “opportunities”. All four words are in my prompt. Most often, the first word in the list is the one that the unit test somehow fails to parse out, but not always.
I have similar challenges with the portion of the lab that converts the report to HTML. In this case, the model is failing to parse out the {report} variable placeholder in my prompt, responding that I should provide the report to convert, which causes the unit test to fail because it’s not outputting HTML.
can you post a screenshot of the failed unit test res. please make sure not to post any grade function codes here as it is against community guidelines
I have not taken that course, but a lot of the problems you describe sound like they could be caused by the fact that the runtime state of a Jupyter notebook is not always easy to discern. For example if you work on the notebook for a while and then leave it idle for 10 minutes and come back, the “kernel” may have timed out and been killed. You can see the output of previous cells in the notebook, showing that they were run at some point in the past, but that does not prove that the actual runtime state created by running those cells is still present. So if you then click “Shift-Enter” to run some cell in the middle of the notebook, it may well fail if it depends on state created by previous cells (e.g. imports that were done or global objects that were created).
One easy way to get things back to a consistent state is:
Kernel -> Restart and Clear Output
Cell -> Run All Above
Another tricky thing to be aware of is that if you type new code in a function cell and then just run the test cell that calls that function, it runs the old code again without your changes. You have to actually click “Shift-Enter” on the modified cell to get your changes incorporated into the runtime state. You can easily demonstrate this to yourself:
Take a function that works when you run its unit test and then purposely break it. E.g. multipy the return value by 2. Now run the test cell again and it still works, right? What? But then click “Shift-Enter” on the now broken cell and run the test again and “Kaboom!”
Thanks for your suggestions. I’m familiar with some of the quirks of Jupyter Notebook state. My first path of troubleshooting any problem with them is to re-run all of the cells in order from the top of the notebook.
I’ve performed the experiment you describe as well. The issue remains inscrutable.
