Module 1 Exercise 3 parameter for the prompt is requiring "question" instead of "prompt"

Course Q&A Fine-tuning & RL for LLMs: Intro to Post-training
,
1

In Module 1 lab exercise 3, in the line

response = None
The exercise is graded as “correct” when I use ‘question’ as the name of the parameter for prompt as shown below:
response = llm.generate_response(question=prompt, max_tokens=512)

BUT

The exercise is graded as “incorrect” when I use ‘prompt’ as the name of the parameter for prompt as shown below:
response = llm.generate_response(prompt=prompt, max_tokens=512)

When I checked the code in utils.py, the parameter is called “prompt”

(I did not try calling it without the name for the parameter)

Please help me understand why the grader requires the parameter to be called “question”.

Thank you,

2

“question” is not a valid argument for the generate_response method. If I try using it, I get this error: ServeLLM.generate_response() got an unexpected keyword argument ‘question’. On the other hand, when I pass the argument using the correct name, “prompt”, my evaluation runs fine. I suspect the issue is somewhere else. The grader isn’t perfect, so I would suggest you carefully check your work.

3

Thank you. There is something strange going on as described below:

  1. if I submit with:
    response = llm.generate_response(prompt, max_tokens=512) - it passes the grader

  2. but if I submit with:
    response = llm.generate_response(prompt=prompt, max_tokens=512)
    It fails with the following error message:
    Exercise 3

0/20

Overall Score: 0.00 (0.0/1 exercises) Exercise Results: FAIL ex3: 0.00 - Passed 0/1 tests - Function execution failed: part_evaluate_model_correctness..g..MockServeLLM.generate_response() missing 1 required positional argument: ‘question’ FAILED! Overall score 0.00 was below 0.70 Focus on the exercises marked with FAIL above.

  1. but if I submit with:
    response = llm.generate_response(question=prompt, max_tokens=512)
    It PASSES with the following message:
    Exercise 3

20/20

Overall Score: 1.00 (1.0/1 exercises) Exercise Results: PASS ex3: 1.00 - All tests passed! PASSED! Great work on the Model Training Pipeline Comparison assignment!

Even though I have completed the lab, I would appreciate if this issue could be resolved.
Thanks again for your help.
-Sudip Chahal

4

That’s odd. I will pass the issue to the DLAI team. Thanks.

5

Still getting the same error here.

Exercise 3
0/20
Overall Score: 0.00 (0.0/1 exercises) Exercise Results: FAIL ex3: 0.00 - Passed 0/1 tests - Function execution failed: part_evaluate_model_correctness..g..MockServeLLM.generate_response() missing 1 required positional argument: ‘question’ FAILED! Overall score 0.00 was below 0.70 Focus on the exercises marked with FAIL above.

passing generate_response(question=prompt) works.