Autograder Bug in Module 4 Lab: Placeholder detection failing for tasks 5 and 6

Course Q&A Fine-tuning & RL for LLMs: Intro to Post-training
, ,
1

Hi everyone,

I’m reporting a potential bug in the autograder for the Module 4 Lab (Constitutional AI). I am specifically having issues with Exercises 5 and 6.

Despite correctly including the required placeholders in my prompts, the autograder returns a failure message stating: “Missing problem/solution text labels” or “Missing placeholders.”

Specifically:

  • In Exercise 5, I am passing both {problem} and {solution}.

  • In Exercise 6, I am passing both {cot_solution} and {alternative_solution}.

I have verified that the variable names are spelled exactly as required. It seems the grader’s validation script (likely a regex check) is failing to identify these placeholders even when they are present in the code cell. This is preventing me from achieving a full score despite the logic being correct.

Has anyone else encountered this “invisible placeholder” issue? Is there a specific formatting (like avoiding triple quotes or specific indentation) that the grader requires to “see” these variables?

Any help from the mentors would be greatly appreciated.

Best,
Frank

2

I could not replicate the error. The information provided is limited. Consider

  1. Use explicit labels to ensure clarity. For example,
    Do xyz.
    Problem: {problem}
    Solution: {solution}
  2. Use print statements to verify that input variables are correctly passed to the function.
3

Are you referring to the in-course unit tests of your code?
Or are you referring to submitting your notebook to the grader for your score on the assignment?