The model’s output includes additional narrative and instructions beyond the strictly formatted JSON block. This extra text interferes with the extraction logic, which relies on isolating a clean JSON snippet. As a result, the parser encounters unexpected characters and formatting, leading to errors when attempting to decode the JSON, and ultimately returns an error response or invalid data.
Which course are you attending?
Am doing an independent project and currently not attending any course on DeepLearning.AI
OK. Your post didn’t actually ask any questions, so I was curious what you were working on.
Good luck with your project.
1 Like
@TMosh can you help me in this or suggest me some resources from where i can solve my query?
Sorry, I have no idea what model you’re using.
If you wish for assistance from the community, you’re going to have to provide a lot more information.
1 Like
Am using Llama 3.1 8B instruct model and downloaded it from the HuggingFace.
Am trying to send it an API request through postman and in return i expect it to return a valid JSON response. But the issue is that the model generated extra text before and after the JSON and that is causing error in producing the valid response!
Hopefully someone with experience using those tools will have some suggestions.
1 Like
Use an Explicit “JSON Only” Prompt
The simplest approach is to prompt the model very explicitly, emphasizing that you only want JSON. For instance, you can include instructions such as:
System/Instruction Prompt:
You are a JSON generation assistant. When responding, do not include any text outside of the JSON block. Return only valid JSON, with no extra commentary, disclaimers, or formatting.
Then provide a user prompt with an example of the desired JSON structure. For example:
User Prompt:
Please generate a JSON response of the following form:
{
“graded_questions”: [
{
“maxmarks”: 5,
“finalmarks”: 5,
“explanation”: “Explanation text here”,
“sequenceattemptid”: “N/A”,
“questionattemptid”: “999999”
}
]
}
Important: Output only the JSON. No extra text, disclaimers, or formatting.
This direct approach often works well with LLMs that are “instruct” or “chat” tuned, as they tend to follow instructions to produce a certain format.
1 Like
Thanks for the idea but am using the prompt that has this ONLY JSON thing mentioned.
Below is my prompt
user_prompt = f"“”
You are an AI grading assistant. Grade the student response using THESE EXACT criteria:
1. RELEVANCE FIRST:
- If completely irrelevant: 0 marks regardless of length
- If partially relevant: reduce marks accordingly
2. IF RELEVANT, GRADE BY WORD COUNT:
- 0-5 words: 0 marks
- 6-51 words: 1 mark
- 52-152 words: 2 marks
- 153-249 words: 3 marks
- 250-299 words: 4 marks
- 300+ words: 5 marks
Question: {questionsummary}
Student's Response: "{responsesummary}"
Word Count: {word_count} words
Question ID: {question['questionattemptid']}
Sequence: {question['sequencenumber']}
CRITICAL: YOUR RESPONSE MUST CONTAIN ONLY VALID JSON INSIDE THE MARKERS.
DO NOT ADD ANY TEXT BEFORE OR AFTER THE JSON MARKERS.
ONLY USE DOUBLE QUOTES FOR STRINGS, NOT SINGLE QUOTES.
### BEGIN JSON ###
{{
"graded_questions": [
{{
{{
"maxmarks": 5,
"finalmarks": <DYNAMICALLY_GENERATED_SCORE>,
"explanation": "<GENERATED_EXPLANATION>",
"sequencenumber": "<EXTRACTED_SEQUENCE_NUMBER>",
"questionattemptid": "<EXTRACTED_QUESTION_ATTEMPT_ID>"
}}
}}
]
}}
### END JSON ###
"""
Can you provide a sample of what you are getting from the model? I’ve been able to get the models to provide a response in the exact format requested, similar to what you are doing. The only cleaning I’ve had to do is to remove the triple backticks and a json label that is always output.