Multiple issues with graded assignment in RAG M5 module

Course Q&A Retrieval Augmented Generation
,
1

Thanks to the team for actively reviewing and responding to issues raised in this forum.

I am currently facing multiple issues while working on Assignment #5 and this is blocking my progress, requesting support to review and fix them soon

Issue #1 – Unable to access the model (Blocking)

Description:
When invoking the function generate_with_single_input, the model fails to load.

Error Message:

InvalidRequestError: Error code: 400 - {
"message": "Unable to access non-serverless model meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo.
Please visit https://api.together.ai/models/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
to create and start a new dedicated endpoint for the model.",
"type_": "invalid_request_error",
"code": "model_not_available"
}

Steps to Reproduce:

  1. Run the function generate_with_single_input

  2. Model invocation fails with the error above.

Current Status: Unresolved – blocking assignment progress for the last 2 days.

Issue #2 – Possible model response format mismatch (Need confirmation)

I am not sure if this is an actual issue and would appreciate confirmation.

It appears that the model being used internally is Qwen/Qwen3.5-9B, which seems to return its response in the reasoning field rather than the content field.

However, in the function:

check_if_faq_or_product(query, simplified=False)

the validation logic checks the content field (expecting values like FAQ or Product). Since content is blank when using Qwen/Qwen3.5-9B, the validation fails.

**Current Status:**Unresolved and possibly related to the model access issue above.

Issue #3 – Unable to load get_phoenix_endpoint during library import

Description: While importing the required libraries, the function get_phoenix_endpoint could not be loaded.

Workaround Applied: I temporarily resolved this by adding get_phoenix_endpoint inside utils.py.

Current Status: Temporarily resolved.

1 Like
2

Hi!

Thanks for pointing this issue. We had another model deprecated by our provider together.ai. Right now I’m working on fixing it, but it will take a few hours. I will let you know once it is completely fixed.

Cheers,
Lucas

2 Likes
3

Hi! The issue has been fixed. Please close and open the assignment to fetch the new version. Let me know if the issue persists.

4

Thank you Lucas for your work! However the issue persists. It looks like the assignment code is not updated on my side. The error is still: InvalidRequestError: Error code: 400 - {“message”: “Unable to access non-serverless model meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo. Please visit Together AI | The AI Native Cloud to create and start a new dedicated endpoint for the model.”, “type_”: “invalid_request_error”, “code”: “model_not_available”}

5

Hi @changjia.chen!

Is this in M5 as well?

6

Yes. Please find the attached.

1
11948×1117 190 KB

7

Hi @lucas.coutinho - I am able to proceed forward now with assignment 5, thanks for all the help

8

@changjia.chen

you need to refresh your work environment as the lab were updated again yesterday to new exception error with another model deprecated.

9

Thanks Deepti! Now my assignment 5 works but my assignment 2 fails because of missing models in the model folder. Please find the attached image of error message.

In addition, assignment 5 still requires a manual modification of models to Qwen (the original functions have a default of Llama but they are locked and cannot be modified). I suggest a update on those functions too (e.g. generate_params_dict() )

A revision of the assignments codes would be appreciated! Many thanks!

1
11180×1186 90.2 KB

10

@changjia.chen

Did you refresh your work environment to get updated new notebook uploaded in your work environment. As you know together.ai deprecated their models resulting in recurring newer exception even after resolving consistent errors, labs were frequently updated for rag courses including the ungraded labs. So kindly click restore original version, for having to work on the latest version of notebook.

Let me know if you still continue to encounter the issue.

regards
Dr. Deepti

11

Thanks! I refreshed and reloaded many times. The notebook did not updated and the issue persists. I will try it later today and see if it works.

12

you clicked on the 3 dots on the top right corner, and then click Restore original version?? this is the way to refresh work environment.

13

Thanks! Now it’s working!

1 Like