Getting a 404 error when running base_completion = generate_stream(prompt)

Short Course Q&A Reinforcement Fine-Tuning LLMs with GRPO
1

Anyone know the correct URL for the PREDIBASE_MODEL_QWEN_URL?? I am using
shared qwen2-5-7b-instruct URL which works fine in the terminal but keep getting a 404 error in the notebook when running base_completion = generate_stream(prompt) cell. I tried posting the URL I am using but the forum won’t allow it.

2

The call initialises the model in predibase so it is the correct URL and API key.

3

NotFoundError Traceback (most recent call last)
in <cell line: 0>()
----> 1 base_completion = generate_stream(prompt)

4 frames
/usr/local/lib/python3.11/dist-packages/openai/_base_client.py in request(self, cast_to, options, stream, stream_cls)
1032 )
1033 log.debug(“request_id: %s”, response.headers.get(“x-request-id”))
→ 1034
1035 try:
1036 response.raise_for_status()

NotFoundError: Error code: 404

4

If you look inside the notebook in lab 3, you will note a comment that says that we will find how to deploy an RL fine-tuned model; thus, after this lab, we should be able to obtain the URL.

5

From the shared endpoint you do not need to specify a URL;

from predibase import Predibase

pb = Predibase(api_token="<PREDIBASE_API_TOKEN>")
client = pb.deployments.client("qwen3-8b")

# Basic generation with customizable parameters
response = client.generate(
    "Explain quantum computing in simple terms.",
    max_new_tokens=100
)
print(response.generated_text)

# Stream responses for real-time output
for response in client.generate_stream(
    "Write a story about a robot learning to paint.",
    max_new_tokens=200
):
    print(response.token.text, end="", flush=True)

To use the OpenAI Python SDK with Predibase models, refer to the documentation here: Querying Models - Predibase