Change Timeout parameter in aisuite with ollama?

Course Q&A Agentic AI
,
1

I am trying to run the notebook locally using ollama, to test how small model may work with agentic workflows.

I set the model to ollama:local-model and it runs and connects well to ollama, my problem is that because inference is slow, (specially with reasoning models) i reach a timeout error, and the workflow dies.

I tried to pass as a parameter a timeout value, but it does not seem to work. Basically I do something similar to this:

import aisuite as ai

client = ai.Client()

response = client.chat.completions.create(
    model="ollama:gpt-oss",
    messages=[{"role": "user", "content": prompt}],
    temperature=0,
    timeout=150,
)

Any ideas how to increase the waiting time so it does not time out?

I paste the Traceback error for other possible searches:

Traceback (most recent call last):
File “/home/david/MOOC/agentic_AI_AndreNg/02_sql/Lab2.py”, line 307, in
feedback, sql_V2 = refine_sql(
~~~~~~~~~~^
question=question,
^^^^^^^^^^^^^^^^^^
…<3 lines>…
model=“ollama:gpt-oss”
^^^^^^^^^^^^^^^^^^^^^^
)
^
File “/home/david/MOOC/agentic_AI_AndreNg/02_sql/Lab2.py”, line 274, in refine_sql
response = client.chat.completions.create(
model=model,
…<2 lines>…
timeout=150,
)
File “/home/david/miniconda3/envs/ooo/lib/python3.13/site-packages/aisuite/client.py”, line 245, in create
response = provider.chat_completions_create(model_name, messages, **kwargs)
File “/home/david/miniconda3/envs/ooo/lib/python3.13/site-packages/aisuite/providers/ollama_provider.py”, line 52, in chat_completions_create
raise LLMError(f"An error occurred: {e}")
aisuite.provider.LLMError: An error occurred: timed out

Somehow I need that the timeout parameter reaches the httpx.post method, as in per the traceback:

Traceback (most recent call last):
File “/home/david/miniconda3/envs/ooo/lib/python3.13/site-packages/aisuite/providers/ollama_provider.py”, line 41, in chat_completions_create
response = httpx.post(
self.url.rstrip(“/”) + self._CHAT_COMPLETION_ENDPOINT,
json=data,
timeout=self.timeout,
)

I just do not know where to include this parameter in the chat_completion_create method.

2

hi @Basillicus

check your error, it has the solution, although you added the parameter time out, you didn’t change chat completion create parameter for time out which is assigned as self.timeout

Being said that I feel if you add the timeout parameter as 150, then changing in the response code can cause your agentic workflow slower, so check if in parameters, you instead limit token or chunk (this is my personal suggestion to check which would yield better results)

3

Thanks @Deepti_Prasad for your reply. However, I am not supoused to change the chat_completions_create code in the aisuite/providers/ollama_provider.py file. I should be able to pass the tiemout parameter from the Client object when i create the chat completion. But it does not seem to work the way I had.

I found the solution looking into the aisuite code. The parameter has to be set when creating the client and pass the dictionary with the correspoding provider configuration:

client = ai.Client(
    {
        "ollama": {
            "timeout": 200
        }
    }
)

The bad news are that gpt-oss:20b model does not seem to give the right answer. It is not able to revise the SQL query properly, so it keeps giving the wrong negative result in the Lab2.

At least now I can keep trying new ways and models without running out of time! :smiley:

1 Like
4

can you share the negative result with lab2, so if any learners are exploring ollama, can look at your post for query and resolution of the issue.

probably the SQL query parameters don’t match with the metadata??

5

Yes, I can share the results, but I think that can be material for a different topic. This thread was about changing the timeout parameter in aisuite for ollama provider.

Maybe we can open a thread where we can explore the behaviour of local models at different tasks? That would be interesting actually. I would really love to see small models doing the agentic stuff right, but so far I can not see good results. It would be great having more people exploring this topic.

1 Like