Hello,
is it possible to create a multiple tools agent with a local llm? all the examples i’m seeing online are all with Chat models which requires an API key (Mistral, Cohere, etc.).
Otherwise any recommandation of another method to accomplish the same goal ?
Thanks in advance.
1 Like
Hi @mo_ayeb
Yes, you can! Choose your models (from HuggingFace, etc.), set them up locally, integrate them into your application, manage responses, test thoroughly, and deploy.
I’m also willing to hear other people’s ideas!
1 Like
Hello @Alireza_Saei,
yeah i did that even with Ollama but for simple application like RAG ,
in my case i’m trying to create an Agent with multiple tools, and i’m finding issues, the base models from HuggingFace or Ollama apparently doesn’t have bind_tools method so they can’t be used in this context, what can be used is chat models like ChatMistralAI which requires an API key 
Hi I am also very interested into this setup. I am thinking to a series of local models at different sizes, 4B, 9 - 12B, 35B, 70B and also cloud based via API.
It is not clear what is actually possible with such models, which size etc…
1 Like