Hey everyone,
I have been exploring large language models for a personal side project building a smart assistant that handles some repetitive tasks & interacts with user data in a meaningful way. I started with OpenAI’s API & while it works smoothly, the cost seems such as it could add up fast if I try to scale or integrate it into more features.
That is making me seriously consider switching to open-source models such as LLaMA 3, Mistral or maybe even Mixtral But I am unsure about the performance differences & how much effort it would take to host and manage them. I have got decent hardware and a bit of ML experience but not sure if that is enough.
It kind of reminds me of choosing between a quick, guided reactjs course online versus going through raw documentation and trial-& error both have pros & cons.
Want to hear your thought, Has anyone here gone the open-source route? Was the trade-off in setup time worth the flexibility?
Thank you.