Hi,
I’d like to use AI to create a ChatBot that answers students’ questions as first level help desk about these arguments:
I could train the system with a lot of QA emails and a lot of tutorials.
Which is the best framework to do this job?
Would you use LangChain, Rasa or the ChatGPT API directly?
Thank you very much for your suggestions.
Claudio
Hi Claudio,
I feel, with the new personalized GPT’s you can build with ChatGPT, that would be the easiest way of creating such a bot.
If your knowledge of Python, Pandas, Tensorflow and / or PyTorch, and API’s is really good, you could build a model yourself. But that’s quite a bit harder to do.
Just my 2 cents.
Kind regards,
Alex van Winkel
Computer Science Teacher
I will suggest you follow the typical generative AI development lifecycle that was also mentioned in one of Andrew Ng’s course:
The steps are:
| Step | Scope | My suggestions |
|---|---|---|
| Define a use case | Use case needs to be specific and narrow. | I will suggest you breakdown your big use cases into smaller ones. For example, start from answering student’s queries about event calendar. |
| Curate training data | Gather all the data for this specific use case. You will then understand what are the data pre-processing is needed too | Certain types of data are easier to be processed by LLMs, for example, PDF and text are easier than tables and images. |
| Experiment with LLMs | Test the performance on LLMs. | You can start from the largest LLMs, call the APIs. This is to proof your use case is feasible with current technology. Langchain is always needed. The challenge you will face later is probably related to data privacy, security, cost, etc. Then you will need to consider whether you will need open-source models. Among all open-source models, you will need to then consider if you need very large model or smaller models can meet your needs. |
| AI Governance | To build guardrails to ensure model’s output is helpful, honest, and honest? | LLMs have the risks, including hallcinations, generating harmful output, breach of data privacy, etc… The guardrails to be considered here include manual review of the output, prompt engineering so this model won’t be mis-used, any models like RLHF, or classification models. |
| Deployment | Platform to run this App? Load balance Cost management | How big is your user group? Do you need to manage the incoming queries if you are calling APIs so you can manage your cost? |
| Other security matters | IAM setup Intranet |
These are the steps I could think of to use LLMs to build an App like a bot. If you are using Azure cloud, you can also consider CoPilot Studio to start. But it is then tightly coupled with Microsoft. Otherwise you could start to build it using Python.
Thank you for you answer.
After following the introductory course to Lang Chain I think that a framework like this really simplifies the job of implementing a good chatbot.
It introduces some concepts as chains, agents, evaluations, ecc. that help you very much.
It seems interesting also the idea of introducing the use of a vectorial database to reduce the number of queries towards ChatGpt.
Claudio
Thank you for your advices.
Yes it is important to define a roadmap for the project, probably it would be useful to try a little spike with a few of data and features to uderstand better the context.
My doubts are on the massive use of a proprietary solutions like ChatGpt, because for small projects a solutions like Rasa could be implemented on premise or in a private cloud.
Howere following the course on LangChain I can see that it allows you to use also vectorial databses and agents to reduce the numeber of calls to tools like ChatGpt.
Probably this could be a good compromise.