I’ve been applying for jobs and found that writing a cover letter for each position was tedious. I also delved into LLM and Langchain, hoping to leverage them for a project to aid in my job hunting. So, I developed Anukool under the GPL license. While it’s far from perfect, it has proven very useful to me, and I hope it benefits you as well. All I have to do is provide it with a PDF containing information about me such as my experience, skills, projects, etc and it will use this information along with the job description to generate a cover letter for me. Since I’m new to ML and LLM, any advice or feedback is greatly appreciated, and contributions are also welcome. I plan to utilize Llama-2 soon to further open-source the project.
cool idea. I plan on checking it out and in the meantime I have a question about its use of local models. Currently I have gguf models that are shared by a couple of llama.cpp applications as well as through python. What model formats does your application support?
I tried out the llama-2 7b model by TheBloke. The project should support all the models supported by the CTransformers class by long-chain. Please free to contribute to the project if you have any optimization/changes in mind.
yes I finally got around to trying and can confirm gguf model format is compatible. This means I can reuse downloaded models.
There are some considerations to using local models if one doesn’t want to use your model directory but I dropped you some github issues for them.
Now I need to figure out how to offload inference to gpu, at least some layers.
For personal use I can’t see the advantage over using my chat-gpt subscription as I get gpt4 performance at no additional api call costs. But I do appreciate seeing and learning from what you have put together. I had been playing with llama-index and gradio for similar rag and webui capabilities rather than fastai and langchain/huggingface so it was cool to see things done another way.
I’m glad to hear that you’re able to use the project finally!
I’ve already started addressing the considerations you mentioned in the GitHub issue, and I believe I’ve found a way to tackle them. I’ll submit a PR for it as soon as possible.
Regarding the device choice for embedding & generation, I initially wanted to implement something for Nvidia GPUs, but lacking access to one, I’ve set the project to utilize CPU only. If you could figure out a way to configure this based on the user’s PC, that would be fantastic. Perhaps you could submit a PR for it.
While the project primarily serves as a learning tool, it offers several advantages over using just GPT-4:
I’m able to utilize local models on my PC, ensuring all data remains on my device.
I can upload huge PDF files about myself without worrying too much about token length.
The existing API allows me to set up a cron job for automated job applications without manual intervention.
Above all, this project serves as a foundation for both myself and others navigating through LLMs and ML, providing a platform to build various interesting projects.
I want to express my gratitude for your time and valuable insights!