Protecting my data

Short Course Q&A Knowledge Graphs for RAG
1

So after watching these short courses on how you can effectively extract information from texts using LLMs and RAG, I had an idea that I wanted to do something for my company where I will build a knowledge graph database to store all our important documents, then build a chatbot that will be able to answer any questions my colleagues may ask it. I want to know if it is advisable since these documents are confidential and we don’t want the LLMs using it as a training set and potentially leaking it to the public.

2

@Nana_Adjei
Utilize open-source models like Llama3, Gemma, Mistral, platforms such as Hugging Face. Use Ollama to run these models locally .it will protect your data. RAG is all about LLM retrieve information from your knowledge graph for specific queries, rather than retraining the model with your data.

This approach ensures your confidential documents are secure, not exposed to external services, and not used for external training.

2 Likes
3

@yash.kavaiya3 Thanks for the response. Based on the course I watched over here I was able to create a RAG system using OpenAi and the Neo4j Db. But I don’t know how I can modify that to use a local LLM like the ones you mentioned.
I guess what I want to know is, how does one modify what we did in the short course on RAG to use Local LLMs. Which parts of the code do I modify?

4

@Nana_Adjei You have to change this part

image
image1163×302 34.6 KB

Here I use Gemini
You can add other local LLM Object

chain = GraphCypherQAChain.from_llm(
    ChatGoogleGenerativeAI(model='gemini-1.5-pro', max_output_tokens=8192, temperature=0.0),
    graph=graph,
    cypher_prompt=CYPHER_GENERATION_PROMPT,
    qa_prompt=CYPHER_QA_PROMPT,
    verbose=True,
    validate_cypher=True
)

Please Mark as Solution if you like

2 Likes
5

Thank you very much brother @yash.kavaiya3