Source_documents are returned even if the question has no context related

Short Course Q&A LangChain: Chat with Your Data
1

Just to understand I’m doing the right thing and how to manage this kind of scenario.

I’m playing with the code of the lesson 6 (the part without the memory) trying to ask for a question not related the information contained into pdf documents.

just the important part of the code for reference


  qa_chain = RetrievalQA.from_chain_type(
        llm,
        retriever=vectordb.as_retriever(),
        return_source_documents=True,
        chain_type_kwargs={"prompt": QA_CHAIN_PROMPT},
    )

    result = qa_chain({"query": question})

My question is : “What about SpaceX ?”
The answer I get back is correct :
“There is no mention of SpaceX in the given context, so I don’t know the answer to that question. Thanks for asking!”

What I don’t understand is why the “source_documents” property contains documents with metadata.

I expected to find the “source_document” property empty since these documents don’t contain information about the question.

So my questions are :

  • Am I missing something?
  • How to handle this scenario when you want show “source_documents” but in this case are not related to the question ?

Thanks
Luca