Hello folks,
when including questions of my own to evaluate the system,
I get answers indicating the context does not correspond to the
topic.
For example, if asking:
*"What are steps to take for hiking?"*
The response goes a long the lines:
Based on the given context information, the steps to take for hiking are not mentioned. Therefore, I cannot provide an answer to the query.
Perhaps is is getting confused about the relationship between the words “steps” and “hiking”.
It’s actually a good idea to try own questions to understand the feedback functions. For your example I would expect low context relevance and groundedness but still good answer relevance (because the LLM uses its own pretrained knowledge) if the topic is related to the indexed docs. If its not related at all, there might be intermediate prompts preventing an answer (which is what you want mostly in a real-world scenario) .
Why it would do so? It is so poor to recognize mispellings?
I I would agree if the topic were a relatively very recent one.
And as far as I understand we are using a no more than 2 years trained
system