Lesson 5 update request | QAGenerateChain

Short Course Q&A LangChain for LLM Application Development
1

Hi all,
Would it be possible to update Lesson 5? It appears that the evaluations module has changed. I believe it happened in version 0.0.258 and the current notebook environment is running langchain 0.0.179. If running the latest, predictions = qa.apply(examples) throws an error.

…
ValueError: Missing some input keys: {‘query’}

Thanks,
ChuckM

2

HI Chuck,

The lesson is still not updated so I came up with this workaround (it might help you). We define a new examples list excluding the manual elaborated ones and extracting the query-set pair from ‘qa_pairs’ as follows:

modified_examples = [example['qa_pairs'] for example in examples[2:]]
# we add the hardcoded examples again to modified_examples
predictions = qa.apply(modified_examples)

Note : Please make sure to replace all the following instances of examples by modified_examples or just define examples again following above idea (although I prefer having a new variable so you can recall this issue if you revisit the code later on)

Regards,
Crozz

1 Like
3

Hi Crozz,
Being new to this it took me a while to work it out, I modified the approach slightly to just include the line:

#Lesson code
new_examples = example_gen_chain.apply_and_parse(
    [{"doc": t} for t in data[:5]]
)

# Additional line to review qa_pairs keys from LLM generated examples
new_examples = [item['qa_pairs'] for item in new_examples]

I also had to change the second last section as follows to get the predicted grade

print("Predicted Grade: " + graded_outputs[i]['results'])

Thanks for pointing me in the right direction,
Will

1 Like