Building and Evaluating Advanced RAG Applications

For the first notebook: Lesson 1: Advanced RAG Pipeline

There is a an import utils, bringing in a package with various functions such as get_openai_api_key() and get_prebuilt_trulens_recorder

Since this is not a standard python package, how do I replicate this functionality in my own environment (off platform)? Can this package be pip-installed from somewhere (not by “pip install utils” which apparently installs a totally different package).

Can any transparency be offered regarding this? Or is all this part in a black box?

Thanks,
Josh

Never mind; I guess it’s in the utils.py which is a local file on the platform. Sorry I’m not not familiar with this cloud platform.

Those are the custom functions written by the team and only work on the platform. The alternatives to those functions are as below:

OpenAI API Key setup: https://platform.openai.com/docs/quickstart?context=python

Trulens Recorder: You can use TruLlama instead of their custom function. However, TruLlama is not providing the required metrics such as Context and Answer Relevance, and Groundedness.
Check out TruLlama implementation here: https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.16.0/trulens_eval/examples/quickstart/llama_index_quickstart.ipynb

Hope this helps!

Ah OK,
Are you saying the functions in utils.py don’t work off platform? (My cursory glance suggests that they would.) Anyway, OpenAI API key setup is basically just a way to assign a password within the notebook, without it actually saving it in the notebook. ( import getpass is another alternative)

Regarding the more substantial TruLens package: So basically are you saying this tutorial is teaching how to evaluate RAGs using a proprietary tool (Trulens) as one of its key components? Or is TruLens open source? If it is, why couldn’t it be run off platform?

Sorry for so many questions. Just trying to understand how I can deploy what I learn in this tutorial after/outside the tutorial itself.

On my PC (Ubuntu/WSL2, trulens-eval 0.18.2 installed with pip), the notebooks run well & the utils.py files execute without errors.