How to load a Chroma_collection.add with documents from multiple files

Short Course Q&A Advanced Retrieval for AI with Chroma
1

I tried to use SimpleDirectoryReader to load the content of 100’s of small csv files that have text content in them:
reader = SimpleDirectoryReader(input_dir=“./data-in-documents/”)
data_from_CSVdocuments = reader.load_data()

However, when I try to load data_from_CSVdocuments into chroma_collection.add, I get a “TypeError: ‘Document’ object is not subscriptable” error

chroma_collection.add(ids=ids, documents=data_from_CSVdocuments)

Does anyone have a suggestion on how to load the content of many CSV files that I can, in turn, pass to add to a Chroma collection? All of the examples provided either are of a small number of files, or one PDF.

2

Chroma does require the parameter to support an array like interface and hence the bug.
We have 2 options here:

  1. Write code to add document from SimpleDirectoryReader from llama_index to chroma compatible format.
  2. File a ticket on github for a feature request with the team that owns SimpleDirectoryReader