During the course “Orchestrating Workflows for GenAI Applications”, we use airflow to run periodically. I have noticed how the entries are being duplicated, I have fixed this using other methods (moving the files to different folder after they are embedded for first time), but I was wondering if there is another way in Airflow.
Also I could use the equivalent to chromadb upsert in weaviate i guess.
I recommend you post your message in the forum area for that course.
The “AI Discussions” forum is for general topics.
1 Like