Metadata filter before keyword & sematic search?

[shxan]

So, the video recommends Metadata Filter to be done AFTER conducting parallel keyword search and sematic search. Isn’t it more resource efficient to run Metadata Filter BEFORE the searches?

The scores should be the same and Metadata Filter is only run once.

[Girijesh]

Dear @shxan,

Good question.

Metadata Filter Before Search – Faster, uses fewer resources, but may miss relevant results.
Metadata After Search – Slower, uses more resources, but maximizes recall and supports dynamic filtering.

–
Keep Learning AI with DeepLearning.AI - Girijesh

[ribarola]

Hi @shxan

One reason that justifies the image in the video is the one expressed by Girijesh, with which I agree.

I’m leaving you a link to the post that refers to the same topic, there I gave my opinion; and you can also find other opinions.

Why metadata filtering is done after rather than before?

I don’t understand why metadata filtering is done after rather than before [image] If we are going to discard documents, it makes more sense to discard them right away before wasting computational time to compute semantic similarity

Regards

[balaji.ambresh]

@Girijesh Could you please provide an example where a relevant result might be missed if we apply metadata filter(s) before search?

@Girijesh

Metadata Filter Before Search – Faster, uses fewer resources, but may miss relevant results.

I’m referring to cases where we’re not using dynamic metadata filters to refine search results.