Advise on finding matching elements

Hi, a newbie here : )
How would you approach solving the following case:

  • Database of clothing items
  • Customer has a text input, where they enter a description of a clothing item they own, and want to find a “look” for it, where “look” is a set of different items (top+bottom+shoes+accessories+bag) that match best aesthetically together.

I’m trying to wrap my head around how to reason around this “together” part. For example: I enter “grey t-shirt with a rockband”. It’s not enough to get separately best matching shoes for this t-shirt and best matching skirt for this with t-shirt. All items, t-shirt and shoes and skirt should look great together.

Note: not considering image input, at this point its text. So I guess I will need to wire langchain or something to this system.

I would appreciate a hint in what direction I should be looking to.

Thanks!
D

The traditional machine learning method would be to start with a database of labeled “matching outfits”.

If you cannot find such a database, you’re rather stuck with pursuing that method.

Another method (much more complicated) would be to find a collection of images that show ‘matching outfits’. Then you train an image model to segment each image into separate components (like tops, shoes, accessories), and train it to recognize what parts go together (i.e. specific combinations are highly correlated).

It’s not a simple task.

1 Like

There are likely a lot of other methods, those are just two that came to mind quickly.

1 Like

Thank you so much, @TMosh !