ExampleGen in Production

I was wondering how is ExampleGen used in production? I understand that their outputs can be feeded into the TFDV components of TFX to validate schema, skews, and others.

But I get lost since ExampleGen generates a train & eval split, and I don’t find why you would split the data in production into train & eval.

Any answers?


Hello @rzagmarz and wellcome to the forum,

I think that the eval split can be used for model monitoring in production environment, that’s my opinion. It can help you to make decisions when to regularly update model.

The thing it is that is a little confusing to make train/dev split in production. My understanding is that since TFX could also be used on the modeling/development phase, using ExampleGen makes a lot of sense. But when you deploy your model by also using TFX, ExampleGen is not used anymore, you use more the TFDV components to check for skews & others.

I’m still somehow confused. Thanks!