Stuck on Kubleflow and TFX usages

Hi, I’m Gabriel a Data Scientist, thanks for this space.

After some video of this specialization, mainly the course 4, I have some confusions in my mind about how and when use Kubleflow and TFX. I know that this frameworks are so powerfull, but in what context I’m suposed to use. Some problems on Data Science is very dinamyc, the source of data change along the time, the struct of data can also change, with my current experience I can see the power of this tools, but I can’t see when I can use.

Maybe if some of you guys that was have used in a some case, could give some example of success. Please, if some of you could, I really appreciate that.

Could you please explain why where to use TFX / Kubeflow is confusing?

Hi, thanks for repply, its just I work on small Projects, that dont need automatize all steps, I think that to use a automated pipelines end-to-end, the system has to be well defined, like, the flow of input data dont change to fast, like user data from a application, like netflix or some other problems like that. Imagine that the company that you work have some datas, that is not storaged consistently, you have to get that data from other coworkers, its very dificult to automatize all of steps. I think that a example of real world problem solved using this tools can help me visualize all things.

Got it. Will let you know if I come across such information.

Maybe making an automated ETL pipeline with Python and Apache Airflow would work.