Week2_lecture video

[Basira_Daqiq]

This is a screenshot from the lecture video ‘Hellow World with tf. Transform’, I am really kinda confused about what is happening here:

so I think based on the lecture that the preprocessing_fn is a pre-defined function run by the tft_beam.AnalyzeAndTransformDataset, and the result is a tuple returned to the raw_data, and raw_data_metadata,

some questions:
what does the second line ’ with …’ do?
where is the data passed in, what is the input/ where is it getting the raw data from?

Thanks for your help!

[chris.favila]

Hi Basira. I think you’ll get a better picture of what’s happening when you try the ungraded lab after that. That block of code forms a pipeline where the raw data and metadata are fed to the preprocessing function. The 2nd line opens a context manager, which you can think of as a temporary database that the pipeline can use. Tensorflow Transform follows Apache Beam syntax and you can browse some interactive notebooks here to see the typical setup.