I am able to run the 4.1 Landing Zone part in Terraform. (got ‘SUCCEED’ in the three Glue Jobs)
But when i go to the next section 4.2 Transformation Zone
I keep getting a FAILED when I run the glue jobs there
The latest error is:
I don’t know where or how to trouble shoot this.
Hello @weiweiwong
In line 93 of the de-c4w4a1-transform-songs-job.py file, the code was not able to access the “duration” column of the df data frame. This data frame is read from a csv file in S3 in the address s3://{source_bucket_path}/landing_zone/db_songs/ingest_on={ingest_date_str}/. ingest_date is read from terraform configs. Based on another issue you have raised, my guess is that the problem is with the --ingest_date variable in your terraform/modules/transform_job/glue.tf file.
Thank you again, Amir.
Hello @Amir_Zare! I ran into several issues in the past couple of days, some at my own fault and some system troubleshooting too. And here it comes: ever since I raised a first issue with Coursera, a system failure, and they kindly pointed me to one of the threads here, each and every time I turned to this topic for hints I was able to advance on my own. Your name popped up at almost every turn, it truly accumulated to a solid definition of Mentor here. Therefore, I briefly wanted to say ‘thank you’ for the coaching! And on this occasion, another big ‘thank you’ to the whole team of people involved in this excellent course–the presentation, the materials, length and depth, the practical learning environment etc. What a great job!!! 5/5 stars & <3! Happy learning everyone!
Hello @M4a
Thank you for your warm message; it really made me smile. I’m so glad that you have enjoyed your journey and that we’ve been able to help. Cheers! 
