4 -curated zone

[lloyd.vince]

Hello,

I’ve attempted to resolve the errors in Assignment 2 (4 – Curated Zone) multiple times, but I’m still encountering issues. I’ve attached the screenshots below—could you please review them and help me identify the problem?

Thank you.

[Georgios]

Hello @lloyd.vince,

Your code in the .py files and transform_etl/glue.tf looks correct, I could reproduce your issue with the failed to start jobs. In your first run it seems the job has not been deployed and afterwards you didn’t include --job-name. Could you make sure you succesfully run terraform apply and deployed the jobs in AWS glue. Thanks

1. use --job-name to run the job or you get this error (e.g. aws glue start-job-run --job-name de-c3w2a1-ratings-transformation-job | jq -r '.JobRunId'):
   

   

2. If the job is not deployed with terraform you get this error:
   

3. Check in AWS glue if all the jobs are deployed and run them in the correct order:
   

   

[lloyd.vince]

Hi Georgio,

3.2 is running but 4.1 is not. I think there’s a problem with my code in
terraform/assets/transform_etl_jobs
de_c3w2a1_batch_transform.py
assets/transform_etl_jobs/de_c3w2a1_ratings_to_iceberg.py

I tried to use the get latest version but it doesnt refresh. Can you provide me the undedited files of these three (transform_etl_job).

Thanks

[lloyd.vince]

@Georgios

aws glue start-job-run --job-name de-c3w2a1-ratings-transformation-job | jq -r ‘.JobRunId’

All good. I fixed the problem. THanks