Getting the errors in start of the code, doubt of distorted AWS configuration in the module: Assignment 4: Capstone Project Part 1 - ETL and Data Modeling

When creating a post, please add:

  • Module # must be added in the tags option of the post. (ex: Module-1) Module 4
  • Link to the classroom item you are referring to: capstone assigment 1
  • Description (include relevant info but please do not post solution code or your entire notebook)

Hello @Noddy,

Sorry for the inconvenience you are facing. It seems you are having issues with terraform apply or successfully running extract/transform jobs. There are a few other learners who had those issues with terraform because of existing resources when running terraform apply (you could check the errors.txt file when if you run terraform apply -no-color 2> errors.txt).

If you use the forum search tool you could see people who submitted the lab refresh form, that takes 1-2 business days to complete. This form also helps with Insufficient Lake Formation issues that cause the jobs fail so you can’t complete the lab. Please take the time in the meantime to search or give a description which resource/job cause the issue. Thank you

Hello Georgios, i did the lab refresh but continuously getting the terraform crashing and glue job failing due to iam issues:oder@b28f5984edc5:~/project/terraform$ aws glue get-job-run --job-name de-c4w4a1-api-users-extract-job --run-id jr_0f74f642ecd157fb1e8cc3ad821fdc148b530a604e29324e5f80acc555bc90c0 --output text --query “JobRun.JobRunState”

An error occurred (AccessDeniedException) when calling the GetJobRun operation: User: arn:aws:sts::027931171522:assumed-role/voclabs/user4008267=qfnjbwcmaqgp is not authorized to perform: glue:GetJobRun on resource: arn:aws:glue:us-east-1:027931171522:job/de-c4w4a1-api-users-extract-job with an explicit deny in an identity-based policy

Hello @Noddy,

Sorry for the inconvenience but unfortunately I couldn’t reproduce the issue. It looks like you succesfully used start-job-run in all the extract jobs, unless you would be getting an AccessDeniedException if the name was incorect. The issue would be different if you were not getting the AccessDeniedException from get-job-run command only for api-users-extract-job but for rds-extract-job as well. However you could use the AWS console to check the outcome and see why the jobs are failing: