C4W4A2 - Capstone Lab - Error when trying to run glue job

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
1

Hi - I am on section 2.4 of the lab, and i have the following error in my terminal. I did not encounter this before when we had to run the other glue jobs. There seems to be a permissioning issue?

Outputs:

data_lake_bucket = “de-c4w4a2-058264501359-us-east-1-data-lake”
glue_api_users_extract_job = “de-c4w4a2-api-users-extract-job”
glue_catalog_db =
glue_json_transformation_job = “de-c4w4a2-json-transform-job”
glue_rds_extract_job = “de-c4w4a2-rds-extract-job”
glue_role_arn = “arn:aws:iam::058264501359:role/Cloud9-de-c4w4a2-glue-role”
glue_sessions_users_extract_job = “de-c4w4a2-api-sessions-extract-job”
glue_songs_transformation_job = “de-c4w4a2-songs-transform-job”
scripts_bucket = “de-c4w4a2-058264501359-us-east-1-scripts”
serving_schema = “deftunes_serving”
transform_schema = “deftunes_transform”
(jupyterlab-venv) abc@439bf7f51172:~/workspace/terraform$ aws glue start-job-run --job-name glue_api_users_extract_job | jq -r ‘.JobRunId’

An error occurred (AccessDeniedException) when calling the StartJobRun operation: User: arn:aws:sts::058264501359:assumed-role/VSCodeInstanceRole/i-0603aea5df47196f8 is not authorized to perform: glue:StartJobRun on resource: arn:aws:glue:us-east-1:058264501359:job/glue_api_users_extract_job because no identity-based policy allows the glue:StartJobRun action
(jupyterlab-venv) abc@439bf7f51172:~/workspace/terraform$ aws glue start-job-run --job-name glue_sessions_users_extract_job | jq -r ‘.JobRunId’

An error occurred (AccessDeniedException) when calling the StartJobRun operation: User: arn:aws:sts::058264501359:assumed-role/VSCodeInstanceRole/i-0603aea5df47196f8 is not authorized to perform: glue:StartJobRun on resource: arn:aws:glue:us-east-1:058264501359:job/glue_sessions_users_extract_job because no identity-based policy allows the glue:StartJobRun action
(jupyterlab-venv) abc@439bf7f51172:~/workspace/terraform$ aws glue start-job-run --job-name glue_rds_extract_job | jq -r ‘.JobRunId’

An error occurred (AccessDeniedException) when calling the StartJobRun operation: User: arn:aws:sts::058264501359:assumed-role/VSCodeInstanceRole/i-0603aea5df47196f8 is not authorized to perform: glue:StartJobRun on resource: arn:aws:glue:us-east-1:058264501359:job/glue_rds_extract_job because no identity-based policy allows the glue:StartJobRun action
(jupyterlab-venv) abc@439bf7f51172:~/workspace/terraform$

2

Hi @weiweiwong - When you run the glue job, you have to use the actual name of the glue job not the name of the output variable.

So one of the outputs you got:
glue_api_users_extract_job = “de-c4w4a2-api-users-extract-job”

When you run the glue job, the command should be:
aws glue start-job-run --job-name de-c4w4a2-api-users-extract-job | jq -r ‘.JobRunId’

According to the command you shared, you used the name of the output variable:
aws glue start-job-run --job-name glue_api_users_extract_job | jq -r ‘.JobRunId’
and I think that’s why you got the error.