Hi everyone,
I’m currently working on **Assignment 2: Building a Data Lakehouse with AWS Lake Formation and Apache Iceberg** in the “Data Storage and Queries on AWS” course.
**Assignment link:**
### Issue:
When trying to run the AWS Glue job as per the lab instructions, I get the following error: AccessDeniedException: User: arn:aws:sts::113591280722:assumed role/voclabs/user4176883=krthmdabtwlyis not authorized to perform: glue:StartJobRun on resource: arn:aws:glue:us-east-1:113591280722:job/lakehouse-lab-rds-ingestion-etl-job
### What I’ve Tried:
I attempted Terraform apply to deploy the infrastructure.
I do not have the permission to add policies via the AWS Console.
IAM role updates via Terraform fail due to lack of permissions or existing roles.
please help grant `glue:StartJobRun` permission to the voclabs-assumed role or guide me on how to proceed without console access?
Note: I did refresh the lab assignment via the form but didnot get any initmation if its refreshed and i tried after two days but still the same error exists.
I tried using the job names exactly as provided in the Terraform output, but the same AccessDeniedException error persists. Please find the screenshot attached below for your reference:
@Amir_Zare i had already submitted this form last week, i didnt get any notification of lab refresh but still tried the lab work after 2-3 days and still got the same issue.
@harshi1 why does the project have the name “lakehouse-lab”? Did you choose this name manually? There’s no problem with your aws account but the project name should be de-c3w2a1 (defined as a terraform variable). Did you make sure to run source scripts/setup.shbefore running the terraform commands? If you ever had to open a new terminal, you would need to run again this command source scripts/setup.sh (to automatically assign values to terraform variables), otherwise you will be prompted to enter them manually.
I initially saw project = “lakehouse-lab” in the Terraform output because I had not sourced scripts/setup.sh before running terraform apply. Later, I did run source scripts/setup.sh and confirmed the environment variable was correctly set using echo $TF_VAR_project, which returned de-c3w2a1. However, the Terraform output still shows project = “lakehouse-lab”, likely because this value was hardcoded earlier in the terraform.tfvars file or provided as a default in variables.tf before sourcing the setup script.I’m currently encountering an AccessDenied error when trying to run the Glue job. I attempted to add the required permission (glue:StartJobRun), but I do not have sufficient access rights to modify the IAM role policies. Due to these permission limitations, I wasn’t able to update the role to include the necessary actions.Please find attached the screenshot for echo $TF_VAR_project, which confirms the value is correctly set to de-c3w2a1**:**
(AccessDeniedException: User: arn:aws:sts::113591280722:assumed role/voclabs/user4176883=krthmdabtwlyis not authorized to perform: glue:StartJobRun on resource: arn:aws:glue:us-east-1:113591280722:job/lakehouse-lab-rds-ingestion-etl-job)
I guess the first time that you run the code with wrong inputs has broken the terraform components, and you don’t have the access to amend it. I suggest you fill out the lab refresh form again and this time try to do it all correctly from the beginning.
Im running into the same issue. Whenever im deploying the terraform its asking bunch of values like curated_db_name, vpc_id so and so forth - i have checked in the cloudformation and setup.sh files and supplied the values to it but again i get this error.
You are probably missing step 1.2.4. The values to terraform variables are set when you run source scripts/setup.sh. If your terminal instance crashes, you need to rerun this command before continuing on.
I followed the steps including running source scripts/setup.sh step which is 1.2.4. After that running terraform for the glue jobs in 3.2 step works perfect for landing etl. But when deploying iceberg jobs on step 4.3 that’s when it’s asking values from setup file for the variables to which I provide it ( which I should not correct? - because it will pull from setup.sh file). It throws an error saying that ec2 has no enough permissions when I go to aws glue on the consol that says permission error.
Hi Amir,
I tried refreshing the lab, but that didn’t fix the issue. The Glue role was missing Lake Formation access, and terraform kept trying to recreate a role that already existed. Once I switched to using the existing Glue role ARN and granted the right permissions, the jobs finally ran successfully.
Hi @Amir_Zare As soon as I run the terraform init command AWS starts reconnecting, this is happening again and again. Not able to proceed further. Please help to resolve this issue
Programming Assignment: Assignment 2: Building a Data Lakehouse with AWS Lake Formation and Apache Iceberg
