C4W4 CapStone2: "ErrorMessage": "AnalysisException: Insufficient Lake Formation permission(s) on de_c4w4a2_silver_db (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 5064fe94-f23e-4f65-84d1-677b694925a7; Proxy: null)",

CAPSTONE1 COMPLETED

CAPSTONE2

JOBS 1,2,3 in Terraform succeeded

JOB4

JOB4=‘de-c4w4a2-json-transform-job’
JOB_RUN_ID4=aws glue start-job-run --job-name $JOB4 | jq -r '.JobRunId'

aws glue get-job-run --job-name $JOB4 --run-id $JOB_RUN_ID4
“ErrorMessage”: “AnalysisException: Insufficient Lake Formation permission(s) on de_c4w4a2_silver_db (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 5064fe94-f23e-4f65-84d1-677b694925a7; Proxy: null)”,

JOB5=‘de-c4w4a2-songs-transform-job’
JOB_RUN_ID5=aws glue start-job-run --job-name $JOB5 | jq -r '.JobRunId'

“ErrorMessage”: “AnalysisException: Insufficient Lake Formation permission(s) on de_c4w4a2_silver_db (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 97fbde83-97cb-4140-8bc0-9203cf0d6392; Proxy: null)”,

Can you please help?

Hello @eadgc5,

Could you submit this form, it should fix Insufficient Lake Formation but it takes 2 business days to complete. So @hawraa.salami could check it and try their best to refresh your lab at the earliest. Thank you

Having gone past this - I was seeing a Glue job error in 2.4
I fixed it and in trying to rerun - I saw terminal hang. Reconnect Now. Stuck for ever.

cd terraform;

terraform apply -no-color 2> errors.txt;

terraform init;

terraform plan;

terraform apply -target=module.extract_job;

terraform apply -target=module.transform_job;

terraform apply -target=module.serving;

terraform apply -no-color 2> errors.txt; was used subsequently

but still Window pops up saying “reconnecting”. This is annoying. It happens too often.

LAB ID: qaaasbueolvj

How do I unblock myself?

Error: creating Glue Connection (de-c4w4a2-connection-rds): operation error Glue: CreateConnection, https response error StatusCode: 400, RequestID: 5d57a9e0-d2ad-4d27-a4c9-ea17840012d5, AlreadyExistsException: Connection already exists.

with module.extract_job.aws_glue_connection.rds_connection,
on modules/extract_job/glue.tf line 2, in resource “aws_glue_connection” “rds_connection”:
2: resource “aws_glue_connection” “rds_connection” {

Error: creating IAM Role (de-c4w4a2-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: a9062d83-d117-49f7-a960-e134682df284, EntityAlreadyExists: Role with name de-c4w4a2-glue-role already exists.

with module.extract_job.aws_iam_role.glue_role,
on modules/extract_job/iam.tf line 1, in resource “aws_iam_role” “glue_role”:
1: resource “aws_iam_role” “glue_role” {

Error: pq: Schema “deftunes_serving” already exists

with module.serving.redshift_schema.serving_schema,
on modules/serving/redshift.tf line 1, in resource “redshift_schema” “serving_schema”:
1: resource “redshift_schema” “serving_schema” {

Error: pq: Schema “deftunes_transform” already exists

with module.serving.redshift_schema.transform_external_from_glue_data_catalog,
on modules/serving/redshift.tf line 9, in resource “redshift_schema” “transform_external_from_glue_data_catalog”:
9: resource “redshift_schema” “transform_external_from_glue_data_catalog” {

seems like something not idempotent

please help unblock

terminal exits after terraform plan

Hello @eadgc5,

Sorry for the inconvenience It seems I could reproduce the issue and it happens in other labs with terraform as well. Those resources are created when terraform apply hangs and you have to reload the lab. Unfortunately you will have to manually delete them or try in a new lab session to clear the already exists errors. If it continues feel free to submit the form so a lab refresh clears them. Thank you

Not sure how to do it. Please help.

Hello @eadgc5,
You should be able to find them in the AWS console and delete them e.g.

  1. de-c4w4a2-connection-rds in AWS glue >connections :
  2. de-c4w4a2-glue-role in IAM > Roles :

It was also throwing an error on schema already existing etc. Any fix for that?

Hello @eadgc5,

I am not sure but a learner had a similar issue in this post and he had to remove the schemas from Redshift. Could you try in a new lab session that might clear those resources from the start. Hope it helps

Please help me on that as well :slight_smile:

Seeking help as I have been spending a lot of time on infra issues beyond me and I need to focus on other things. I am 99.9% done and just need the certificate to showcase on my profile.

@eadgc5 I understand it’s frustrating having one assignment left and running on those errors. You could always ask for a lab refresh if you think you can’t make progress. Perhaps a staff @hawraa.salami could assist you further. Hope it helps

Let me know when it has been refreshed

? let me know

my subscription is just getting extended for no reason or value… please expedite!

You should receive a notification when your lab is reset. It probably arrives by email.

Hello @eadgc5,

You should be getting notifications in the email you filled in the form you submitted. I just received an email like this yesterday, it shouldn’t take more than 2-3 business days after you submitted so please check it. Thank you

Should I submit a new form? Do you see one?