C4W4 - Capstone Project Part 1 and 2 - Glue Catalog Database Already Exists Error

[eudata]

I am working through Capstone Part 1 and I’m receiving the error below when I run terraform apply for the transformation glue job in section 4.2 Transformation Zone. This is preventing the glue jobs from being created and as a result I am unable to complete anything past this section. I am experiencing the same issue in Capstone Part 2.

I completed the lab refresh form yesterday, but I’m not sure if my labs have been refreshed yet. My subscription ends this Sunday so I need to complete these tasks before then.

[Amir_Zare]

Hello @eudata
I am sorry for the inconvenience. Generally, the lab refresh takes up to 3 days to complete since they are done manually. You will be notified when it is done.
@esanina can we prioritize @eudata’s lab refresh since their subscription deadline is coming soon?

[esanina]

Hello @eudata, thank you for submitting the form and for your patience. All of the lab accounts related to the reported issues have been refreshed, it should be working now. Please let me know if you have any issues.

During the lab if you have problems with some existing resources, you can delete them manually or run terraform destroy command.

[eudata]

@esanina @Amir_Zare confirming the issue has been resolved with the refresh. Appreciate your assistance with this!

[sinh_x]

Hi Esania,

I got simial error similar error to this original post when running command terrafom apply -target=transform_job which result

│ Error: creating Glue Catalog Database (de_c4w4a2_transform_db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: 734bb3bf-bc13-496d-891b-bb6aa082268d, AlreadyExistsException: Database already exists.
│ 
│   with module.transform_job.aws_glue_catalog_database.transform_db,
│   on modules/transform_job/glue.tf line 1, in resource "aws_glue_catalog_database" "transform_db":
│    1: resource "aws_glue_catalog_database" "transform_db" {
│ 

I already tried terra destroy but it still result the same errors. Should I create another post?

Thank you my course deadline is on Dec-29, 2024!

[Amir_Zare]

Hello @sinh_x
You can fill out the lab refresh form and try again after it’s done.

[vishal1238]

hello @esanina , @Amir_Zare

I am currently working on Assignment 5: Capstone Project Part 2 - Data Quality, Orchestration, and Visualization. While executing the transformation Glue jobs (glue_json_transformation_job and glue_songs_transformation_job) using Terraform, both jobs are failing.
issue: AnalysisException: Unable to verify existence of default database: com.amazonaws.services.glue.model.AccessDeniedException: Insufficient Lake Formation permission(s) on default (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException;

any solution?

[uebelvan]

Hi, are you saying since my lab timed out yesterday due to an issue I couldn’t resolve without help from this forum, it takes up to 3 days for the lab to be MANUALLY refreshed? That is an unacceptable waste of students’ time. Please refresh mine and let me know once it is so I can try and wrap up this course.

[Amir_Zare]

Hello @vishal1238
I’m sorry for the inconvenience. Did you execute extract jobs and wait for them to succeed before running the transformation jobs?

[Amir_Zare]

Hello @uebelvan
I’m sorry that you have to wait. Please, fill out the form, and we will inform you as soon as the refresh is done so that you can continue with the lab.

[uebelvan]

Ok thanks. Today is the last day of my subscription, so would be nice to get priority fix on this. Didn’t anticipate running into so many issues with the last lab.

[vishal1238]

Yup @Amir_Zare , after all three extract jobs SUCCEEDED, I ran the transform jobs, but they FAILED. I tried changing the ingestion date to the current date instead of 2020-02-01, but it doesn’t seem to be the issue since it’s mainly for partitioning. Please help resolve this issue, as I can’t access the certificate.

[vishal1238]

hello @esanina , i tried this, still the transform jobs are failing

[Amir_Zare]

Hello again @vishal1238
Can you please fill out the lab refresh form and see if that helps?

[vishal1238]

Hi @Amir_Zare , I’ve submitted the form and hopefully, the issue will be resolved in the next couple of days. I’ll give it another try on 23-12-2024. Thanks so much for your help!

[uebelvan]

Hi Amir, any updates here? I filled the form out as per your instructions but haven’t been notified about a lab refresh yet. Thanks!

[Amir_Zare]

Hello @uebelvan
Your lab has been refreshed. Can you please try again to see if the issue is resolved?

[cristianvasquez]

I need my lab to be refreshed as well, I’m encountering the same errors

[Amir_Zare]

Hello @cristianvasquez
Have you filled out the lab refresh form?