C4W4: Capstone part 2: Issue Creating Glue Catalog Database – Already Exists Exception

[khusha]

Hi,
I ran into an issue while working on the AWS Glue module in the capstone project. When trying to create the Glue Catalog Database de_c4w4a2_silver_db, I received the following error:
Error: Create Database – Already Exists Exception: Database already exists.

I had already created the de_c4w4a2_silver_db database earlier in the process, so Terraform is now failing when trying to create it again.
Could you please advise on how I should handle this?

[hawraa.salami]

@khusha at what step did you get this error?

Was it in step 2.3? If so, did you make sure to run terraform apply in this order:

terraform apply -target=module.extract_job
terraform apply -target=module.transform_job
terraform apply -target=module.serving

Please let me know what previous steps you took before you got this error so you can better know what to do next time to avoid getting this error?

[khusha]

Hi ,
Yes, I got this error during Step 2.3 when running:
terraform apply -target=module.transform_job .Before that, I followed the steps in this order:

1. Ran terraform init
2. Applied the extract job module:
   terraform apply -target=module.extract_job
3. Then I moved to transform_job and ran:
   terraform apply -target=module.transform_job That’s where the error occurred saying the Glue database de_c4w4a2_silver_db already exists.
   I believe this happened because I had previously created the Glue database manually while trying to troubleshoot something earlier. So when Terraform tried to create it again, it threw the Already Exists Exception.

[hawraa.salami]

@khusha Alright, you could have deleted it manually in the console and tried again the terraform apply (for the transform module).

So for next time, when you try again the lab, try not to create it manually. But you could always manually delete it.

[khusha]

Hi ,
I wanted to clarify what happened with the assignment. Initially, when I ran the glue_json_transformation_job, it didn’t run properly due to a configuration issue. I submitted the assignment at that point.
Later, when I tried restarting it, the job failed with an error saying the Glue Catalog Database de_c4w4a2_silver_db already exists. I believe it was created during the earlier attempt, and now when I re-run the job, it tries to create it again and throws an AlreadyExistsException.
Just wanted to let you know I didn’t create the database manually — this seems to be from the initial execution. Could you please let me know the correct way to handle this?
Thanks for understanding!

[hawraa.salami]

@khusha Every time you start a new session (after the first session of 3 hours ends), you start with a fresh AWS account with no resources from the previous session. (unless the AWS account has some issues). So just make sure you’re doing the terraform apply, module by module, as stated in the lab instructions.

I can see you now have a current lab session and I checked the resources in your AWS account, I don’t see that catalog database in your account. Is everything alright with the current session? If not, can you explain where you are and share a screenshot of any error you’re encountering?

[khusha]

Thank you for checking and confirming that the catalog database isn’t currently in the account. I think what happened is — during my previous session, I ran into an issue where the glue_json_transformation_job didn’t execute properly. When I returned and started a new session, I followed the instructions again, but now I’m getting an error when running Terraform:
Error: creating Glue Catalog Database (de_c4w4a2_silver_db): AlreadyExistsException: Database already exists.
Since you mentioned it should be a fresh AWS environment, I’m a bit confused as to why this error is happening. I’ve been applying the modules one at a time as instructed.
I’m attaching a screenshot of the error and would really appreciate your guidance on how to proceed from here.

[hawraa.salami]

@khusha I think in this case, I would need to assign to you a new AWS account. I have your lab id. Not sure if you’re currently active in the current session, I’ll wait and then refresh your AWS account.

[hawraa.salami]

@khusha I refreshed your account. Please let me know when you try again.

[junming1202]

Hi, I encountered the same issue here. I got the error below when running “terraform apply -target=module.transform_job -no-color 2> errors.txt”. I looked into AWS Glue console data catalog table but couldnt find any existing data catalog table. Can you please help me out here? Btw, i also find that the VS Code crashes very often when i was doing the assignment. I have successfully run the transform_job module during the first time doing the lab assignment. However, i keep getting this error after the lab session windows crashes or down. The “Reload window” appears when it stops working.

---

Error: creating Glue Catalog Database (de_c4w4a2_silver_db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: 0560968c-abfe-4935-a068-9782b3f57d88, AlreadyExistsException: Database already exists.

with module.transform_job.aws_glue_catalog_database.transform_db,
on modules/transform_job/glue.tf line 1, in resource “aws_glue_catalog_database” “transform_db”:
1: resource “aws_glue_catalog_database” “transform_db” {

[hawraa.salami]

@junming1202 Can you try again the lab? There was an issue in your AWS account, I re-assigned a different AWS account.

• When you try again the lab, make sure to follow the steps of the lab in the order they’re stated

• We contacted Coursera regarding the disconnection issue to understand the reason, we’re still waiting for their reply.

  • if you ever get the disconnection issue, you can close the window, and then open the lab again. Please avoid rebooting the lab in the middle of a session
  • when you open a new terminal, make sure to always type source scripts/setup.sh
  • and then continue the lab from where you left off

• Another tip: you can append to any command this: || echo "$?" (i.e., any_command || echo "$?"), this will prevent the terminal from crashing and prints the error in the terminal.

[junming1202]

Hi @hawraa.salami,
Thanks for looking into this. I went to try the lab again just now but somehow the “GO TO AWS CONSOLE” link is not working now. Can you please help take a look?

Thanks.