I get following error when run:
terraform apply -target=module.extract_job -no-color 2> errors.txt
Error: creating Glue Connection (de-c4w4a1-connection-rds): operation error Glue: CreateConnection, https response error StatusCode: 400, RequestID: 8c06dad1-755d-440b-9364-014b878afbb0, AlreadyExistsException: Connection already exists.
with module.extract_job.aws_glue_connection.rds_connection,
on modules/extract_job/glue.tf line 2, in resource “aws_glue_connection” “rds_connection”:
2: resource “aws_glue_connection” “rds_connection” {
Error: creating IAM Role (de-c4w4a1-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: ff06c055-6aa7-4800-9f7b-35ae6aa52988, EntityAlreadyExists: Role with name de-c4w4a1-glue-role already exists.
with module.extract_job.aws_iam_role.glue_role,
on modules/extract_job/iam.tf line 1, in resource “aws_iam_role” “glue_role”:
1: resource “aws_iam_role” “glue_role” {
Hello @Tan_Quoc_Bao_Nguyen,
This error happens when the lab server reboots when you terraform apply. You could wait for a new lab session to clear those resources or they could be deleted manually from the AWS console. Thank you:
- Select
de-c4w4a1-connection-rds from AWS glue > connections and delete:
- Select
de-c4w4a1-glue-role from IAM > Roles and delete and type the role name:
I get following error after run 3 first glue job, could you help me to fix that?
Hello @Tan_Quoc_Bao_Nguyen,
I could reproduce the issue, it seems you have a typo in the de-c4w4a1-api-extract-job.py script. Probably in line 56-57 that you use the request end/start dates:
request_api_url = (
f"{api_url}?start_date={request_start_date}&end_date={request_end_date}"
I think you used request_start_dt instead. Thank you
I get following error:Error: creating Glue Catalog Database (de_c4w4a1_silver_db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: fedd46ff-e236-4337-9ec8-6a81d4b56469, AlreadyExistsException: Database already exists.
with module.transform_job.aws_glue_catalog_database.transform_db,
on modules/transform_job/glue.tf line 2, in resource “aws_glue_catalog_database” “transform_db”:
2: resource “aws_glue_catalog_database” “transform_db” {
When I run the second Terraform apply for transform jobs, could you help me to fix it?
Hello @Tan_Quoc_Bao_Nguyen,
You could try to manually delete it from AWS glue databases. If it doesn’t appear there and you get the same error with Insufficient Lake Formation. Please submit this form for a lab refresh, note it takes 2 business days to complete. Thank you
1 Like
how would I know whether my problem is resolved?
Hello @Tan_Quoc_Bao_Nguyen,
You should be notified in 1-2 business days in the email you submitted in the form. It should say something like this. Thank you:
The issue you reported in Data Engineering Lab Issue report form regarding your lab “Week 4 Assignment 4: Capstone Project Part 1 - ETL and Data Modeling“ of the ”Course 4 - Data Modeling, Transformation, and Serving” has been resolved. You may try opening the lab again.
