C4W4 - Capstone - Part 1: Error: reading S3 Bucket

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
,
1

Hi everyone,

I did all the steps correctly, but when I ran terraform plan, I received the error below:

Error: reading S3 Bucket (de-c4w4a1-989262269729-us-east-1-data-lake): operation error S3: HeadBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "https://de-c4w4a1-989262269729-us-east-1-data-lake.s3.amazonaws.com/": dial tcp: lookup de-c4w4a1-989262269729-us-east-1-data-lake.s3.amazonaws.com on 127.0.0.11:53: no such host
  with module.extract_job.data.aws_s3_bucket.data_lake,
  on modules/extract_job/s3.tf line 2, in data "aws_s3_bucket" "data_lake":
   2: data "aws_s3_bucket" "data_lake" {

@Amir_Zare , could you take a look?

Thanks!

2

I solved the issue fixing the script location:

    script_location = "s3://${var.scripts_bucket}/de-c4w4a1-extract-songs-job.py"

, but now I’m facing another one in the step 4.2.6:


module.extract_job.aws_iam_role.glue_role: Creating...
module.extract_job.aws_glue_connection.rds_connection: Creating...
╷
│ Error: creating Glue Connection (de-c4w4a1-connection-rds): operation error Glue: CreateConnection, https response error StatusCode: 400, RequestID: 7b3a60c3-9fe9-4891-9b1d-6e59e788e116, AlreadyExistsException: Connection already exists.
│ 
│   with module.extract_job.aws_glue_connection.rds_connection,
│   on modules/extract_job/glue.tf line 2, in resource "aws_glue_connection" "rds_connection":
│    2: resource "aws_glue_connection" "rds_connection" {
│ 
╵
╷
│ Error: creating IAM Role (de-c4w4a1-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: a3bcdb4d-0a08-463e-9366-4b608671d438, EntityAlreadyExists: Role with name de-c4w4a1-glue-role already exists.
│ 
│   with module.extract_job.aws_iam_role.glue_role,
│   on modules/extract_job/iam.tf line 1, in resource "aws_iam_role" "glue_role":
│    1: resource "aws_iam_role" "glue_role" {

How can I fix it?

3

I solved the previous issue following these steps: C1W4 - terraform apply does not work - #2 by Georgios

But now I have another issue:


Error: creating Glue Catalog Database (de_c4w4a1_silver_db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: 3778781d-938b-4806-a932-832567706a60, AlreadyExistsException: Database already exists.

  with module.transform_job.aws_glue_catalog_database.transform_db,
  on modules/transform_job/glue.tf line 2, in resource "aws_glue_catalog_database" "transform_db":
   2: resource "aws_glue_catalog_database" "transform_db" {
4

I solved the previous issue running terraform apply -target=module.serving

Thanks!