Week 4 Programming Assignment: Assignment 3 Lab Environment Issues

Course Q&A Data Engineering Introduction to Data Engineering
,
1

The Programming Assignment: Assignment 3: Building End-to-End Batch and Streaming Data Pipelines Based on Stakeholder Requirements keeps getting disconnected towards the middle of then steps. Somewhere around Step 2 (Creating vector DB) or Step 4 (Streaming Pipeline). This causes the the Lab to reload and then when I run the setup files again and try to run terraform apply, I get the below error. Its either this disconnection issue or the terminal gets terminated abruptly which is not letting me complete this assignment and this course(Did not face any lab issue in previous assignments of this course). I think I filled some form for a lab reset from some other post, can you please check if that request is received and whether that is the appropriate solution for this issue.

Error: creating Glue Catalog Database (de-c1w4-ml-db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: 36149771-1734-40fa-8a49-6d9e6ca015a4, AlreadyExistsException: Database already exists.

Error: creating IAM Role (de-c1w4-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: 8642a2ff-7110-43a7-b55c-0accbe04298d, EntityAlreadyExists: Role with name de-c1w4-glue-role already exists.

2

Hi, did you re-attempt to complete the lab?

From the error, I looks like tou are trying to provision some resources that are already available.

3

Yeah, managed to complete the lab. I had to manually delete the resources from aws console.

4

Excellent @Vikram_Rao So wil close this post then. We are glad that this is working now!