Programming Assignment: Assignment 5: Capstone Project Part 2 - Data Quality and Orchestration and Visualization CRASHES!

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
,
1

When I respond with “yes” after running the terraform apply -target=module.transform_job command the system disconnects - multiple times. I have had problems with this one lab for over a week. Every other lab worked fine without incident. Can you reset this lab in my account so I can start over fresh?

2

Hello @rfink2024
You can request a lab refresh via this form. Please, wait up to 3 business days after filling out the form and then try again.

3

I’ve filled out the form. When you refresh my lab, can you do a complete refresh? I’ve asked for a refresh twice and after each refresh when I go back into my lab, I’ve noticed that it’s not completely cleaned out. I’d prefer to start assignment 5 capstone project part 2 all over. Can you do that?

4

Hello @rfink2024
The lab refreshes are handled by the staff, and I personally don’t have the access to do that. However, I believe your lab must be completely refreshed after each request, and no components should linger from the past sessions. Is your experience different than this?

5

I got similar experience in the capstone project.

  1. one is existing resources issues, I tried to use import existing resources into terraform state and delete resource amnaully, both failed.

I have made a mistake in the extract job glue.tf but when I modify it and want to redeploy the terraform has this
module.serving.redshift_schema.transform_external_from_glue_data_catalog: Creation complete after 0s [id=106749]
╷
│ Error: creating Glue Connection (de-c4w4a2-connection-rds): operation error Glue: CreateConnection, https response error StatusCode: 400, RequestID: 29703988-94c5-43a7-a938-92098e4d3b63, AlreadyExistsException: Connection already exists.
│
│ with module.extract_job.aws_glue_connection.rds_connection,
│ on modules/extract_job/glue.tf line 2, in resource “aws_glue_connection” “rds_connection”:
│ 2: resource “aws_glue_connection” “rds_connection” {
│
╵
╷
│ Error: creating IAM Role (de-c4w4a2-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: f7b028de-9493-4e53-b7c7-6b6da674420d, EntityAlreadyExists: Role with name de-c4w4a2-glue-role already exists.
│
│ with module.extract_job.aws_iam_role.glue_role,
│ on modules/extract_job/iam.tf line 1, in resource “aws_iam_role” “glue_role”:
│ 1: resource “aws_iam_role” “glue_role” {
│

  1. The second issue is that terminal crashes so frequently not sure why.

If we can just have the lab to by default to set up the part 1 stuff maybe the experience can be a little smoother.

I have submitted lab refresh request already.