C4W4 Capstone Project Part 1 - Cannot continue after session expired

[calvinwy]

1. Session expired and restarting the assignment workspace
2. re-run the terraform procedure for the extraction steps
3. Errors appears saying something about the IAM already exist
4. Try to restart the workspace and clean using terraform destroy
5. Every time I run terraform apply, the terminal crashes
6. Tried to use the terraform apply -no-color 2> errors.txt to check the error message, it says: “Error: No configuration files. Apply requires configuration to be present. Applying without a configuration would mark everything for destruction, which is normally not what is desired. If you would like to destroy everything, run ‘terraform destroy’ instead.”

Please help, since I got a message that I need to finish the assignment by Tuesday to get the certificate and I have to work during weekdays, I have to finish it by tomorrow.

[calvinwy]

After step 6, I try to run terraform init and terraform plan again, it is prompting for input.

coder@95f4c4477deb:~/project/terraform$ terraform plan
var.catalog_database
  Curated DB name

  Enter a value: 

[calvinwy]

After waiting for a night, to get a fresh setup on the AWS, it seems to work the first time I run terraform. But the same thing happen after I proceed the exercise when I run the terraform part for the transform section.

[calvinwy]

I tried to capture the error at the output, it seems like problem on the AWS side:

Error: creating Glue Connection (de-c4w4a1-connection-rds): operation error Glue: CreateConnection, https response error StatusCode: 400, RequestID: bb57127a-cced-4e06-a5bd-e1bf4c3f6906, AlreadyExistsException: Connection already exists.

  with module.extract_job.aws_glue_connection.rds_connection,
  on modules/extract_job/glue.tf line 2, in resource "aws_glue_connection" "rds_connection":
   2: resource "aws_glue_connection" "rds_connection" {


Error: creating IAM Role (de-c4w4a1-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: aeb17fe1-e060-4ecb-97f5-b3c1ef091411, EntityAlreadyExists: Role with name de-c4w4a1-glue-role already exists.

  with module.extract_job.aws_iam_role.glue_role,
  on modules/extract_job/iam.tf line 1, in resource "aws_iam_role" "glue_role":
   1: resource "aws_iam_role" "glue_role" {


Error: pq: Schema "deftunes_serving" already exists

  with module.serving.redshift_schema.serving_schema,
  on modules/serving/redshift.tf line 2, in resource "redshift_schema" "serving_schema":
   2: resource "redshift_schema" "serving_schema" {


Error: pq: Schema "deftunes_transform" already exists

  with module.serving.redshift_schema.transform_external_from_glue_data_catalog,
  on modules/serving/redshift.tf line 12, in resource "redshift_schema" "transform_external_from_glue_data_catalog":
  12: resource "redshift_schema" "transform_external_from_glue_data_catalog" {

[calvinwy]

Hi @Georgios ,

I am in an urgency to complete this last project, it is the last piece for my certificate… I have received an email with the following message: "You are only eligible for the Specialization certificate if you have an active subscription and enrolled in all 4 courses before the April 8, 2024 deprecation date. After April 8, 2025, certificates will not be issued for Courses 2, 3, 4 or the full Specialization. "

Thanks in advance!!!

Calvin

[Georgios]

Hello @calvinwy,

Your second issue after you created the errors.txt file you had to run source scripts/setup.sh before you go to terraform commands since your terminal crashed after terraform apply -no-color 2> errors.txt. There is this post that might help you from other learners to delete those resources about your last issue. After waiting for a new lab session those resources seems to be cleared and you could also submit this form for a lab refresh, note that it takes 2 business days to complete.

The email message that you got is not in my knowledge and perhaps a staff @Mubsi knows better about active subscriptions and deadlines. Hope it helps

[calvinwy]

As I mentioned in the note above, I have to complete the course asap to obtain my certificate, and I have all requirements for the specialization already except the first part of the capstone project.

The lab is so poorly designed while the terraform comment cannot be ran after executing the “extraction” part of the exercise. I cannot believe I have to wait for 2 days every time to reset the infrastructure for each of the “extract”, “transform” and “serving” section.

I cannot wait for 2 more business days for the infrastructure reset while this will exceed the deadline, please help!!!

[Mubsi]

Hi @calvinwy,

If you come across issues related to the lab as described within the course. Please fill the form, and we shall try our best to reset your account at the earliest.

Depending on the number of queries we receive on the daily basis, it can take some time for attend to all of them.

Thank you for your patience,
Mubsi

[calvinwy]

I am still experiencing issue resetting the cloud deployment. This time, when I do terraform apply it says that the database already exist and then it crashed.

@Georgios , could you please give some guidance?

It seems to be the same issue as: C4W4 - Capstone Project Part 1 - Terraform Apply Error : Database Already Exists

Also, because terraform crashed, it seems like I need another AWS account reset again…

[calvinwy]

@Georgios any solution to the problem?

[Georgios]

Hello @calvinwy,

Sorry for the inconvenience, the issue you are facing could be Insufficient Lake Formation Permissions. Could you try deleting this database aws glue delete-database --name de_c4w4a1_silver_db. If the issue continues you could resend the form as @Mubsi suggested and choose Insufficient Lake Formation from the options. Hope it helps

[calvinwy]

Thanks @Georgios, everything works and it is done.