C3W2 Assignment, what value should I type in?

Course Q&A Data Engineering Data Storage and Queries
1

Hi all,

I am trying to complete the 4.3.3 section for the assignment 2.

4.3.3. Deploy the Glue Jobs following the same steps as in section 3.2. You will need to execute three Glue Jobs: glue_csv_transform_job, glue_ratings_transform_job, glue_ratings_to_iceberg_job (job names de-c3w2a1-csv-transformation-job, de-c3w2a1-ratings-transformation-job and de-c3w2a1-ratings-to-iceberg-job). Each of them should take around 3 minutes, you can run them at the same time.

Initially, I received this error.

An error occurred (EntityNotFoundException) when calling the StartJobRun operation: Failed to start job run due to missing metadata.

Therefore, I tried to rerun the terraform plan and got asked to enter a value with the following prompt in the terminal below?

coder@eb8a87529434:~/project/terraform$ ^C
coder@eb8a87529434:~/project/terraform$ terraform plan
var.curated_db_ml_table
Table for ML at curated DB

Enter a value:

I have also tried to source scripts/setup.sh to reset the environment from this post but the terraform apply crashed after typing yes

How do I troubleshoot or enter the following value? Thanks! My lab code is vgofkhqxoqxn

2

Hello @Loo_Guan_Yee,

You could see the errors.txt file after running terraform apply -no-color 2> errors.txt to see if there are any existing resources that you need to delete (e.g. de-c3w2a1-rds-connection and find it, AWS Glue>Connections).
Then you need to run source scripts/setup.sh each time the terminal crash for terraform plan and try again with terraform apply.

You could try in a new lab session that might clear those resources and fill this form for a lab refresh if it continues. It should take 2 business days so you will need to wait a bit longer since it is weekend. Thank you

3

I having the same issue. The screen is terminating after I enter Yes after terraform apply

4

Hello @HeyChong,

Could you run this command terraform apply -no-color 2> errors.txt instead of terraform apply. You should be able to see a errors.txt file and some existing resources that cause the terminal to terminate.

Run source scrtips/setup.sh and go to terraform folder run the commands like normally and check the errors.txt file for those resources and try to find them e.g. delete de-c3w2a1-rds-connection from AWS glue console > Data Catalog> connections.

If the issue continues you could ask for a lab refresh form.

5

I have the same issue. Is this problem solved? If so what are the steps that where made. I have already tried refreshing the lab resources.

6

Hello @Rzmn,

If your terminal crashed you need to run source scripts/setup.sh again to set those variables and afterwards run terraform apply -no-color 2> errors.txt instead. Could you check at the errors.txt file for any existing resources and delete them. Thank you

7

Hello @Georgios,
I have run the setup script again and have set up the variables and followed the process again but Iā€™m getting the following result in the terminal while running terraform again:

image
image630Ɨ377 14.4 KB

Thanks for any help and also sorry for the late response.

8

Hello @Rzmn,

Yes, at this point you need to ctrl+c to stop the command and use cd .. to go back to project folder in the cli. If source scripts/setup.sh runs successfully you can go to terraform folder then use the terraform commands.

Use terraform init like in your screenshot then terraform plan, and finally terraform apply -no-color 2> errors.txt in case the terminal crash again. You need to run source scripts/setup.sh each time you restart the terminal, hope it helps

9

Hi @Georgios,

Managed to reopen the Lab assignment to redo the processes in the Terraform. Thanks!

1 Like