Week 2 - Assignment 2 - Error in script

Course Q&A Data Engineering Data Storage and Queries
1

Hello,
I am working in Week 2 Assignment Data Lake formation in AWS Apache Iceberg. When I try Part 4, in particular Deploying the script de-c3w2a1-ratings-to-iceberg-job it fails. When I go to AWS console, I get the error “getObject: NoSuchKey: The specified key does not exist”.
Thanks in advance

2

Hello @hugoaita
Apparently, one of the keys specified in the code for this glue job is not present in the data. Based on lines, 88 to 111 of the python script file, de_c3w2a1_ratings_to_iceberg.py, this script reads data from the the /landing_zone/json/ratings/ in your S3 bucket.

image
image785×592 25.3 KB

Please, inspect the content of this path. This is done in step 3.2.4. Also, please make sure that the jobs in step 3.2.3 succeed before moving on to running the jobs in part 4.

3

Hello Amir,
Now when I run 'terraform plan" again, it asks me for var.curated_db_ml_table, var.curated_db_ratings_table, var.presentation_db_table_sales, var.project…
After that, it throws errors:

Error: no value for required value on variables.tf line…

Thanks in advance

4

Terraform variables are defined upon running source scripts/setup.sh, as instructed in step 1.2.4.
image
Please, make sure you run this script in the same terminal instance that you run your terraform commands. If the terminal crashes and you open a new one, you need to rerun this script.