C4W4- Lab CAPSTONE Project- 2-Terraform-Data Quality-Error

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
1
  • C4W4 # (Capstone Project -2)
  • Facing below error when executing terraform apply -target=module.data_quality based on the set 3.1.1 in the notebook.

module.data_quality.aws_glue_data_quality_ruleset.users_dq_ruleset: Creating…
module.data_quality.aws_glue_data_quality_ruleset.sessions_dq_ruleset: Creating…
module.data_quality.aws_glue_data_quality_ruleset.songs_dq_ruleset: Creating…
╷
│ Warning: Applied changes may be incomplete
│
│ The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully updated. Run the following command to verify that no other changes are pending:
│ terraform plan
│
│ Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part of an error message.
╵
╷
│ Error: creating Glue Data Quality Ruleset (songs_dq_ruleset): operation error Glue: CreateDataQualityRuleset, https response error StatusCode: 400, RequestID: b7d69be3-28ea-4ad2-b58d-fe07e17b2ef6, InvalidInputException: Entity Not Found (Service: AmazonDataCatalog; Status Code: 400; Error Code: EntityNotFoundException; Request ID: d2c92443-a372-4577-8dea-e6dd8855c9b2; Proxy: null)
│
│ with module.data_quality.aws_glue_data_quality_ruleset.songs_dq_ruleset,
│ on modules/data_quality/glue.tf line 1, in resource “aws_glue_data_quality_ruleset” “songs_dq_ruleset”:
│ 1: resource “aws_glue_data_quality_ruleset” “songs_dq_ruleset” {
│
╵
╷
│ Error: creating Glue Data Quality Ruleset (sessions_dq_ruleset): operation error Glue: CreateDataQualityRuleset, https response error StatusCode: 400, RequestID: 80930cf8-4900-48cc-afd9-df6f95a57307, InvalidInputException: Entity Not Found (Service: AmazonDataCatalog; Status Code: 400; Error Code: EntityNotFoundException; Request ID: 5acc79c0-f39f-4150-b136-8a3ab8cc83d4; Proxy: null)
│
│ with module.data_quality.aws_glue_data_quality_ruleset.sessions_dq_ruleset,
│ on modules/data_quality/glue.tf line 10, in resource “aws_glue_data_quality_ruleset” “sessions_dq_ruleset”:
│ 10: resource “aws_glue_data_quality_ruleset” “sessions_dq_ruleset” {
│
╵
╷
│ Error: creating Glue Data Quality Ruleset (users_dq_ruleset): operation error Glue: CreateDataQualityRuleset, https response error StatusCode: 400, RequestID: 00721c9e-eebf-46ff-afc6-400ac581b9dd, InvalidInputException: Entity Not Found (Service: AmazonDataCatalog; Status Code: 400; Error Code: EntityNotFoundException; Request ID: 4b6829c1-f647-4e72-a18e-312f329cae9c; Proxy: null)
│
│ with module.data_quality.aws_glue_data_quality_ruleset.users_dq_ruleset,
│ on modules/data_quality/glue.tf line 19, in resource “aws_glue_data_quality_ruleset” “users_dq_ruleset”:
│ 19: resource “aws_glue_data_quality_ruleset” “users_dq_ruleset” {

Please help.

Thank you

2

Heĺlo @knvsiddarth,

There was a learner with the same issue, post. Could you check those tables are created from the previous steps and all the extract/transform jobs are successful and run with the correct order. Thank you