While deploying the Glue job in assignemtn 2 (section 4.3.3) I am trying to run the job here: aws glue start-job-run --job-name “de-c3w2a1-csv-transformation-job” | jq -r ‘.JobRunId’ but I get the error: An error occurred (EntityNotFoundException) when calling the StartJobRun operation: Failed to start job run due to missing metadata.
When I check the add_metadata function in the file …terraform/assets/transform_etl_jobs/de_c3w2a1_batch_transform.py, I completed this section below:
Replace `None` with the appropriate `source` metadata string, which will be the “classic_models_mysql”.
source_pd\['source'\] = "classic_models_mysql"
Please is there anything I am missing? I can’t seem to figure why the job is faling. Any help will be appreciated