Error in Capstone Lab 1

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
1

Working on Capstone Lab 1.
In section 4.2.7 the job de-c4w4a1-json-transform-job is failing after 2 minutes or so with the following error:
An error occurred while calling o168.createOrReplace. Cannot find catalog plugin class for catalog ‘glue_catalog’: org.apache.iceberg.spark.SparkCatalog

Based on my research it looks like something isn’t being installed properly. Is there a step missing in the instructions? I have gone over everything and the code in /config/workspace/terraform/assets/transform_jobs/de-c4w4a1-transform-json-job.py looks correct. I need guidence as I am dead in the water.
Thank you.

2

Hello @Leland_Grant
Unfortunately, I was not able to reproduce the problem. Did you make sure to run terraform init in step 4.2.6, before applying the changes made for the transform module? Had the glue job runs from step 4.1.7 succeeded before you run the transform jobs?

3

@Amir_Zare I did every step and the previous Glue jobs ran successfully. I will try it again this morning and let you know how it goes.
Thanks,
Lee

4

@Amir_Zare I complete the lab. Not sure what the issue was, I didn’t do anything different. Thank you for trying to recreate the issue.

Lee

1 Like