Step 2.5 Capstone Project Part 2 Insufficient Lake Formation permission

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
1

AnalysisException: Insufficient Lake Formation permission(s) on de_c4w4a2_transform_db (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 5e6a46fa-e8ce-4bb0-9905-d3d8a5f8768a; Proxy: null) was reported for the two glue jobs ( glue_json_transformation_job and glue_songs_transformation_job) in step 2.5 Capstone Project Part 2. Could you please let me know how to fix it ?

2

Hello @MiloAndelic
Unfortunately, I was not able to reproduce the issue. Can you please provide a screenshot of where you get the issue? If possible, please, include as much detail as possible from the available logs.

3

Thanks @Amir_Zare . I have just tried again and it passed this time. However, I faced the problem when running 4.2 - DAG for Songs Data in RDS Source as it failed. Please find the related details below:

[2024-10-20, 14:04:39 UTC] {taskinstance.py:3310} ERROR - Task failed with exception
Traceback (most recent call last):
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py”, line 767, in _execute_task
result = _execute_callable(context=context, **execute_callable_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py”, line 733, in _execute_callable
return ExecutionCallableRunner(
^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/operator_helpers.py”, line 252, in run
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/models/baseoperator.py”, line 406, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/operators/glue.py”, line 223, in execute
glue_job_run = self.glue_job_hook.job_completion(self.job_name, self._job_run_id, self.verbose)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/hooks/glue.py”, line 297, in job_completion
ret = self._handle_state(job_run_state, job_name, run_id, verbose, next_log_tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/hooks/glue.py”, line 346, in _handle_state
raise AirflowException(job_error_message)
airflow.exceptions.AirflowException: Exiting Job jr_df9fbaec9871e8282bba5f8baf23277af20b42635f7d3cdfbeb8f8d23a622d6e Run State: FAILED

1 Like