C4W4 -Capstone Project Part 1: Permission error

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
1

While running the transformation Glue jobs, I get an error:

 "AnalysisException: Insufficient Lake Formation permission(s) on de_c4w4a1_silver_db (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 6ac1a4f1-1c80-43b5-842c-8dec22d8196e; Proxy: null)

And the error is true - no permission is granted.

In C3W2 we did the same thing, but granted the permissions like this:

  1. Add policy statement to the Glue job role in policies.tf
  statement {
    sid    = "AllowLakeFormationAccess"
    effect = "Allow"
    actions = [
      "lakeformation:GetDataAccess"
    ]
    resources = [
      "*",
    ]
  }
  1. Allow access with lake formation
import boto3

bucket_name = f"de-c4w4a1-{account_id}-us-east-1-data-lake"
catalog_database = "de_c4w4a1_silver_db"
data_location_arn = f"arn:aws:s3:::{bucket_name}"
glue_role_arn = f"arn:aws:iam::{account_id}:role/Cloud9-de-c4w4a1-glue-role"

lf_client = boto3.client("lakeformation", region_name="us-east-1")
lf_client.grant_permissions(
    Principal={"DataLakePrincipalIdentifier": glue_role_arn},
    Resource={
        "DataLocation": {"ResourceArn": data_location_arn},
    },
    Permissions=["DATA_LOCATION_ACCESS"],
    PermissionsWithGrantOption=["DATA_LOCATION_ACCESS"],
)
lf_client.grant_permissions(
    Principal={"DataLakePrincipalIdentifier": glue_role_arn},
    Resource={"Database": {"Name": catalog_database}},
    Permissions=permissions,
)

Unfortunately I don’t have permissions to grant the access with lake formation.

I have to finish the course today, probably without this lab.

But I hope the lab will be fixed for the other participants.

2

Hello @YAtOff
I have already completed the lab, and the necessary permissions were given to the glue jobs. In your case, there might be a problem with the s3.tf or glue.tf files in the modules/transform_job/ that the jobs are failing to connect the database. Please, check these files and try again.