C4W4 Capston Part 1: AnalysisException Error Glue job run on api users extract

Course Q&A Data Engineering Data Modeling, Transformation, and Serving
, ,
1

I would like to ask regarding the error that I got when starting glue job in section 4.1.7 for part 1 capstone project. I keep getting an AnalysisError on the AWS Glue Console for the api-users-extract job only while the other two extrac jobs always successful. I did rerun the job multiple times and still the same result.

I also already submitted a request to refresh the lab too but it is still the same as before.

image
image1350×296 68.2 KB

2

Hello @Youhorng,

It seems I could reproduce the error. Could you check you are using the correct api_end_date. Thank you:

    # Set `"--api_start_date"` to `"2020-01-01"`
    "--api_start_date"      = "2020-01-01"
    # Set `"--api_end_date"` to `"2020-01-31"`
    "--api_end_date"        = "2020-01-31" <----USE 2020-01-31 instead of 2020 -01-01
    # Replace the placeholder <API-ENDPOINT> with the value from the CloudFormation outputs
    "--api_url"             = "http://ec2-***.compute-1.amazonaws.com/users"

This is the error if you are using end date 2020-01-01 instead. Hope it helps:

C4W4A1_analysis_exception_wrong_date
C4W4A1_analysis_exception_wrong_date1584×769 202 KB

3

I getting this error for

User: arn:aws:sts::058264395471:assumed-role/voclabs/user3923357=crhcxbsvcuog is not authorized to perform: logs:DescribeLogStreams on resource: arn:aws:logs:us-east-1:058264395471:log-group:/aws-glue/jobs/error:log-stream: because no identity-based policy allows the logs:DescribeLogStreams action

4

Hello @HeyChong,

When you run the glue job, you have to use the actual name of the glue job not the name of the output variable.

So one of the outputs you got:
glue_api_users_extract_job = “de-c4w4a1-api-users-extract-job”

When you run the glue job, the command should be:
aws glue start-job-run --job-name de-c4w4a1-api-users-extract-job | jq -r ‘.JobRunId’

Hope it helps

5

Thanks! I run it directly from AWS console still encounter this error.

6

Hello @HeyChong,

That is strange, you get this error when you run the command or when the job fails in the AWS glue console. Please fill this form for a lab refresh. Thank you