C2W4 assignment aws s3 sync errpr

Course Q&A Data Engineering Source Systems, Data Ingestion, and Pipelines
,
1

Nothing appears on the Airflow UI after I run the following in cmd: aws s3 sync src/dags s3://de-c2w4a1-077659555592-us-east-1-raw-data/dags
no python file in the bucket

image
image652×466 10.7 KB

2

Hello @Tan_Quoc_Bao_Nguyen,

I think you try to upload on the raw data bucket, could you run with the dags bucket instead, thank you:

aws s3 sync src/dags s3://de-c2w4a1-077659555592-us-east-1-dags/dags

3

Should I change the variable on the airflow too? since it’s raw bucket too

image
image1273×547 11.6 KB

4

Hello @Tan_Quoc_Bao_Nguyen,

The variable bucket_name in the airflow should be raw data bucket instead of dags bucket, you can change that. thank you

Use: de-c2w4a1-077659555592-us-east-1-raw-data

C2W4_bucket_name_var
C2W4_bucket_name_var914×399 31.3 KB

5

I uploaded to the other bucket, but there still isn’t anything appearing on the airflow?

image
image1192×100 15.9 KB

6

Hello @Tan_Quoc_Bao_Nguyen,

You used /dag/ instead of /dags/
aws s3 sync src/dags s3://de-c2w4a1-077659555592-us-east-1-dags/dag
Correct is with s in the end:
aws s3 sync src/dags s3://de-c2w4a1-077659555592-us-east-1-dags/dags

Thank you

1 Like
7

Thanks a lot, no more error!

1 Like