Course 2 Week 4 Graded Programming Assignment 4: Building an Advanced Data Pipeline With Data Quality Checks no DAGs appear in the Airflow

Course Q&A Data Engineering Source Systems, Data Ingestion, and Pipelines
1

Hi guys,

After accomplishing all Excercises 1 - 4, and uploading generated DAGs to the corresponding S3 bucket I can not see them in Airflow to launch them.
What can be an error? It passed more than 10 minutes since uploading to S3.

Best regards,
Anton.

2

Hello @CalmScout,
Its wierd you are not getting any import errors after you sync the three files. Could you check you are uploading to the correct file location/dags?
aws s3 sync src/dags s3://de-c2w4a1-<ACCOUNT_ID>-us-east-1-mwaa-dags/dags
If you are getting any erros afterwards can you post a screenshot. Thanks

3

Hi @Georgios,

No error after running the following command:
image

What can be wrong? It’s the second time I reach this place in the lab and Airflow doesn’t see DAGs.

4

The grader is complaining about my Python files:

image
image1477×511 65.6 KB

How to check what’s wrong with them?

5

@CalmScout, I could try to reproduce your issue. Did you get three outputs the first time you run the command? I can’t see any files uploaded in the screenshot, can you confirm this?Thanks

6

Hi @Georgios,
Thank you for the phenomenal answering speed. The problem was that I forgot to replace one of “None” to “read_parquet” in the template.py and that error propagated to the created .py files. After fixing that issue everything is working!
Thank you!

7

Glad it fixed @CalmScout