Course 2 Week 4 Practice Lab 2: Airlfow 101 - Best Practices

Hi all,

While running a practice lab 2 in Course 2, Week 4, after filling in all the gaps in the simple_dag.py I am receiving the error:

aa24f292a5e5
*** Found local files:
***   * /opt/airflow/logs/dag_id=simple_dag/run_id=manual__2024-11-19T14:26:43.333720+00:00/task_id=extract_and_load_orders/attempt=1.log
[2024-11-19, 14:26:50 UTC] {local_task_job_runner.py:123} ▶ Pre task execution logs
[2024-11-19, 14:26:50 UTC] {base.py:84} INFO - Retrieving connection 'mysql_connection'
[2024-11-19, 14:26:51 UTC] {base.py:84} INFO - Retrieving connection 'mysql_connection'
[2024-11-19, 14:26:51 UTC] {taskinstance.py:3310} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 767, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable
    return ExecutionCallableRunner(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 406, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/transfers/sql_to_s3.py", line 189, in execute
    data_df = sql_hook.get_pandas_df(sql=self.query, parameters=self.parameters)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/common/sql/hooks/sql.py", line 282, in get_pandas_df
    with closing(self.get_conn()) as conn:
                 ^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/mysql/hooks/mysql.py", line 196, in get_conn
    return MySQLdb.connect(**conn_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/MySQLdb/__init__.py", line 121, in Connect
    return Connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/MySQLdb/connections.py", line 195, in __init__
    super().__init__(*args, **kwargs2)
MySQLdb.OperationalError: (1045, "Access denied for user 'admin'@'10.2.0.32' (using password: NO)")
[2024-11-19, 14:26:51 UTC] {taskinstance.py:1225} INFO - Marking task as FAILED. dag_id=simple_dag, task_id=extract_and_load_orders, run_id=manual__2024-11-19T14:26:43.333720+00:00, execution_date=20241119T142643, start_date=20241119T142650, end_date=20241119T142651
[2024-11-19, 14:26:51 UTC] {taskinstance.py:340} ▼ Post task execution logs
[2024-11-19, 14:26:51 UTC] {standard_task_runner.py:124} ERROR - Failed to execute job 3 for task extract_and_load_orders ((1045, "Access denied for user 'admin'@'10.2.0.32' (using password: NO)"); 1797)
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/task/task_runner/standard_task_runner.py", line 117, in _start_by_fork
    ret = args.func(args, dag=self.dag)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/cli_config.py", line 49, in command
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/cli.py", line 115, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 483, in task_run
    task_return_code = _run_task_by_selected_method(args, _dag, ti)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 256, in _run_task_by_selected_method
    return _run_raw_task(args, ti)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/cli/commands/task_command.py", line 341, in _run_raw_task
    return ti._run_raw_task(
           ^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/session.py", line 97, in wrapper
    return func(*args, session=session, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 3004, in _run_raw_task
    return _run_raw_task(
           ^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 273, in _run_raw_task
    TaskInstance._execute_task_with_callbacks(
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 3158, in _execute_task_with_callbacks
    result = self._execute_task(context, task_orig)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 3182, in _execute_task
    return _execute_task(self, context, task_orig)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 767, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable
    return ExecutionCallableRunner(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/models/baseoperator.py", line 406, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/transfers/sql_to_s3.py", line 189, in execute
    data_df = sql_hook.get_pandas_df(sql=self.query, parameters=self.parameters)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/common/sql/hooks/sql.py", line 282, in get_pandas_df
    with closing(self.get_conn()) as conn:
                 ^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/mysql/hooks/mysql.py", line 196, in get_conn
    return MySQLdb.connect(**conn_config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/MySQLdb/__init__.py", line 121, in Connect
    return Connection(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.11/site-packages/MySQLdb/connections.py", line 195, in __init__
    super().__init__(*args, **kwargs2)
MySQLdb.OperationalError: (1045, "Access denied for user 'admin'@'10.2.0.32' (using password: NO)")
[2024-11-19, 14:26:51 UTC] {local_task_job_runner.py:266} INFO - Task exited with return code 1
[2024-11-19, 14:26:51 UTC] {taskinstance.py:3900} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2024-11-19, 14:26:51 UTC] {local_task_job_runner.py:245}

from the module extract_and_load_orders. Do you have any suggestions on how to interpret the error and any suggestions for fixing it?

Best regards,
Anton.

Hi guys,

Sorry, I forgot to fill in the password in the connections of the Airflow. Filling it solves the issue - allows to establish a connection to the MySQL DB.

Best,
Anton.