Issue in Module 4 – Task 3: Batch and Streaming Pipelines

Course Q&A Data Engineering Introduction to Data Engineering
1

During Module 4 – Programming Assignment 3: “Building End-to-End Batch and Streaming Data Pipelines based on Stakeholder Requirements”, I was unable to complete the batch pipeline step that requires exploring the ratings table in the RDS MySQL database.

When I ran the provided AWS CLI command to retrieve the RDS endpoint:

aws rds describe-db-instances \
  --db-instance-identifier de-c1w4-rds \
  --output text \
  --query "DBInstances[].Endpoint.Address"

I received the error:

Invalid endpoint: https://rds..amazonaws.com

Details:

  • The error shows that the CLI is missing a configured region, causing the endpoint URL to be incomplete.

  • Adding --region us-east-1 did not resolve the issue.

  • Attempting aws configure set region us-east-1 failed with “Permission denied”, because the lab environment restricts modifications to ~/.aws/config.

Impact:
This prevents me from retrieving the RDS endpoint and blocks further steps in building the batch pipeline for this activity.