March 2025 - C1W4 Programming Assignment: Assignment 3: Building End-to-End Batch and Streaming Data Pipelines Based on Stakeholder Requirements I am getting an error on the last terraform init of C1W4 showing error message terminal process “/usr/bin/bash” terminated with exit code: 1.
Hello @amar97
The issue with terraform crashing has to do with existing resources. You could run terraform apply -no-color 2> errors.txt instead of terraform apply . That will create a errors.txt file that will show info about the terminal crash.:
Then you could go to AWS console and find and delete those resources:
- de-c1w4-ml-db in AWS glue > databases
- de-c1w4-rds-connection in AWS glue > connections
- de-c1w4-glue-rol in IAM > roles
You could run terraform apply -no-color 2> errors.txt again and if there is a crawler you need to delete it as well. You could find it in AWS Glue > crawlers:
If the issue continues you could wait a few hours for a new lab session or you could submit this form for a lab refresh. Note it takes 1-2 business days since it is a manual process performed by the engineers Hope it helps
2 Likes
Now it crashed after terraform apply, when one needs to confirm and write yes.
What I got on the error.txt file is:
I tried to follow the steps on the console, but didn’t find any database where you told me to look for:
Nothing in the connections either:
I only found the role, but when I tried to delete it, it showed a message saying I don’t have the access to do so.
No crawlers either. I’m submitting the form.
OBS: I do see the database available on RDS
Hello @jmacedo,
First of all thanks for submitting the form, the lab refresh should fix the missing de-c1w2-analytics-db shown as existing. You only need to check the AWS console in ETL jobs to see the status of de-c1w2-etl-job (hopefully when terraform apply deploys the job after the lab refresh).
The connection and crawler will be useful later in the course in: Assignment 3. There seems to be an issue when the lab reboots when you terrraform apply and those resources are created and shown as existing. You need to delete them manually (de-c1w4-ml-db, de-c1w4-rds-connection and de-c1w4-glue-role) to create the vector-db.
You need to wait 1-2 business days for the lab refresh and check for those resources. Also note after afew hours that a new lab session might clear them without checking the console. Hope it helps
Hello Georgios,
I am facing a similar problem as jmacedo.
In my error.txt file is the following message:
Error: creating Glue Catalog Database (de-c1w2-analytics-db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: 1c316ea4-57cc-4c1b-87b1-245ec192a54f, AlreadyExistsException: Database already exists.
with aws_glue_catalog_database.analytics_database,
on glue.tf line 1, in resource "aws_glue_catalog_database" "analytics_database":
1: resource "aws_glue_catalog_database" "analytics_database" {
Error: creating IAM Role (de-c1w2-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: 1d20883f-f05e-4042-94f5-775a543bd3d9, EntityAlreadyExists: Role with name de-c1w2-glue-role already exists.
with aws_iam_role.glue_role,
on iam_roles.tf line 1, in resource "aws_iam_role" "glue_role":
1: resource "aws_iam_role" "glue_role" {
But I just checked the console and there isn’t any database created. Before running the terraform commands I’ve deleted the connection, but besides the connection, there was nothing in the console. No database, no catalog nothing.
I also have no permissions to delete my role.
How should I proceed? I’ve been trying for a week to complete this first lab but I’ve always had some kind of error.
Hello @jaqs,
Please submit this form since a lab refresh gets it fixed. Note it takes 1-2 business days to complete(a bit longer because it is weekend). Sorry for the inconvenience but how come you waited a week in the first lab, did you have other permissions issues when you first run the glue job. Hope it helps
Hi Georgios, the first time I ran the lab I had a problem while running the glue job. It never finished successfully. I don’t know why the error happened though.
Hello jacs,
Yes, that happens because of Insufficient Lake Formation Permissions, it’s something we are working on. For now the solution is to fill the lab refresh form. However it takes 1-2 business days and it is weekend. It is a manual process performed by the engineers. Also you will be notified when it resolves so you can try again and succeed with the glue job and complete the lab. Hope it helps