Hey @hawraa.salami I did run the lab again, the first terraform initialization & the glue jobs ran well and the embeddings were created and the vector db’s were created. I am now getting an error when trying to implement the streaming pipeline. I did generate the errors.txt file after running terraform apply here’s the error message.
Error: creating Glue Catalog Database (de-c1w4-ml-db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: 329f7fd8-30a8-47f5-9139-e1865d1b680c, AlreadyExistsException: Database already exists.
with module.etl.aws_glue_catalog_database.ml_database,
on modules/etl/glue.tf line 1, in resource “aws_glue_catalog_database” “ml_database”:
1: resource “aws_glue_catalog_database” “ml_database” {
Error: creating Glue Connection (de-c1w4-rds-connection): operation error Glue: CreateConnection, https response error StatusCode: 400, RequestID: 46cf95a7-8db3-4a9c-9116-321c3eb27172, AlreadyExistsException: Connection already exists.
with module.etl.aws_glue_connection.rds_connection,
on modules/etl/glue.tf line 6, in resource “aws_glue_connection” “rds_connection”:
6: resource “aws_glue_connection” “rds_connection” {
Error: creating IAM Role (de-c1w4-glue-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: 2a612e5a-57dd-401f-b8c1-7b98fbe8bc42, EntityAlreadyExists: Role with name de-c1w4-glue-role already exists.
with module.etl.aws_iam_role.glue_role,
on modules/etl/iam_roles.tf line 1, in resource “aws_iam_role” “glue_role”:
1: resource “aws_iam_role” “glue_role” {
UPDATE: I deleted the resources mentioned above through the AWS Console but was then met with different errors:
Error: creating IAM Role (de-c1w4-rds-role): operation error IAM: CreateRole, https response error StatusCode: 409, RequestID: 4098ae43-0c4f-4c9b-a7f0-bb4bde5aaa8e, EntityAlreadyExists: Role with name de-c1w4-rds-role already exists.
with module.vector_db.aws_iam_role.rds_role,
on modules/vector-db/iam_roles.tf line 1, in resource “aws_iam_role” “rds_role”:
1: resource “aws_iam_role” “rds_role” {
Error: creating RDS DB Subnet Group (de-c1w4-vector-db-subnet-group): operation error RDS: CreateDBSubnetGroup, https response error StatusCode: 400, RequestID: 5aff7b22-c523-40f4-886c-f4759cf1f0f2, DBSubnetGroupAlreadyExists: The DB subnet group ‘de-c1w4-vector-db-subnet-group’ already exists.
with module.vector_db.aws_db_subnet_group.vector_db_subnet_group,
on modules/vector-db/rds.tf line 1, in resource “aws_db_subnet_group” “vector_db_subnet_group”:
1: resource “aws_db_subnet_group” “vector_db_subnet_group” {
Error: creating Security Group (de-c1w4-vector-db-sg): operation error EC2: CreateSecurityGroup, https response error StatusCode: 400, RequestID: c1d3f429-a6f9-4f0e-b2ce-3b5ba1a05963, api error InvalidGroup.Duplicate: The security group ‘de-c1w4-vector-db-sg’ already exists for VPC ‘vpc-09df03d28804533c5’
with module.vector_db.aws_security_group.vector_db_sg,
on modules/vector-db/rds.tf line 10, in resource “aws_security_group” “vector_db_sg”:
10: resource “aws_security_group” “vector_db_sg” {
