Can be the lab be restarted ?
The AWS Glue job exchanging the failed to run. I want to start all over again
Hello @bravus,
If your job failing with Insufficient Lake Formation Permissions you could ask for a lab refresh with this form, it takes 1-2 business days to process.
Also the lab does reset after a new lab session when 2 hours pass and clears previously created resources.
You can safely delete any files you edited and use get latest version from help to recreate them as before. Then use reboot and your lab should be fresh. Thank you
Hello Georgios,
I have tried this lab few times, but I am not getting some folders. Also when I do the terraform configuration. the terminal always crash after terraform apply. It even crash using the terraform apply -no-color 2> error.txt. Is there a way I can restart the whole lab all over again ? from start thanks
Hello @bravus,
When you run terraform apply -no-color 2> error.txt the only difference is that the terminal still crashes but it should create an error.txt file. In the error.txt file there should be additional information like a log that says which resources are causing terraform to fail.
I donāt think there is another option other than wait a few hours for the lab to reset those resources unless you can investigate with the help of the error.txt and delete them in the AWS console (manually one by one).
If the lab refresh form doesnāt help you reset your lab resources you could contact Coursera support to help you further. Hope it helps
Hello @Georgios,
I started to have problems yesterday, with permissions. The Glue job run failedā¦ I tried to restart everything but could not access the community site (my company blocked it) and never made it.
Today Iām using another computer and could finally read this. I gladly saw your reset instructions, though something is not right. The terraform plan command returns:
ā ~/project/terraform$ terraform plan
ā
ā No changes. Your infrastructure matches the configuration.
ā
ā Terraform has compared your real infrastructure against your configuration and found no differences, so no changes
are needed.
The de-c1w4-etl-job isnāt created, so my lab is stalled.
How can I effective and completely reset the lab?
Thanks!
Hello @nmsleal,
Effectively in this lab you change main.tf and outputs.tf files progressively you uncomment the parts of the lab to create the glue job. If the glue job run fails is a sign that you have Insufficient Lake Formation Permissions and you need to ask for a lab refresh.
The issue with main.tf and outputs.tf is that Coursera saves your progress and the parts youāve uncomment remains the same. If the files has all the blocks commented terraform plan should give no changes output. However you can safely delete those files and go to lab help and use get latest version. That will make the files reappear and start uncommenting from the beginning.
Hope it helps
Yes, it helped me realise I had not uncommented the newly generated scriptsā¦ Thanks!
Iāll have to wait for this lab expiration, as Iām stuck at the terraform apply command, that disconnects my session when I run itā¦
1 Like
thanks, Georgios
I want to say that the terminal crashes in the section Adding embeddings to the vector database. When I entered this command psql --host= --username=postgres --password --port=5432 with their corresponding passowords and user name the terminal crash every time after I enter password.
Also, in AWS in the RDS on Databases I donāt have the data base de-c1w4-vector-db
Hello @bravus,
You need to add the endpoint in the psql command as host. You should get the password from the command output after you created the vector database, it should be 8 characters long. Also add the correct bucket name in the embeddings. Thank you
Hello @nmsleal,
You could try to uncomment only the extract blocks and try terraform init from the start but use terraform apply -no-color 2> error.txt instead. In the errors.txt file you will find existing resources.
You need to delete those resources if you go to the AWS console. This process should save some time rather than waiting a few hours to refresh your lab. It might take some effort but hopefully it will work. Thanks
Hi,
I ended up waiting for the next day, then I deleted terraformās lock file and the two .tf files I had uncommented, got a āLatest versionā, and restarted the lab. All went fine till the apply thing, where errors.txt got this content:
Error: creating Glue Catalog Database (de-c1w4-ml-db): operation error Glue: CreateDatabase, https response error StatusCode: 400, RequestID: fa2a61ba-b642-4eb3-ba84-05d3eaba272a, AlreadyExistsException: Database already exists.
with module.etl.aws_glue_catalog_database.ml_database,
on modules/etl/glue.tf line 1, in resource āaws_glue_catalog_databaseā āml_databaseā:
1: resource āaws_glue_catalog_databaseā āml_databaseā {
I could not find any de-c1w4-ml-db database to delete, so Iām stuck againā¦
Thanks!
Georgios Nikolitsis via DeepLearning.AI <notifications@dlai.discoursemail.com> escreveu (quarta, 12/03/2025 Ć (s) 23:14):
Hello @nmsleal,
Sorry for the inconvenience, you should be able to find de-c1w4-ml-db in AWS glue if you go to databases:
If it still doesnāt appear and after a few hours in a new lab session it continues you need to fill the lab refresh form. It should take 1-2 business days to process, thank you
I did that, to no availā¦
Georgios Nikolitsis via DeepLearning.AI <notifications@dlai.discoursemail.com> escreveu (quinta, 13/03/2025 Ć (s) 14:01):
My Lab ID is tzotpnjqyqll
Nuno Leal <nmsleal@gmail.com> escreveu (quinta, 13/03/2025 Ć (s) 14:04):
Hello @nmsleal
Yes you should fill that in the Combined Data Engineering Lab Issue Report from the link:
you need an email/name
choose your interface
choose an option if you canāt find the provided AWS resources.
Course 1 - Introduction to Data Engineering/ ASSIGNMENT 3
and fill the lab ID. It should take 1-2 business days
I sent the form one hour ago. I hope it is fulfilled on time for the weekendā¦ Thanks!
1 Like
Hello @Georgios,
I received the confirmation email yesterday late afternoon, so I started the lab again, but didnāt generate all the files again. Everything worked as expected and I made it to the submission, though my grading is only 50% because some ācredencialsā file is missingā¦
This whole lab is so frustratingā¦ Iāll restart and do everything again, looking for that damned file, but will search the topic here first, as this seems to be some known issue.
Thanks for being so supportive!
Hello nuno,
Sorry for all those issues, I went through all the Assignment 3: Building End-to-End Batch and Streaming Data Pipelines Based on Stakeholder Requirements. First thing to do is to use get latest version without deleting anything-just re-comment the Vector_db and Streaming-Inference modules from the main.tf and outputs.tf files.
If you have tried running the de-c1w4-etl-job successfully it seems you have no permissions issues. Afterwards you should be able to uncomment the Vector_db part and create the vector database and use terraform output vector_db_master_password to get the password for later.
Here one thing I noticed that after I submit the grader I passed with 100 (instead of 50 for just creating the database and run the glue job), not sure why but you could try and submit after that point and see if you get credited.
I am not sure about the credentials you mentioned but if you did everything correct inside lambda de-c1w4-model-inference you should be able to see a new folder inside the recommendations_bucket, it should start with 2025. Hope it helps
