I hope you’re well. I’m writing to report persistent issues with the hands-on lab involving AWS Glue (programming assignment Week 2 of Course 3). Every time I run the lab, I encounter problems that prevent me from progressing.
Specifically:
The Glue jobs remain stuck in “Running” for an unusually long time, and sometimes even time out.
When trying to check the job status or logs, I receive permission errors such as: “This IAM user does not have permission to view Log Groups in this account…”
I’ve attempted to rerun the lab several times, but the problems persist. It seems that the environment is not resetting cleanly between runs.
Because of this, I’m unable to complete the exercise or move forward in the course. I kindly request that my lab environment — including the AWS resources — be fully reset or reviewed so I can proceed with a clean state.
@Diana_Bohorquez Can you explain at what step you’re getting this error? It looks like something is not configured well in the script of the glue jobs.
A couple of clarification items:
Every time you finish a session, the AWS account automatically goes through a cleaning process. So when you start a new session, the AWS resources are automatically fully reset for you.
If you want the lab files to be reset, you would need to do that manually. I can explain what to do in this case if you also want your lab files to be reset.
I’m trying to run step 3.2.3 of the lab, where we’re instructed to start the Glue jobs using the aws glue start-job-run command. However, every time I attempt this, both jobs get stuck and eventually return a TIMEOUT status.
I’ve tried this multiple times — restarting the lab environment and following all the instructions — but the result is always the same.
Additionally, I read on another forum that it’s possible to recover the original lab scripts by deleting the current ones and clicking “Get Latest Version” from the Lab Help tab. I tried that too, but unfortunately it didn’t resolve the issue.
Could you please advise how I can proceed? Is there a problem with the scripts or something else in the setup?
@Diana_Bohorquez Did you make sure to run “source scripts/setup.sh”? Were you asked to prompt the variables for terraform and did you manually assign variable like my-glue-scripts-bucket?
Yes, I always run source scripts/setup.sh at the beginning.
Regarding your second question, I’m not sure if I was prompted for the Terraform variables, so I might have assigned some values manually without realizing it.
How could I proceed in order to deal with this issue?
@Diana_Bohorquez If the problem was only related to running the glue job,
to get the glue job id, you can either type terraform output or check the glue job in the AWS console
you don’t need to re-do terraform apply because resources have been already created, you just need to make sure you have the right job id
please avoid rebooting the lab during the lab session, this will make your lab environment out of sync with the AWS resources that have been created, hence the issue “resources already existed”
you can append || echo "$?" to any command to prevent the terminal from crashing.
you can also run the glue job manually in the AWS console.
I run source scripts/setup.sh → cd terraform , terraform init, terraform plan.
After that, I run aws glue start-job-run --job-name <GLUE-JOB-NAME> | jq -r '.JobRunId' , and this time I could get every ID for every glue job.
When I tried to run the jobs (aws glue get-job-run --job-name <GLUE-JOB-NAME> --run-id <JOB-RUN-ID> --output text --query "JobRun.JobRunState), this time I get FAILED message.
When I went to AWS Console, I goy a message that I don’t know how to debug.
I am having issues with this assignment as well. I get to section 3.2 where you use terraform to get the Glue jobs going and it crashes on me during ‘terraform apply’. I get this error message:
│ Error: Reference to undeclared module
│
│ on outputs.tf line 24, in output “glue_ratings_transform_job”:
│ 24: value = module.transform_etl.glue_ratings_transform_job
│
│ No module call named “transform_etl” is declared in the root module.
But I’m not sure what the root module is?
I tried running the command terraform apply -no-color 2> errors.txt as suggested in the instructions and it does create a text file with an error message but the terminal still crashes. I do not see any Glue jobs in the AWS console. Here is the error message in the .txt file generated:
Error: No configuration files
Apply requires configuration to be present. Applying without a configuration
would mark everything for destruction, which is normally not what is desired.
If you would like to destroy everything, run ‘terraform destroy’ instead.
Any help would be appreciated. I know there is a way to reset labs/assignments but I cannot find those instructions. Thanks!
