Error When Submitting W3 Assessment

Course Q&A TensorFlow: Advanced Techniques Specialization Custom Models, Layers and Loss Functions with TF
1

Greetings Team,

Happy Monday!

Please help me figuring out why I am getting this error message when my assessment is being graded:

[ValidateApp | INFO] Validating ‘/home/jovyan/work/submitted/courseraLearner/W3_Assignment/C1W3_Assignment.ipynb’
[ValidateApp | INFO] Executing notebook with kernel: python3
2023-07-10 20:09:57.155398: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library ‘libnvinfer.so.6’; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2023-07-10 20:09:57.155531: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library ‘libnvinfer_plugin.so.6’; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2023-07-10 20:09:57.155544: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-07-10 20:09:58.237487: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library ‘libcuda.so.1’; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2023-07-10 20:09:58.237520: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
2023-07-10 20:09:58.237559: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ip-10-2-123-145.ec2.internal): /proc/driver/nvidia/version does not exist
2023-07-10 20:09:58.237933: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2023-07-10 20:09:58.269038: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2999995000 Hz
2023-07-10 20:09:58.271397: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55ca0542eb20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-07-10 20:09:58.271436: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
[ValidateApp | ERROR] Traceback (most recent call last):
File “/opt/conda/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py”, line 589, in run_cell
msg = self.kc.iopub_channel.get_msg(timeout=timeout)
File “/opt/conda/lib/python3.7/site-packages/jupyter_client/blocking/channels.py”, line 57, in get_msg
raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/nbgrader/apps/validateapp.py", line 72, in start
    validator.validate_and_print(filename)
  File "/opt/conda/lib/python3.7/site-packages/nbgrader/validator.py", line 340, in validate_and_print
    results = self.validate(filename)
  File "/opt/conda/lib/python3.7/site-packages/nbgrader/validator.py", line 311, in validate
    nb = self._preprocess(nb)
  File "/opt/conda/lib/python3.7/site-packages/nbgrader/validator.py", line 290, in _preprocess
    nb, resources = pp.preprocess(nb, resources)
  File "/opt/conda/lib/python3.7/site-packages/nbgrader/preprocessors/execute.py", line 41, in preprocess
    output = super(Execute, self).preprocess(nb, resources)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 438, in preprocess_cell
    reply, outputs = self.run_cell(cell, cell_index, store_history)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 597, in run_cell
    raise TimeoutError("Timeout waiting for IOPub output")
TimeoutError: Timeout waiting for IOPub output

[ValidateApp | ERROR] nbgrader encountered a fatal error while trying to validate ‘submitted/courseraLearner/W3_Assignment/C1W3_Assignment.ipynb’

Thanks in advance,

2

Hi @Omar_Said_Brito_Sala ,

Really difficult to figure which problem are you having. It seems like the grader is having some kind of timeout. Or that can’t access to the GPU.

The messages related to TensorFlow and CUDA suggest that the code in your notebook is trying to use GPU resources that are not available.

How much time need your notebook to finish in your environment?

Maybe is just a temporal problem with the Cousera environment.

3

Greetings Pere,

It takes up to 75 seconds in total.

I reran it today and it is still throwing that error at me. There is a weird timeout in the grader.

Do you have suggestions?

4

Hi Omar,

75 seconds is not a problem, it can’t be a timeout problem.

Just be sure that you are using a tf verskion compatible with the grader and colab.
with this lines at the begginig of your notebook:
!pip install tensorflow==2.9.0
!pip install keras==2.9.0

Are you using your own environment to create the model, or are using colab?

5

Did you change any of the reserved cells in your notebook and modified, that’s when grader became confused that is for your first error log.

For the second error log, you can refer this [Week 3] Image Segmentation Grading Issue - #2 by ravenJB

as it has similar error log issue, see that if helps. Let us know if you are still running the same issue.

Thank you
DP

7

Did you comment the last cell, that needs to be commented?

8

IMPORTANT

Before submitting, please make sure to follow these steps to avoid getting timeout issues with the grader:

  1. Make sure to pass all public tests and get an accuracy greater than 97%.
  2. Click inside the training code cell above.
    3. Select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to comment the entire block. All lines should turn green as before.
  3. From the menu bar, click File > Save and Checkpoint. This is important.
  4. Once saved, click the Submit Assignment button.

As given in instructions at the last of assignment Comment the code in training cell it is causing problem for grader.

1 Like
9

Hello,

Please do not create multiple comments on multiple threads, rather make a fresh/new post with the concern or error you are getting with your assignment work. Choose the correct specialisation, course and week with assignment name. If possible like in your case you are having a grading issue, so take a screenshot of the error or grader output for the mentor to have better understanding to be able to help you out. Creating posts under old posts would create more confusion as one does not know if the old error header matches with your concern.

Regards
DP

10

That solution was related to same grader output because I was face same error. Thats why I posted this solution if someone else is in seek. By the way thanks for suggestions.