# Advanced Computer Vision with TensorFlow C3W1\_Assignment taking really long to train

**URL:** <https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739>\
**Category:** Advanced Computer Vision with TensorFlow\
**Created:** [December 21, 2023, 3:56am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739 "2023-12-21T03:56:54Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sha\_Hossain](https://avatars.discourse-cdn.com/v4/letter/s/977dab/32.png) [@Sha\_Hossain](https://community.deeplearning.ai/u/Sha_Hossain)\
**Post date:** [December 21, 2023, 3:56am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/1 "2023-12-21T03:56:54Z")

</div>

This post is regarding week 1 assignment on Predicting Bounding Boxes.  
I finished the assignment and the training starts as expected and i can see the loss going down with each epoch. But each epoch is taking 10 minutes. And the assignment asks us to produce a model with 50 epoch run! I am not being able to generate a final model and submit.

Did anyone else having similar issue?

---

<div class="post-metadata">

**Author:** ![Deepti\_Prasad](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/deepti_prasad/32/165831_2.png) [@Deepti\_Prasad](https://community.deeplearning.ai/u/Deepti_Prasad)\
**Post date:** [December 21, 2023, 6:18am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/2 "2023-12-21T06:18:00Z")

</div>

You still didn’t tell why you are not able to submit your model. Kindly go back and check each steps where you could make any change for better training

---

<div class="post-metadata">

**Author:** ![Sha\_Hossain](https://avatars.discourse-cdn.com/v4/letter/s/977dab/32.png) [@Sha\_Hossain](https://community.deeplearning.ai/u/Sha_Hossain)\
**Post date:** [December 21, 2023, 6:39am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/3 "2023-12-21T06:39:09Z")

</div>

Sorry about the confusion. Since each epoch is taking 10 minutes, 50 epoch run will require 500 minutes that is ~8 hours. Is that expected? Or something is broken?  
Happy to share my colab if that can be useful for addressing the issue.

Thanks!

---

<div class="post-metadata">

**Author:** ![Deepti\_Prasad](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/deepti_prasad/32/165831_2.png) [@Deepti\_Prasad](https://community.deeplearning.ai/u/Deepti_Prasad)\
**Post date:** [December 21, 2023, 7:15am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/4 "2023-12-21T07:15:03Z")

</div>

is your runtime connected to GPU??

---

<div class="post-metadata">

**Author:** ![Sha\_Hossain](https://avatars.discourse-cdn.com/v4/letter/s/977dab/32.png) [@Sha\_Hossain](https://community.deeplearning.ai/u/Sha_Hossain)\
**Post date:** [December 21, 2023, 8:25pm UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/5 "2023-12-21T20:25:03Z")

</div>

![Screenshot from 2023-12-21 12-24-07](https://global.discourse-cdn.com/dlai/original/3X/5/2/522143681d923c90b80773d78751e9cff6db272a.jpeg)

Yes using GPU.

---

<div class="post-metadata">

**Author:** ![Deepti\_Prasad](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/deepti_prasad/32/165831_2.png) [@Deepti\_Prasad](https://community.deeplearning.ai/u/Deepti_Prasad)\
**Post date:** [December 21, 2023, 8:40pm UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/6 "2023-12-21T20:40:18Z")

</div>

Can I know why you are doing assignment on copy saved ? I can see in the screenshot you have two browsers opened with the same assignment?

The reason I am asking this is, the browser shown here shows none of cell were run down, so if this is not the original assignment which you have done, then kindly check the assignment where you are having long training time issue for runtime connection.

Next training time is dependent of various factors your original data, preprocessing data, model algorithm, compile model. Check if you have used the correct optimizer as per instructions given. These general pointers to reduce your training time but getting the desired accuracy and loss.

If all of these has been correctly addressed and still you are having the same issue, then send the download notebook ipnyb via personal DM. Click on my name and then message.

Regards  
DP

---

<div class="post-metadata">

**Author:** ![Sha\_Hossain](https://avatars.discourse-cdn.com/v4/letter/s/977dab/32.png) [@Sha\_Hossain](https://community.deeplearning.ai/u/Sha_Hossain)\
**Post date:** [December 22, 2023, 4:40am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/7 "2023-12-22T04:40:50Z")

</div>

The assignment specially asks for creating a copy, running it and then uploading the model.

I am double checking all the exercises. Will reach out if nothing else is found. Thanks!

 ![Screenshot from 2023-12-21 20-38-14](https://global.discourse-cdn.com/dlai/original/3X/d/1/d1725bb0f332979e4e731d78d88363cb51300e34.png)

---

<div class="post-metadata">

**Author:** ![Deepti\_Prasad](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/deepti_prasad/32/165831_2.png) [@Deepti\_Prasad](https://community.deeplearning.ai/u/Deepti_Prasad)\
**Post date:** [December 22, 2023, 8:10am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/8 "2023-12-22T08:10:04Z")

</div>

Hello @Drew_Murray

in your codes, correction required

1. define the TensorFlow Keras model using the inputs and outputs to your model  
for this you used tf.keras.models.Model which should tf.keras.Model

2. This is not error but your batch size is too big creating the training time. Try reducing it. In that exercise, in instructions it mentions you to refer ungraded lab for hints. as told you in my other previous comment keeping model simple and batch size in relation to the dataset will help get better result as for this assignment training is again dependent of steps per epoch which is dependent of batch\_size. I cannot give you direct answer for this, try grabbing the hints 🙂

3. In the same exercise 6, where you define length of validation dataset with percent of batch\_size for validation steps per epoch, you need to do same for the training set too. Most of the learners are missing this important pointer for this assignment.

Do these corrections, let me know if your issue is resolved.

@Sha_Hossain, check if your point 2 and 3 I mentioned has been done accordingly.

Regards  
DP

---

<div class="post-metadata">

**Author:** ![Sha\_Hossain](https://avatars.discourse-cdn.com/v4/letter/s/977dab/32.png) [@Sha\_Hossain](https://community.deeplearning.ai/u/Sha_Hossain)\
**Post date:** [January 8, 2024, 3:16am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/9 "2024-01-08T03:16:34Z")

</div>

For posterity, the key issue is discussed here,

> [@TF-AT: Using the Fallback Runtime](https://community.deeplearning.ai/t/tf-at-using-the-fallback-runtime-to-avoid-slow-training-in-colab/530596):
>
> Google Colab updates its runtime periodically to use newer versions of packages like Tensorflow. This can sometimes affect the use of GPUs for notebooks that require older versions, or cause problems in installing required packages. To get around it, please use the fallback runtime before running any of the code in your notebook. Here are the steps: Make sure that you are already connected to a runtime (CPU or GPU, depending on the default used in the assignment). You can click on that on the…

Also the assignment now has this note, **NOTE (12/30/2023):** **A recent update in Google Colab disables the use of GPUs for the required Tensorflow version in this course. With that, you might notice a slow training time _per epoch_ (e.g. 5 minutes per epoch). While we update the grader, please use the fallback runtime instead so you can use the GPU and have faster training**

---

<div class="post-metadata">

**Author:** ![Deepti\_Prasad](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/deepti_prasad/32/165831_2.png) [@Deepti\_Prasad](https://community.deeplearning.ai/u/Deepti_Prasad)\
**Post date:** [January 8, 2024, 6:49am UTC](https://community.deeplearning.ai/t/advanced-computer-vision-with-tensorflow-c3w1-assignment-taking-really-long-to-train/523739/10 "2024-01-08T06:49:01Z")

</div>

Yes this issue is currently going on with tensorflow advanced specialised course especially for course 3 and course 4. So if anyone is getting stuck, one should try fallback runtime solution as this issue is occurring because of version and update mismatch.

Regards  
DP
