In labs with Coffee Roasting (C2_W1_Lab02_CoffeeRoasting_TF and C2_W1_Lab03_CoffeeRoasting_Numpy), the TF implementation of prediction (model.predict function) is much slower than the custom implementation using Numpy (my_predict). This is clearly visible in the last cells of both labs, when generating final graphs - the TF version takes 69 seconds, while the Numpy one takes 1 second on my computer.
Why is that?
It might be because numpy package is optimized for matrix operations better than just tensorflow tensors!
The “Coffee roasting using numpy” lab doesn’t do any training, it uses pre-computed weights (which came from the TensorFlow solution) and only does predictions.
When TensorFlow did the NN training, it took over 6000 iterations. The numpy lab wasn’t asked to do that level of work.
I was running only the last step with charts, that had model.predict in the TF version - I thought by that time the model is already trained by previous model.fit step.
Here, to make it easier I removed the plotting code and after all trainings I left only 1 call to model.predict and my_predict and tried to measure their times using same test data and using %timeit.
Numpy version - 1 millisecond:
TF version - 50 milliseconds (50 times more!):
Found this discussion on Stackoverflow - python - TF.Keras model.predict is slower than straight Numpy? - Stack Overflow
Seems the problem appears for some datasets and models because of TF’s “eager mode” (whatever that means).
Thank you all, I thought there is some more obvious reason for that.
Here is some information on eager mode:
@TMosh thanks a lot! Now everything seems to be clear.
Also I thought it shouldn’t be a big deal - we can train a model in TF, and build the same model in Numpy with weights from TF for prediction (at least for feed-forward networks, didn’t get to more complex one yet).