C1 W4: Why is b in cache?

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

The vector b doesn’t appear in the back propagation equations (as shown in the ‘Forward and backward propagation’ video). Why is it stored in the linear_cache?

1 Like
2

That’s a good point, but I don’t know the definitive answer. I think there are two possibilities:

  1. They didn’t notice that it wasn’t used in back prop.
  2. They did notice but figured it looked more consistent to include everything. That is to say, if they didn’t include it, then they felt they’d need to explain why and didn’t think it was worth it.
1 Like
3

Thanks for the answer!

4

Hi Weeshun,

To add further what Paul sir has mentioned, your quest on cache is quite interesting. We all know this very well know what cache is all about, right! During each forward function, a corresponding reverse function happens, which is very obvious. So, at each step of this forward pass, some kind of values are stored in the form of cache. These values are used for calculating the gradients during back propagation. If you want to have some depth on vectors like (weights and biases) and how the linearity changes to non-linearity and what made such changes, read this paper here. You will find it useful in many ways and will clear most of your queries on the application of linear and non-linear functions.

5

Hi Rashmi,

Thanks for the comments. But I understand why arrays and vectors are stored in cache. That’s not my question. The lecture and programming assignment explain that we save A, W, and b in cache (linear_cache in the programming assignments). My question was why b is stored. If you look at the back propagation formulas, you can see that we need to know A and W to compute the derivative terms. But b is not required for the calculation. We will need to know b when updating parameters, but they are extracted from the parameters variable in the programming assignments.