Stochastic gradient decent?


Is stochastic gradient decent useful (in engineering projects or research)…? Is the research in this area hot…?

Thanks!!!

Hi @Wz111 ,

Stochastic gradient descent (mini-batch of size=1) is not much used in real scenarios as you loose the optimization benefits of vectorization.

What is more frequently used are mini-batches with a carefully selected size to maximize the performance gains in your particular hardware.

We have discussed a little about this topic in this other post: What is the main benefit of minibatch size?

1 Like

Thanks… One additional question… Is stochastic approach/random process or the relevant useful in deep learning?