Stochastic gradient decent?

[Wz111]

Is stochastic gradient decent useful (in engineering projects or research)…? Is the research in this area hot…?

Thanks!!!

[javier]

Hi @Wz111 ,

Stochastic gradient descent (mini-batch of size=1) is not much used in real scenarios as you loose the optimization benefits of vectorization.

What is more frequently used are mini-batches with a carefully selected size to maximize the performance gains in your particular hardware.

We have discussed a little about this topic in this other post: What is the main benefit of minibatch size?

[Wz111]

Thanks… One additional question… Is stochastic approach/random process or the relevant useful in deep learning?