Generators to split mini-batches

I am studying python in more depth in parallel to DLS and recently I came across generators. If I understood it right I can use it to create iterators in a way that the data contained inside of it is not calculated right away, but instead evaluated just when called by some other part of the program. Is that mechanism used in DL algorithms? I thought it could help when splitting the mini-batches, for example, but don’t know if it would make any difference since it is only one step of the whole process. Anyway, I was wondering if it is commonly used for huge dataset or other situations in machine learning. tks

Yes, you could reimplement the minibatch logic here using generators, but they are trying to keep things relatively simple in terms of the level of python sophistication that is required. Later in Courses 4 and 5, you’ll see that TF/Keras support batches, which are effectively generators for dealing with stepping through a large dataset.

1 Like

In fact you can sort of tell that the course staff was just learning python when they created these courses. They’d been doing everything in MATLAB up to that point. E.g. notice the way they defined the APIs in Week 4 of Course 1 to pass the string name of the activation functions and then use a “case” statement down in the subroutine to call the appropriate target routine. You can pass function object references as parameters in python.

And here in the minibatch logic in C2 W2 A1, you really don’t need a whole separate set of logic to deal the case that there’s a last partial minibatch. You can index off the end of an array in python if you are using a “range” specifier and it just does the “Right Thing ™” and truncates to the actual size.

1 Like