Lesson 3: functions init_batch, merge_batches, filter_batch used to implemented continuous batching

Short Course Q&A Efficiently Serving LLMs
1

Could someone let me know where to find the scripts of the following three functions: init_batch, merge_batches, and filter_batch? These three functions are used in implementing continuous batching in Lesson 3. I could not find them in the Jupyter Notebooks of Lesson 1 or Lesson 2.

Thanks!

2

Try looking in the File menu in the course notebook.

3

@TMosh , thanks! Could you point me to the URL of the official course notebooks? I can not find them at deeplearning.ai or Coursera. Thanks again.

4

The notebooks are found within each course.

5

For example:

image
image1869×955 126 KB