Is the model training on the CPU or GPU?

Course Q&A Generative AI with Large Language Models
1

In the notebook, i don’t see any use of Sagemaker sdk apis. is the model training on CPUs

Harish

2

Such a model should be normally trained on a GPU, especially if it involves large matrices.

3

The ml.m5.2xlarge instance used in the Sagemaker labs does NOT have a GPU, so the labs run on CPU. Only the c instances have GPUs. Presumably, they used GPUs to do teh real tuning.

See Available Studio Instance Types - Amazon SageMaker

1 Like
4

So, there’s no material available to download and run the examples locally on a GPU? How should I set everything up to run on a GPU? This is a core aspect of LLM training. I understand they provided access to an AWS CPU machine, but students need to be able to run this on their own GPU—or at the very least, learn how to set it up properly.

5

DL.AI’s courses cover machine learning methods, demonstrated on a standardized platform.

They don’t cover how to run these methods locally.

The individual technical support required for that would exceed their capability to provide. There are simply too many platforms and operating systems and toolsets to be able to cover everyone’s needs.

1 Like