Calculated loss must be on the original device: cuda:0 but device in use is cpu

Short Course Q&A Finetuning Large Language Models
1

Hi, I am trying to train GPT2 on my local machine using the course code provided in the Lamini course.

Request help for the following error:

Error: Calculated loss must be on the original device: cuda:0 but device in use is cpu

Full error message:

image
image736×467 24.9 KB

The trainer stops at batch 289:

image
image786×354 18.3 KB

Using:

torch=2.4.1+cu124=pypi_0
lamini=3.1.3=pypi_0
transformers=4.45.1=pypi_0

I have tried the following without success:

  • Adding this code to utilities.py, super(Trainer, self).init
self.device = torch.device('cuda')
self.model = self.model.to(device)
self.total_loss = torch.Tensor([0]).to(device)
  • Modifying these lines to add device:
    with self.compute_loss_context_manager():
    loss = self.compute_loss(model, inputs).to(‘cuda’)
2

Solved: I switched to a higher version of transformers, and that resolved the issue.