Run the Labs on A100 GPU

Course Q&A Generative AI with Large Language Models GenAI with LLMs Resources
1

I have completed the course and labs successfully.

The lab environment in AWS was a CPU machine. I have access to an A100 GPU and wanted to run it on that.

When I run the code from the notebook on my A100 it is still running on CPU. Can anyone please advice on how to make it run on GPU ?

Also, if I want to make use of the accelerate library by HugginFace, then how should I modify the notebook ?

2

Hi Amit,

Maybe a long shot but have you tried to check CUDA availability in your environment?
I think to utilize the amazing hardware capabilities of an A100 GPU, you need Nvidia’s CUDA env. and if you use pytorch you can check whether the GPU is ready :

torch.cuda.is_available()
(returns a boolean)

also you can check is the env. ready:
torch.cuda.is_initialized()

I have struggled with a less fancy, but CUDA enabled Nvidia card, similar issue, card is present but CPU is being used - I had to reinstall the CUDA drivers, reload pytorch and the card responded after.

Now, before I do something I run this little code below and check the outcome:

import torch
print(f’is CUDA available: {torch.cuda.is_available()}‘)
print(f’number of GPUs: {torch.cuda.device_count()}’)
print(f’device capability: {torch.cuda.get_device_capability()}‘)
print(f’device name: {torch.cuda.get_device_name()}’)
print(f’current device properties: {torch.cuda.get_device_properties(torch.cuda.current_device())}‘)
print(f’is CUDA initialized: {torch.cuda.is_initialized()}’)

I hope it helps.

Cheers,
Gabor

1 Like
3

If the instructions from @Gabor_Akacz give an error, probably you need to first set-up properly the proper CUDA driver.

4

thanks for your comment, I assumed the CUDA drivers are all in place, should have been mentioning to install/update them. Good call @Nydia !

1 Like